Virtink: A Lightweight Kubernetes Native Virtualization Management Engine

Today, we are excited to announce the Virtink open-source project, a more lightweight Kubernetes native virtualization management engine.

Unlike the KubeVirt project, Virtink does not consider support for emulation of legacy hardware devices and desktop application capabilities. Instead, it focuses on running modern cloud virtualization workloads on Kubernetes. Therefore, Virtink is implemented based on the modern Cloud Hypervisor¹ , which can be installed in Kubernetes on any CPU platform that supports virtualization, providing a more secure and lightweight way to support virtualization workloads.

At the same time, we also released the knest open-source project. Knest is a handy command-line tool that leverages Virtink’s capabilities to easily support Kubernetes-in-Kubernetes scenarios, allowing development and operations teams to create multiple Kubernetes clusters on bare metal without relying on traditional virtualization platforms.

Background

Since the advent of Kubernetes, it has brought revolutionary changes to cloud-native infrastructure. Its powerful distributed scheduling capabilities, simple and user-friendly cluster resource abstraction, and rich third-party extension capabilities have made it the de facto standard for cloud-native platforms, accelerating the migration of more applications and middleware.

However, during the era of cloud computing and earlier virtualization, a large number of applications were based on virtual machines. Although some applications have been modified or even rewritten to adapt to the Kubernetes platform in today’s containerization wave, there are still a considerable number of applications that have not been containerized due to difficulties in transformation or large workloads, thus failing to leverage Kubernetes’ powerful capabilities.

Additionally, some newly developed system-level applications may require a different kernel from the host machine and cannot share the kernel with the host, necessitating virtualization capabilities to run an independent kernel.

This coexistence of containers and virtual machines, which cannot be managed together, increases the difficulty and workload of operations and maintenance, and this issue will become more pronounced over time. Allowing Kubernetes to orchestrate both containers and virtual machines is a feasible solution.

The KubeVirt project, as a pioneer in this field, has attracted significant attention over the years. KubeVirt is more positioned to support the capabilities of traditional virtualization platforms. With the increasing functionalities supported by traditional virtualization platforms, the code complexity has also risen, leading to significant overhead for running each virtual machine.

In contrast, Virtink is fully aimed at modern cloud workloads, combining the latest advancements in virtualization technology to provide users with a more lightweight and secure virtualization management engine tailored for cloud workloads.

KubeVirt Designed for Traditional Virtualization Loads

To better support the capabilities of traditional virtualization platforms, KubeVirt utilizes a combination of traditional libvirt with QEMU as its virtualization software. KubeVirt requires running independent libvirtd and launcher processes within each pod hosting a virtual machine. Observations show that the memory usage of each libvirtd process is generally over 30MB, while each launcher process typically consumes around 80MB, resulting in an average additional memory overhead of over 110MB per virtual machine. When the number of virtual machines reaches a certain level, this memory overhead becomes considerable.

The virtual machine images supported by KubeVirt mainly use QCOW2. The packaging method of QCOW2 is relatively slow and cumbersome, and building a QCOW2 image often takes several to dozens of minutes, which differs significantly from the Dockerfile packaging method for containers.

High operational overhead and slow image packaging determine that KubeVirt is a relatively heavy virtualization component, more suitable for carrying traditional virtualization loads.

Virtink Designed for Modern Cloud Virtualization Loads

We find that the virtualization needs on Kubernetes are typically to run modern operating systems and server-side loads. These loads do not require a desktop control console like traditional virtualization, nor do they need to support emulation capabilities for legacy hardware devices. However, they do require less additional resource overhead, a more secure runtime environment, and faster startup speeds. Virtink is designed specifically for running such modern cloud virtualization loads.

Virtink is an abbreviation for Virtualization in Kubernetes, and its initial design goals are as follows:

How Does Virtink Achieve Lightweight?

Utilizing Lightweight Cloud Hypervisor

Cloud Hypervisor¹ is an open-source KVM virtual machine manager written in Rust, primarily focused on supporting modern cloud loads with minimal hardware emulation. Its main advantages are security and lightweight. By providing minimal hardware emulation, it reduces the attack surface while also lowering memory overhead.

No Additional Resident Process Overhead

Virtink does not require running libvirtd and launcher processes for each VM to manage them, completely eliminating any overhead outside of the VM.

Before a VM runs, Virtink starts a leading process called prerunner to handle network and virtual machine configuration. After completing these configurations, this process exits upon VM startup, thus not occupying memory and CPU for extended periods. The state management for virtual machines is handled by a daemon running on each node. It monitors the running status of the virtual machines through the Cloud Hypervisor’s API socket and issues management commands as necessary.

Supports Using Container Images as Virtual Machine Rootfs

Cloud Hypervisor¹ supports direct kernel boot. By specifying both the kernel and rootfs, it can quickly launch virtual machines. The advantage of this feature lies not only in the startup speed of the virtual machine but also in the ease of building and packaging the rootfs, which only requires a typical root partition and does not need to partition boot areas, UEFI partitions, etc., like typical boot disks.

Based on the direct kernel boot feature, Virtink supports using container images as the rootfs for virtual machines, which allows the packaging and building of rootfs to fully utilize Docker’s toolchain and ecosystem, greatly accelerating and facilitating the construction and release of virtual machines. Below is an example Dockerfile used to build an Ubuntu rootfs.

A Demo

Perfectly Supports Kubernetes in Kubernetes

Currently, mainstream public cloud platforms provide direct support for Kubernetes, making it very easy to create and manage a Kubernetes cluster on these cloud platforms. However, achieving similar capabilities in private data centers requires significant human and material resources. It requires a powerful distributed virtualization platform in the data center and strong support for Kubernetes from this virtualization platform, making the creation and maintenance of Kubernetes clusters straightforward.

The emergence of Virtink provides a simpler and lighter solution to this problem. The virtualization capabilities offered by Virtink can effectively support the workloads of Kubernetes virtual machine clusters, providing Kubernetes as a service capability. Due to the relatively lightweight nature of Virtink, it can offer higher virtual machine running density than KubeVirt, allowing for more Kubernetes nodes to run.

We have simultaneously developed a command-line tool, knest, for creating nested Kubernetes clusters on top of the Kubernetes cluster. With this tool, it is easy to create any number of nested Kubernetes clusters, achieving Kubernetes as a service.

Roadmap

Virtink is currently at version v0.8, supporting the minimum functionality set for running virtualization loads, such as supporting CNI networks and CSI storage, and has support for both x86 and ARM platforms. Features on the roadmap include hot migration, host PCI device (SR-IOV network cards, GPUs, etc.) passthrough, vCPU binding, and hot-plugging virtual machine disks.

Knest is currently at version v0.2, supporting the creation and scaling of nested Kubernetes clusters, and future versions of knest will continue to enhance usability and cluster operation and maintenance capabilities.

The Virtink² and knest³ projects are hosted on GitHub under the open Apache License 2.0, allowing you to use and modify them freely. You are also welcome to provide feedback on issues, request features, and contribute to the code.

We welcome you to scan the QR code below to add the SmartX open-source community administrator on WeChat, join the Virtink community, and discuss Virtink with cloud-native experts and engineers. Alternatively, you can contact us via email ([email protected]) to share your ideas and questions.

Click to read the original text to learn more about the Virtink open-source project。