From Virtualization to Cloud Native

In recent years, Cloud Native has become one of the hottest concepts in the IT industry, with many internet giants actively embracing it. When it comes to Cloud Native, we must understand the protagonist of this article – containers. Container technology has become a crucial part of the Cloud Native ecosystem. As an advanced virtualization technology, containers have become the standard infrastructure for software development and operations in the Cloud Native era. Before we delve into containers, let’s start with virtualization technology.

What Is Virtualization Technology

1961 – IBM709 implemented time-sharing systems

The first implementation of virtualization technology in computer history occurred in 1961, when the IBM709 computer divided CPU usage into multiple very short (1/100 sec) time slices, with each time slice executing different tasks. By polling these time slices, a single CPU could be virtualized or disguised as multiple CPUs, making each virtual CPU appear to be running simultaneously. This is the prototype of the virtual machine.

The functions of containers are similar to those of virtual machines. Both containers and virtual machines virtualize at different levels of the computer, using logic to represent resources, thus breaking free from the constraints of physical limitations and improving the utilization of physical resources. Virtualization technology is an abstract and rich concept that has different meanings in various fields or levels.

Here, let’s roughly discuss the hierarchical structure of computers. For most software developers, computer systems can be divided into the following hierarchical structure:

Application Layer
Library Layer
Operating System Layer
Hardware Layer

Each layer provides an interface upwards, while each layer only needs to know the interface of the next layer to call lower-level functions to achieve upper-level operations (without needing to know the specific operational mechanisms of the lower levels).

However, due to early computer manufacturers producing hardware according to their own standards and specifications, the compatibility of operating systems across different computer hardware was poor; similarly, different software also had poor compatibility across different operating systems. Thus, developers artificially created an abstraction layer between the layers:

Application Layer
Library Layer
	API Abstraction Layer
Operating System Layer
	Hardware Abstraction Layer
Hardware Layer

In the context we are discussing, virtualization is the creation of a new abstraction layer between the upper and lower layers, allowing upper-level software to run directly in a new virtual environment. In simple terms, virtualization is the creation of interfaces that mimic the original functional modules of the lower layer to “deceive” the upper layer, thus achieving cross-platform development.

Based on the above concepts, we can re-understand several well-known virtualization technologies today:

Virtual Machines: Virtualization technology existing between the hardware layer and the operating system layer.

Virtual machines “forge” a hardware abstraction interface, allowing an operating system and layers above it to be grafted onto the hardware, achieving functionality almost identical to that of a real physical machine. For example, when using an Android virtual machine on a Windows system computer, we can open applications on the Android system using this computer.

Containers: Virtualization technology existing between the operating system layer and the library layer.

Containers “forge” the operating system’s interface, placing functionalities above the library layer on top of the operating system. Taking Docker as an example, it is an isolation container based on the Linux operating system’s Namespace and Cgroup features, capable of simulating operating system functionalities. In simple terms, if a virtual machine encapsulates and isolates the entire operating system for cross-platform applications, then containers encapsulate and isolate individual applications for cross-platform use. Therefore, containers are much smaller than virtual machines and theoretically consume fewer resources.

JVM: Virtualization technology existing between the library layer and the application layer.

The Java Virtual Machine also has cross-platform characteristics, which are essentially due to virtualization. We know that the Java language calls the operating system’s library functions, and the JVM establishes an abstraction layer between the application layer and the library layer, adapting to different operating system library functions through different versions and providing a unified runtime environment for programs and developers, allowing developers to call different operating system library functions.

After roughly understanding virtualization technology, we can now explore the history of the emergence of containers. Although the concept of containers gained global popularity only after the emergence of Docker, countless pioneers had already been exploring this forward-looking virtualization technology before Docker.

The Predecessor of Containers: “Jail”

1979 – Bell Labs invented chroot

One of the main features of containers is process isolation. As early as 1979, during the development of Unix V7, Bell Labs discovered that once a system software was compiled and installed, the variables of the entire testing environment would change. To conduct the next build, installation, and testing, it was necessary to rebuild and configure the testing environment. At that time, a 64K memory stick cost $419, making the cost of “rapidly destroying and rebuilding infrastructure” prohibitively high.

Developers began to ponder whether it was possible to isolate an independent environment for reconstructing and testing software within the existing operating system environment. Thus, a system call function called chroot (Change Root) was born.

chroot can redirect the root directory of a process and its child processes to a new location in the file system, meaning it can separate each process’s file access permissions, preventing it from accessing external files. Thus, this isolated new environment was aptly named Chroot Jail. By copying the necessary system files into the Chroot Jail, software reconstruction and testing could be achieved. This advancement opened the door to process isolation, providing a simple system isolation function for Unix, especially the idea of jail laid the foundation for the development of container technology. However, at this time, the isolation function of chroot was limited to the file system, and processes and network space were not adequately addressed.

Entering the 21st century, VM (Virtual Machine) technology had matured, allowing cross-operating system development through virtualization technology. However, since VMs needed to encapsulate and isolate entire operating systems, they consumed significant resources and seemed too cumbersome for production environments. Thus, people began to pursue a lighter virtualization technology, leading to the emergence of various process isolation technologies based on chroot.

2000 – FreeBSD launched FreeBSD Jail

Twenty-one years after the birth of chroot, FreeBSD version 4.0 introduced a micro-host environment sharing system called FreeBSD Jail, which expanded upon the existing chroot mechanism. In FreeBSD Jail, programs not only have their own file systems but also independent process and network spaces. Processes in the Jail cannot access or see files, processes, or network resources outside the Jail.

2001 – Linux VServer was born

In 2001, the Linux kernel added Linux VServer (Virtual Server), providing virtualization capabilities for Linux systems. Linux VServer also adopts a jail mechanism, capable of partitioning file systems, network addresses, and memory on the computer system, allowing multiple virtual units to run simultaneously.

2004 – SUN released Solaris Containers

This technology was also further developed from chroot. In February 2004, SUN released the beta version of Solaris 10, which added operating system virtualization capabilities called Containers and improved it in the subsequent official version of Solaris 10. Solaris Containers support both x86 and SPARC systems, and SUN created a zone function to work with Containers, where the former is a completely isolated virtual server within a single operating system, achieving process isolation through the boundary separation provided by system resources and zones.

2005 – OpenVZ emerged

Similar to Solaris Containers, it provides virtualization, isolation, resource management, and checkpointing through patches to the Linux kernel. Each OpenVZ container has a set of isolated file systems, users and user groups, process trees, networks, devices, and IPC objects.

The process isolation technologies of this period mainly focused on the Jail model, achieving isolation of process-related resources. However, due to the lack of corresponding use cases in production development, this technology remained limited to a niche and limited world.

At this time, a new technology called “cloud” was quietly emerging…

The Birth of “Cloud”

Between 2003 and 2006, Google published three product design papers, pioneering the concept of distributed computing architecture, laying the foundation for big data computing technology. Based on this, Google proposed the disruptive “Google 101” plan and officially created the concept of “cloud.” Suddenly, terms like “cloud computing” and “cloud storage” became a global sensation. Subsequently, industry giants like Amazon and IBM also announced their respective “cloud” plans, marking the arrival of the cloud technology era.

Since this period, process isolation technology entered a more advanced stage. Under the cloud computing framework proposed by Google, isolated processes were no longer just a Jail isolated from the outside world but needed to function like lightweight containers, capable of being controlled and allocated to achieve cross-platform, high availability, and scalability in distributed application scenarios.

2006 – Google introduced Process Containers, later renamed Cgroups

Process Container was the prototype of “container” technology in the eyes of Google engineers, used to limit, account for, and isolate resources (CPU, memory, disk I/O, network, etc.) for a group of processes. This aligns with the goals of the previously mentioned process isolation technology. Due to technological maturity, Process Container was officially launched in 2006 and entered the Linux kernel mainline the following year, officially renamed Cgroups, marking a reevaluation and realization of the concept of “containers” in the Linux camp.

2008 – Linux container tool LXC was born

In 2008, a complete container technology called LXC (Linux Container) appeared in the Linux kernel by combining the resource management capabilities of Cgroups and the view isolation capabilities of Linux Namespace. We know that a process can call all resources on its physical machine, which can monopolize the available resources of other processes. To limit this situation, Linux kernel developers provided a feature where processes running in a Cgroup are similar to those in a namespace, but Cgroups can limit the resources available to that process. Although LXC provides capabilities to users that are very similar to earlier Linux sandbox technologies like Jails and OpenVZ, with various Linux distributions rapidly occupying the commercial server market, many cloud computing pioneers, including Google, were able to fully utilize this early container technology, allowing LXC to gain far more development space in the cloud computing field than its predecessors.

That year, Google launched the first application hosting platform GAE (Google App Engine) based on LXC, the first to provide a development platform as a service. GAE is a distributed platform service that provides users with development environments, server platforms, hardware resources, and more through virtualization technology. Users can customize their applications on this platform and distribute them using Google’s servers and internet resources, greatly reducing the hardware requirements for users.

It is worth mentioning that Google used a tool capable of orchestrating and scheduling LXC in GAE – Borg (the predecessor of Kubernetes). Borg is a large-scale cluster management system used internally by Google, capable of handling hundreds of thousands of tasks, thousands of different applications, while managing tens of thousands of machines. Borg achieves high resource utilization through permission management, resource sharing, and performance isolation. It can support highly available applications and reduce the probability of failures through scheduling policies, providing task description languages, real-time task monitoring, analysis tools, etc. If isolated containers are like shipping containers, then Borg can be seen as the earliest port system, and LXC + Borg is the earliest container orchestration framework. At this point, containers are no longer just a simple process isolation function but a flexible and lightweight program packaging model.

2011 – Cloud Foundry launched Warden

Cloud Foundry is a well-known cloud service provider VMware’s cloud platform launched in 2009 and is the first project in the industry to formally define the PaaS (Platform as a Service) model. “PaaS projects allow developers to focus on business logic rather than infrastructure through direct management, orchestration, and scheduling of applications” and “PaaS projects use container technology to package and launch applications” are concepts originating from Cloud Foundry. Warden is the resource management container at the core of Cloud Foundry, initially encapsulating LXC, later restructured into an architecture that directly operates on Cgroups and Linux Namespace.

With the continuous expansion of the “cloud” service market, various PaaS projects emerged, and container technology ushered in a period of explosive growth, with a large number of entrepreneurial projects focusing on container technology springing up. Of course, many know the later story, a startup called Docker emerged, making Docker almost synonymous with “containers.”

Docker Emerges

2013 – Docker was born

Docker was initially an internal project of a PaaS service company called dotCloud, which later changed its name to Docker. In its early stages, Docker was similar to Warden, using LXC, but later began to adopt its own developed libcontainer to replace LXC. Unlike other projects that only focused on containers, Docker introduced a complete ecosystem for managing containers, including an efficient, layered container image model, global and local container registries, clear REST APIs, command lines, and more.

Docker itself is also a type of encapsulation of LXC, providing a simple and user-friendly interface for container use. Its biggest feature is the introduction of container images. Docker packages applications and the environments needed to run them into a single file through container images. Running this file generates a virtual container.

More importantly, the Docker project adopted the idea of Git – introducing the concept of “layers” in the creation of container images. Based on different “layers,” containers can include different information, allowing for version management, copying, sharing, and modification, just like managing ordinary code. By creating Docker images, developers can distribute software directly through image hosting repositories like DockerHub.

In other words, the emergence of Docker not only solved the containerization problem at the software development level but also addressed the software distribution phase, providing a complete solution for the software lifecycle process in the “cloud” era.

Soon, Docker gained widespread recognition in the industry, being chosen by many companies as the standard for cloud computing infrastructure construction, and containerization technology became the hottest frontier technology in the industry, with the ecosystem surrounding containers rapidly taking shape.

The Container Wars

The rise of a new technology also brings a new market, and a blue ocean competition over containers is inevitable.

2013 – CoreOS was released

After the explosive popularity of Docker, CoreOS emerged at the end of the same year. CoreOS is a lightweight operating system based on the Linux kernel, designed for the infrastructure construction of computer clusters in the cloud computing era, featuring automation, easy deployment, security, reliability, and scalability. At that time, it had a very prominent label: an operating system designed for containers.

Riding the wave of Docker, CoreOS quickly rose in the cloud computing field, becoming a golden partner for container deployment along with Docker. At the same time, CoreOS made significant contributions to Docker’s promotion and community building.

However, the increasingly powerful Docker seemed to have greater “ambitions.” Not satisfied with just being “a simple basic unit,” Docker independently developed a series of related container components and acquired some containerization technology companies, starting to build its own container ecosystem. This clearly formed a direct competitive relationship with CoreOS.

2014 – CoreOS released the open-source container engine Rocket

At the end of 2014, CoreOS launched its own container engine, Rocket (rkt for short), aiming to compete with Docker. rkt, like Docker, can help developers package applications and dependencies into portable containers, simplifying deployment tasks. The difference between rkt and Docker is that rkt lacks the “friendly features” provided to enterprise users by Docker, such as cloud service acceleration tools and cluster systems. In other words, rkt aims to establish a more pure industry standard.

2014 – Google launched the open-source container orchestration engine Kubernetes

To address the issues of large-scale cluster container deployment and management in hybrid cloud scenarios, Google launched Kubernetes (K8S for short) in June 2014, a container cluster management system. K8S is derived from the previously mentioned Borg, and it has the functionality to manage and orchestrate containers in production environments within hybrid cloud scenarios. Kubernetes introduced the Pod feature, allowing different containers to communicate with each other, facilitating container grouping and allocation.

Thanks to Google’s strong accumulation in large-scale cluster infrastructure construction, K8S, which evolved from Borg, quickly became the industry standard application, regarded as an essential tool for container orchestration. As a significant player in the container ecosystem, Google sided with CoreOS in the container competition between Docker and rkt, marking K8S support for rkt as an important milestone.

2015 – Docker launched the container cluster management tool Docker Swarm

In response, Docker also began to include a container cluster management tool, Docker Swarm, in the 2015 release of Docker 1.12.

Subsequently, Google led a $12 million investment in CoreOS in April 2015 and collaborated with CoreOS to release the first enterprise edition of Kubernetes – Tectonic. From then on, the container landscape was divided into two major camps: the Google faction and the Docker faction.

The competition between the two factions intensified, gradually extending to the struggle for establishing industry standards.

2015 June – Docker took the lead in establishing OCI

Docker, in collaboration with the Linux Foundation, established the OCI (Open Container Initiative) organization, aimed at “formally defining and maintaining specifications for container image formats and container runtimes (“OCI Specifications”), creating an open industrial standard around container formats and runtimes.

July 2015 – Google took the lead in establishing CNCF

Meanwhile, Google, focusing on the “cloud” strategy, also collaborated with the Linux Foundation to establish the CNCF (Cloud Native Computing Foundation) in July of the same year, designating Kubernetes as the first open-source project included in the CNCF management system, aiming to “build cloud-native computing – an infrastructure-centered architecture revolving around microservices, containers, and dynamic application scheduling, and promote its widespread use.”

These two open-source foundations established around container-related open-source projects played a crucial role in promoting the future development of cloud-native technology, complementing each other and creating a series of industry de facto standards, becoming the most active open-source organizations today.

Kubernetes Ecosystem Unifies the Landscape

From Virtualization to Cloud Native

Although Docker has consistently outperformed rkt over the years, becoming the undisputed leader in containers, in the context of a large container technology ecosystem, Docker ultimately lost the later container orchestration battle to Google’s Kubernetes.

As more developers began using Docker to deploy containers, the importance of orchestration platforms became increasingly prominent. Following Docker’s rise, a large number of open-source projects and proprietary platforms emerged to address container orchestration issues. Mesos, Docker Swarm, and Kubernetes all provide different abstractions for managing containers. During this period, choosing a container orchestration platform became a gamble for software developers, as if the chosen platform lost in future competition, it would mean that their subsequent developments would lose market opportunities. Just like the battle between smartphone operating systems Android, iOS, and WP, only the victor can gain greater market prospects, while the loser may even disappear.

The container orchestration platform competition thus began.

2016 – CRI-O was born

In 2016, the Kubernetes project launched the CRI (Container Runtime Interface), a plugin interface that allows kubelet (a cluster node agent used to create pods and start containers) to use different OCI-compliant container runtime environments without recompiling Kubernetes. Based on CRI, an open-source project named CRI-O was born, aimed at providing a lightweight runtime environment for Kubernetes.

CRI-O allows developers to run containers directly from Kubernetes, meaning Kubernetes can manage containerized workloads without relying on traditional container engines (like Docker). This way, on the Kubernetes platform, as long as the container complies with OCI standards (not necessarily Docker), CRI-O can run it, allowing containers to return to their most basic function – to encapsulate and run cloud-native programs.

At the same time, the emergence of CRI-O made those managing and operating software with container technology realize that, compared to Docker’s standard container engine, the Kubernetes technology stack (such as orchestration systems, CRI, and CRI-O) is more suitable for managing complex production environments. It can be said that CRI-O has placed container orchestration tools in a significant position within the container technology stack, thereby reducing the importance of container engines.

With K8S successfully seizing the initiative, Docker made mistakes while promoting its container orchestration platform, Docker Swarm. At the end of 2016, rumors surfaced that Docker might change its standards to better adapt to Swarm. This led many developers to prefer Kubernetes, which had stronger market compatibility, when choosing platforms.

As a result, after entering 2017, more vendors were willing to bet on K8S and invest in building K8S-related ecosystems. The container orchestration competition concluded with a victory for the Google camp. Meanwhile, the CNCF, centered around K8S, also began to develop rapidly, becoming the hottest open-source project foundation. In the past two years, Chinese tech companies like Alibaba Cloud, Tencent, and Baidu have also joined CNCF, fully embracing container technology and cloud-native.

Conclusion

From decades of exploration of process isolation functions in laboratories to the current widespread cloud-native infrastructure construction in production environments, container technology has condensed the efforts of several generations of developers, evolving from a small shipping container to a large modern port. It can be anticipated that for a long time to come, container technology will remain an essential infrastructure for software development and operations.