(Click the WeChat account above to quickly follow)

Source: 宅蓝三木

https://my.oschina.net/kelvinxupt/blog/1602990

Operating System Level Virtualization

Virtualization technologies such as KVM and XEN allow each virtual machine to have its own independent operating system. Unlike these technologies, operating system-level virtualization, also known as containerization, is a feature of the operating system itself that allows multiple isolated user space instances to exist. These user space instances are also known as containers. Ordinary processes can see all the resources of the computer, while processes within a container can only see the resources allocated to that container. In simple terms, operating system-level virtualization groups the computer resources managed by the operating system, including processes, files, devices, networks, etc., and assigns them to different containers for use. Processes running in the container can only see the resources allocated to that container, achieving the purpose of isolation and virtualization.

Implementing operating system virtualization requires the use of Namespace and cgroups technologies.

Namespace

The concept of namespaces in programming languages is introduced to reuse variable names or service routine names without conflict in different namespaces. The Linux system introduces namespaces for similar purposes. For example, in a Linux system without operating system-level virtualization, user-mode processes are numbered starting from 1 (PID). After introducing operating system virtualization, different containers have different PID namespaces, and processes within each container can start numbering from 1 without conflict.

Currently, there are six types of namespaces in Linux, corresponding to six types of resources managed by the operating system:

• Mount point (mount point) CLONE_NEWNS

• Process (pid) CLONE_NEWPID

• Network (net) CLONE_NEWNET

• Inter-process communication (ipc) CLONE_NEWIPC

• Hostname (uts) CLONE_NEWUTS

• User (uid) CLONW_NEWUSER

In the future, namespaces corresponding to time and devices will also be introduced.

The first namespace, the mount point, was introduced in Linux version 2.4.19. At that time, there were no other types of namespaces, so the flag introduced in the clone system call was called CLONE_NEWNS.

Three System Calls Related to Namespaces

The following three system calls are used to operate namespaces:

• clone() — used to create new processes and new namespaces; the new process will be placed in the new namespace

• unshare() — creates a new namespace but does not create a new child process; subsequent child processes will be placed in the newly created namespace

• setns() — adds a process to an existing namespace

Note: These three system calls do not change the calling process’s (calling process) pid namespace, but will affect its child processes’ pid namespaces.

Namespaces themselves do not have names; different namespaces are identified by different inode numbers, which also conforms to the Linux convention of using files for everything. You can check which namespace a process belongs to in the proc file system. For example, to check the namespace of the process with PID 4123:

kelvin@desktop:~$ ls -l /proc/4123/ns/

Total usage 0

lrwxrwxrwx1kelvin kelvin0Dec2616:28cgroup -> cgroup:[4026531835]

lrwxrwxrwx1kelvin kelvin0Dec2616:28ipc -> ipc:[4026531839]

lrwxrwxrwx1kelvin kelvin0Dec2616:28mnt -> mnt:[4026531840]

lrwxrwxrwx1kelvin kelvin0Dec2616:28net -> net:[4026531963]

lrwxrwxrwx1kelvin kelvin0Dec2616:28pid -> pid:[4026531836]

lrwxrwxrwx1kelvin kelvin0Dec2616:28user -> user:[4026531837]

lrwxrwxrwx1kelvin kelvin0Dec2616:28uts -> uts:[4026531838]

Below is a code snippet demonstrating how to use the above three system calls to manipulate process namespaces:

#define _GNU_SOURCE

#include <sys/types.h>

#include <sys/wait.h>

#include <sched.h>

#include <signal.h>

#include <unistd.h>

#include <stdio.h>

#include <string.h>

#include <stdlib.h>

#include <errno.h>

#include <sys/types.h>

#include <sys/stat.h>

#include <fcntl.h>

#define STACK_SIZE (10 * 1024 * 1024)

char child_stack[STACK_SIZE];

int child_main(void* args){

pid_t child_pid = getpid();

printf(“I’m child process and my pid is %d \n”,child_pid);

// The child process will be placed in the new pid namespace created by the clone system call, so its pid should be 1

sleep(300);

// The inode of the namespace will be deleted after all processes in the namespace exit, keeping it for subsequent operations

return0;

}

int main(){

/* Clone */

pid_t child_pid = clone(child_main,child_stack + STACK_SIZE,\

CLONE_NEWPID | SIGCHLD,NULL);

if(child_pid < 0){

perror(“clone failed”);

}

/* Unshare */

intret = unshare(CLONE_NEWPID);// The parent process calls unshare to create a new namespace,

// but does not create a child process. Subsequent child processes will be added to the new namespace

if(ret < 0){

perror(“unshare failed”);

}

intfpid = fork();

if(fpid < 0){

perror(“fork error”);

}elseif(fpid == 0){

printf(“I am child process. My pid is %d \n”,getpid());

// The child process after fork will be added to the namespace created by unshare, so its pid should be 1

exit(0);

}else{

}

waitpid(fpid,NULL,0);

/* Setns */

charpath[80] = “”;

sprintf(path,“/proc/%d/ns/pid”,child_pid);

intfd = open(path,O_RDONLY);

if(fd == –1)

perror(“open error”);

if(setns(fd,0) == –1)

// setns does not change the current process’s namespace, but sets the namespace for subsequently created child processes

perror(“setns error”);

close(fd);

intnpid = fork();

if(npid < 0){

perror(“fork error”);

}elseif(npid == 0){

printf(“I am child process. My pid is %d \n”,getpid());

// The new child process will be added to the pid namespace of the first child process, so its pid should be 2

exit(0);

}else{

}

return0;

}

Execution result:

$sudo./ns

I’mchildprocess andmy pid is1

Iam childprocess.My pid is1

Iam childprocess.My pid is2

Control Groups (Cgroups)

If namespaces isolate from the perspective of naming and numbering, control groups group processes and truly limit and isolate the computational resources used by each group of processes. Control groups are a kernel mechanism that can group processes, track, and limit their computational resource usage. For each type of computational resource, control groups control through so-called subsystems. Existing subsystems include:

• cpusets: used to allocate a set of CPUs to a specified cgroup, and processes in that cgroup can only be scheduled to execute on that group of CPUs

• blkio: limits the block IO of the cgroup

• cpuacct: used to account for CPU usage in the cgroup

• devices: controls the devices that the cgroup can create and use through a whitelist and blacklist

• freezer: used to suspend or wake up a specified cgroup

• hugetlb: limits the usage of hugetlb in the cgroup

• memory: tracks and limits memory and swap usage

• net_cls: marks packets based on the sending cgroup, and the traffic controller will assign priorities based on these marks

• net_prio: sets the network communication priority of the cgroup

• cpu: sets the scheduling parameters of the CPU in the cgroup

• perf_event: monitors the CPU performance of the cgroup

Unlike namespaces, control groups do not add system calls, but implement a file system to manage control groups through file and directory operations. Below is an example of how cgroups can use the cpuset subsystem to bind processes to a specified CPU for execution.

1. Create a shell script that runs indefinitely

#!/bin/bash

x=0

while[True];do

done;

2. Run this script in the background

# bash run.sh &

[1]20553

3. Check which CPU the script is running on

# ps -eLo ruser,lwp,psr,args | grep 20553 | grep -v grep

root 20553 3bash run.sh

You can see that the process with PID 20553 is running on CPU number 3. Now, let’s use cgroups to bind it to CPU number 2 for execution.

4. Mount the cgroups type file system to a newly created directory cgroups

# mkdir cgroups

# mount -t cgroup -o cpuset cgroups ./cgroups/

# ls cgroups/

cgroup.clone_children cpuset.memory_pressure_enabled

cgroup.procs cpuset.memory_spread_page

cgroup.sane_behavior cpuset.memory_spread_slab

cpuset.cpu_exclusive cpuset.mems

cpuset.cpus cpuset.sched_load_balance

cpuset.effective_cpus cpuset.sched_relax_domain_level

cpuset.effective_mems docker

cpuset.mem_exclusive tasks

cpuset.mem_hardwall notify_on_release

cpuset.memory_migrate release_agent

cpuset.memory_pressure

5. Create a new group group0

# mkdir group0

# ls group0/

cgroup.clone_children cpuset.mem_exclusive cpuset.mems

cgroup.procs cpuset.mem_hardwall cpuset.sched_load_balance

cpuset.cpu_exclusive cpuset.memory_migrate cpuset.sched_relax_domain_level

cpuset.cpus cpuset.memory_pressure notify_on_release

cpuset.effective_cpus cpuset.memory_spread_page tasks

cpuset.effective_mems cpuset.memory_spread_slab

6. Add the above process 20553 to the newly created control group:

# echo 20553 >> group0/tasks

# cat group0/tasks

20553

7. Limit the processes in this group to only run on CPU number 2

# echo 2 > group0/cpuset.cpus

# cat group0/cpuset.cpus

2

8. Check the CPU number on which the process with PID 20553 is running

# ps -eLo ruser,lwp,psr,args | grep 20553 | grep -v grep

root 20553 2bash run.sh

The above example simply demonstrates how to use control groups. Control groups operate through files and directories, and the file system is tree-structured. Therefore, if some restrictions are not placed on the use of cgroups, the configuration can become extremely complex and chaotic. Thus, some restrictions were made in the new version of cgroups.

Conclusion

This article briefly introduces the concept of operating system virtualization, as well as the technologies for implementing operating system virtualization—namespaces and control groups. It also demonstrates the usage of namespaces and control groups through two simple examples.

If you found this article helpful, please share it with more people.

Follow