Yang Gengdan Instant Incentive Sharing
@dannyyang
WeBank, as the first internet private bank in China, is also the bank that has thoroughly embraced the “De-IOE” movement while open-sourcing its excellent products. With the trend of domestic hardware localization, domestic chips are widely used in our data centers.
Storage
Relying on our distributed architecture, we use low-power Arm chips and open-source Ceph to build a distributed storage cluster. The distributed storage cluster does not require powerful x86 processors and has higher demands on disk and network compared to other applications. Building storage on Arm servers not only ensures performance but also reduces procurement costs and power consumption in the data center. The Arm-based distributed storage cluster has now been deployed on a large scale in our bank, providing file storage and block storage services for our production systems.
Computation
As a computing node, Arm servers cannot avoid the topic of virtualization in the context of cloud computing. The mainstream x86 hardware virtualization technologies include Intel’s VT-x and AMD’s AMD-V. Does the Arm architecture support virtualization? The answer is certainly yes. By comparing the open-source KVM virtualization technology between Arm and x86, the main difference in configuration lies in the choice of virtualized CPU simulation.
KVM (Kernel-based Virtual Machine) is a full-featured virtualization solution on Linux hardware platforms.
KVM mainly supports three CPU modes:
#
host-passthrough
Libvirt passes the host CPU instruction set directly to the virtual machine. What is seen in the virtual machine is the model of the physical CPU. This mode is generally used when performing nested virtualization; the downside is that hot migration cannot be performed between different CPU models using this mode.
#
host-model
Libvirt selects the closest CPU model based on the current host CPU instruction set from the configuration file /usr/share/libvirt/cpu_map.xml. In this mode, the instruction set of the virtual machine is less than that of the host machine, and the performance is relatively worse than that of host-passthrough. During hot migration, some differences between the target host CPU and the source host are allowed.
#
custom
In this mode, the virtual machine sees the smallest number of CPU instruction sets, hence the performance is relatively the worst, but it has the strongest ability to cross different CPU models during hot migration.
All three modes are supported on the x86 architecture, while on the Arm architecture, through our testing, only the host-passthrough mode is available; the other two modes will throw exceptions. It is recommended to use a newer version from the community. Based on performance data from actual tests in our lab, the differences in disk, network, and memory are not significant; the main difference lies in computing power. Using the stress-ng tool, we tested the CPU through different algorithms. With the same configuration of 8 cores and 16GB of memory, the computing power of the x86 virtual machine is approximately three times that of the Arm virtual machine. This data serves as a reference for migrating applications from x86 to Arm.
|
|
x86 (8cpu*16GMEM) |
arm (8cpu*16GMEM) |
|
1 |
308481 |
111130 |
|
2 |
308391 |
112355 |
|
3 |
308502 |
111343 |
Deploying distributed applications on Arm servers and performing virtualization on Arm servers have both been validated as feasible. Introducing domestic chips allows for CPU diversity, which helps make data centers more robust. We will continue to explore the direction of further component localization.
dao
dao
ge
ge
dian
dian
zan
zan
During the entire process of introducing domestic hardware, Danny actively communicated and coordinated with manufacturers, discussed technology in open-source communities, and made internal architecture changes and code modifications, ensuring the migration and use of core components at the bottom layer, while ensuring high availability and stability for computation and storage. He has made significant contributions to the application of domestic hardware; we hope to continue to strive for greater breakthroughs in the application of domestic operating systems in the future.
To be continued
Want to get more WeStar-like skills
Please continue to follow the TCTP public account!