This article is authorized to be reproduced by the public account “Fresh Date Classroom”, author: Xiaozao Jun.
No matter whether you are in the ICT industry, there is a word you must have heard in the past two years, that is DPU.
As a newly emerging technological concept, DPU is rising at an astonishing speed, becoming a focus of attention in the entire industry and even society. Especially in the investment field, DPU is regarded as a potential technology that will disrupt industries.
What exactly is DPU? How does it differ from the CPU and GPU we are familiar with?
Today, Xiaozao Jun will help everyone thoroughly understand the origin and significance of DPU.
█ What Is DPU
DPU stands for Data Processing Unit.
The CEO of NVIDIA, Jensen Huang, stated in a speech: “DPU will become one of the three pillars of future computing. The standard configuration for future data centers will be ‘CPU + DPU + GPU’. CPU is used for general computing, GPU for accelerated computing, and DPU for data processing.”
It sounds a bit confusing. What kind of data processing does DPU perform? With CPU and GPU, why do we still need DPU?
As we all know, since the birth of computers, they have adopted the famous von Neumann architecture.
Von Neumann Architecture
This is an architecture centered around computation and storage, where the CPU serves as the processing unit responsible for various arithmetic and logical calculations. Memory (RAM) and hard drives (external storage) are responsible for storing data and interacting with the CPU.
In addition to CPU, memory, and hard drives, there are input and output devices like keyboards and monitors. Over time, we later had mice, and then graphics cards and network cards. Eventually, this formed the basic structure of computers as we see today.
With graphics cards, we have GPUs (Graphics Processing Units). Everyone has played games and understands that with the rapid development of multimedia graphics software like games and 3D design, the workload to be processed has become larger and more complex, and the CPU simply cannot keep up. Hence, GPUs were created specifically for image and graphics-related calculations to share the CPU’s burden.
The emergence of DPU follows the same logic. It is also because the CPU cannot handle some complex calculations that task division is needed.
█ What Does DPU Actually Compute?
Finally, we arrive at the key point. After all this, what specific tasks does DPU take on from the CPU?
To be honest, explaining the technical content of the tasks that DPU undertakes can be a bit challenging and not easy to understand. However, we can explain it from the perspective of work scenarios.
Generally speaking, the tasks that DPU undertakes can be summarized in four keywords: virtualization, network, storage, and security.
It is particularly important to remind everyone that DPU is a concept closely related to data centers. In other words, it is mainly used in large-scale computing scenarios like data centers, and not on our personal desktop computers, laptops, or mobile phones (at least not at the moment).
DPU serves cloud computing, with the main role of improving the efficiency of computing infrastructure like data centers, reducing energy waste, and thus lowering costs.
The previously mentioned virtualization, network, storage, and security are very important tasks for data centers and consume a lot of computing resources.
Take networking as an example.
In data centers, there are large amounts of data being transmitted continuously. When the host sends and receives data, it needs to handle massive network protocol processing. According to traditional computing architecture, all this protocol processing is done by the CPU.
Statistics show that to handle 10G of network data at line speed, approximately 4 Xeon CPU cores are needed. This means that merely processing network data packets consumes half the computational power of a high-end 8-core CPU. As data center networks continue to upgrade from 10G to 40G, 100G, and even 400G high-speed networks, how can these performance overheads be sustained?
These costs are so high that they are referred to as “Datacenter Tax”.
Before the business programs even run, simply connecting to network data consumes so much computing resources that it is intolerable. Therefore, companies have proposed the concept of SmartNIC, which offloads network protocol processing from the CPU to the network card to share the CPU’s load.
In 2015, the cloud computing provider AWS was the first to explore this SmartNIC model. They acquired chip manufacturer Annapurna Labs and officially launched the Nitro system in 2017. That same year, Alibaba Cloud also announced a similar function with the Shenlong (X-Dragon) architecture.
In March 2019, NVIDIA spent $6.9 billion to acquire the Israeli chip company Mellanox. NVIDIA combined Mellanox’s ConnectX series high-speed network card technology with its existing technology and officially launched two DPU products in 2020: BlueField-2 DPU and BlueField-2X DPU.
Since then, the concept of DPU has officially entered the public eye. The year 2020 is also referred to as the year of DPU.
Because DPU has such origins with SmartNIC, people generally regard DPU as an expanded and upgraded version of SmartNIC.
On the basis of SmartNIC, DPU also offloads workloads related to storage, security, and virtualization from the CPU onto itself.
In the late 1990s, when virtualization technology represented by VMWare first appeared, it was entirely simulated by software and lacked hardware support, resulting in poor performance and nearly unusable.
Later, in 2005, as technology evolved, the hardware virtualization issues of CPU and memory were gradually resolved, leading to a significant improvement in the performance of virtualization systems and activating the development prospects and value of this technology. As we all know, our entire cloud computing architecture is based on the development of virtualization technology.
The development process of virtualization technology is a process where hardware capabilities continuously replace software capabilities. Earlier, we mentioned AWS’s release of the Nitro system. This system, in addition to SmartNIC, also completed I/O hardware virtualization. It offloaded the virtualization hypervisor from the CPU to dedicated hardware. The result of this is that the performance loss of virtualization technology approaches zero, and the burden on the CPU decreases further.
The same is true for storage.
Data centers now have high demands for storage read and write speeds. As SSD prices gradually decrease, connecting SSDs through local PCIe or high-speed networks to the system has become a mainstream technological route. For distributed systems, based on previous InfiniBand, FC (Fiber Channel), and Ethernet, RDMA (Remote Direct Memory Access) technology has begun to gain popularity.
In RDMA mode, the application’s data no longer goes through the CPU and complex operating systems but communicates directly with the network card. This means that DPU can handle the processing of high-speed interface standard protocols related to storage, further relieving pressure on the CPU.
Finally, let’s look at security.
In the increasingly severe security situation, a large number of encryption algorithms have been introduced to ensure the security and reliability of networks and systems. In the past, these algorithms were all completed by the CPU for encryption and decryption.
But in fact, the network interface is the ideal privacy boundary. Performing encryption and decryption at the network interface is the most reasonable. Therefore, algorithms such as the national secret standard asymmetric encryption algorithm SM2, hash algorithm SM3, and symmetric block cipher algorithm SM4 can actually be computed by DPU. In the future, after the blockchain technology matures, related algorithms can also be offloaded from the CPU to the DPU.
In summary, everyone should also understand that the essence of DPU’s role is offloading, accelerating, and isolating—offloading part of the work from the CPU to itself; utilizing its computational strengths to accelerate these tasks; and achieving isolation of calculations throughout the process.
█ Future Prospects of DPU
DPU is a new type of programmable multi-core processor, a System On Chip (SoC). It conforms to industry standards, has high computational power, and features high-performance network interfaces that can quickly parse and process data, efficiently transmitting data to the CPU and GPU.
The biggest difference between DPU and CPU is that CPU excels at general-purpose computing tasks (it can handle any task, making it quite “mixed”), while DPU is better suited for foundational application tasks (it focuses on specific tasks, making it more “specialized”), such as network protocol processing, routing calculations, encryption and decryption, data compression, and other “dirty work”.
Therefore, DPU is a good helper for CPU, forming an “iron triangle” with CPU and GPU, fundamentally disrupting the computation model of data centers.
This is why DPU is receiving so much attention today.
As mentioned at the beginning of this article, the current popularity of DPU cannot be described with words. The enthusiasm for DPU among capital is even more impressive. Whether it’s giants or startups, they are all entering the DPU race. The DPU market is still heating up, and its development prospects are promising.
Taking NVIDIA as an example, following the BlueField-2 DPU and BlueField-2X, in April 2021, NVIDIA released the new generation of data processors—BlueField-3 DPU.
BlueField-3 DPU
This is the first DPU designed for AI and accelerated computing, optimized for multi-tenant and cloud-native environments, providing network, storage, security, and management services defined by software and hardware at the data center level.
It is claimed that the data center services provided by a BlueField-3 DPU can be equivalent to the services that would require up to 300 x86 cores to achieve. This releases a large amount of CPU resources for running critical business applications.
To realize the core value of DPU in modern data centers, software support is indispensable. In other words, a chip without software is just expensive sand.
To build a more powerful DPU ecosystem, NVIDIA has specifically launched a software development platform tailored for BlueField DPU—NVIDIA DOCA.
DOCA stands for Data Center Infrastructure On A Chip Architecture. With DOCA, developers can quickly create a series of applications and services for networking, storage, security, management, and AI/HPC on NVIDIA BlueField DPU using industry-standard APIs.
In May 2022, NVIDIA released DOCA 1.3. This version not only added 121 new API development interfaces but also included the DOCA Flow library for optimizing data flow insertion, the Communication Channel library, the Regular Expression library, and data path encryption based on OVN, among other features, which were well received by developers.
Architecture of DOCA 1.3
Recently, NVIDIA also released DOCA 1.4, which supports DPU firmware upgrades without rebooting the host, supports 32GB DDR memory on BlueField-2 DPU 25G & 100G w/BMC products, adds support for AArch64 server hosts, and routing functions based on longest prefix matching (LPM).
With the DOCA 1.4 version, developers can more flexibly, simply, and quickly deploy development environments on BlueField DPU, enabling rapid product launches.
█ Conclusion
According to predictions, by 2025, the global market capacity for DPU is expected to reach 12 billion USD.
As Moore’s Law gradually approaches its limits, to utilize computing resources more efficiently, we need to vigorously develop DPU, allowing CPU, GPU, and DPU to perform reasonable divisions of labor, each focusing more on their strengths. This way, we can maximize the energy efficiency of data centers and provide strong and green power for the digital transformation of society.
Disclaimer: This article is a network reproduction, copyright belongs to the original author. If the videos, images, and texts used in this article involve copyright issues, please inform us immediately, and we will delete the content! The content of this article reflects the original author’s views and does not represent the views of this public account or its responsibility for its authenticity.
