Source: Software and Hardware Integration

Author: Chaobowx

1 Comprehensive Introduction to SoC and HPU

What is SoC? SoC stands for System on Chip, also known as a chip on a system. In a narrow sense, SoC integrates the critical components of a system onto a single chip; in a broad sense, SoC is a system. If the Central Processing Unit (CPU) is the brain, then SoC is a system that includes the brain, heart, eyes, and hands.

The above image shows Qualcomm’s mobile SoC processor Snapdragon 810 chip. This chip mainly includes:

• General-purpose CPU: ARM Cortex-A57 and Cortex-A53, mainly used to run Android and other smartphone operating systems and applications;

• Specific scenario processors: Adreno 430 GPU, Hexagon DSP, ISP, multimedia processors, etc.; the GPU is mainly used for 3D gaming scenarios, while the DSP is mainly used for sensor algorithm processing;

• Specific function subsystems: Communication baseband processing supporting 4G LTE, positioning modules such as GPS/Beidou, etc.;

• Connectivity modules such as WiFi, USB, Bluetooth;

• Security processing modules;

• Other peripheral modules.

Ultra-heterogeneous computing integrates various heterogeneous computing architectures, including CPU homogeneity, GPU heterogeneity, and various DSA heterogeneities. An ultra-heterogeneous processor implements ultra-heterogeneous computing architecture at the chip level (which means: ultra-heterogeneous can also be implemented at the board-level multi-chip level, multi-computing node cluster level, and even higher levels such as data centers).

Currently, data centers have three major chips: CPU suitable for application layer work, GPU suitable for elastic acceleration of business, and DPU suitable for acceleration processing of infrastructure layer tasks. From a functional perspective, HPU can be regarded as a fusion of CPU, GPU, and DPU.

From the perspective of the von Neumann architecture, we can understand that a computer consists of five components: input devices, output devices, controllers, arithmetic units, and memory. In modern computers, these are usually classified into three components: processor, which includes controllers and arithmetic units; memory, consistent with the von Neumann architecture; and input-output I/O devices, where input and output are combined as a category of devices for external communication.

Different types of chips are composed of the above three categories of components. There is no essential difference between their memory and I/O devices; the core difference lies in the processor part:

• CPU chips have CPU cores inside.

• GPU chips have millions of efficient small CPU cores. Therefore, GPUs are essentially many-core parallel computing platforms.

• Various DSA chips mainly have DSA accelerator processor cores.

• SoCs include CPU, GPU, ISP, and various ASIC-level accelerator processor cores.

• HPUs have CPU, GPU, and various DSA accelerator processor cores.

From the definition of SoC, ultra-heterogeneous processors can also fall under the category of SoC. However, if we only refer to it as SoC, it does not reflect the essential differences between ultra-heterogeneous processors and traditional SoCs. This is detrimental to our deep understanding of the innovative value and importance of ultra-heterogeneous processors, as well as the need to invest more resources in the innovative technologies and architectures required to support ultra-heterogeneous processors.

2 Difference 1: Single System vs. Distributed Hybrid Multi-System

As the name suggests, SoC is a system on a chip. This means designing a chip for a specific scenario. The system and the chip have a one-to-one relationship that is completely matched. Because systems are diverse, this also means that there will be many types of SoCs.

On the other hand, HPU is positioned as a basic general-purpose processor, targeting a macro-comprehensive computing system and tasks, not aimed at any specific system or task. Therefore, the characteristics of HPU computing are reflected in:

• HPUs usually complete tasks through cluster collaboration;

• A single HPU hardware will run multiple different software systems and tasks;

• Multiple macro distributed large systems run in a hybrid and cross manner on multiple HPUs.

3 Difference 2: Weak Virtualization vs. Hardware Native Virtualization

Virtualization is the core capability difference between HPU and SoC.

SoC, being oriented towards a single system, usually does not need to support virtualization. Some CPU cores in SoCs support virtualization, but the performance overhead of virtualization is relatively high; furthermore, only CPUs support virtualization. Other memory, accelerators, and I/O do not support virtualization.

In contrast, HPU not only requires complete hardware virtualization of CPU and memory but also requires other I/O and accelerator cards to achieve complete hardware virtualization. It is important to emphasize that I/O virtualization does not only refer to the virtualization of I/O interfaces like PCIE SR-IOV or S-IOV, but also to the virtualization of internal processing engines of I/O.

For example, in automotive chips, the automotive EE architecture is currently undergoing a disruptive change, transitioning from the traditional ECU and DCU architecture to the CCU (Central Control Unit) architecture. The most distinct difference between multi-domain integrated autonomous driving CCU chips and DCUs lies in whether virtualization is supported. In the CCU, each VM is equivalent to a traditional DCU SoC system.

4 Difference 3: Integrated Software and Hardware vs. Separate Software and Hardware

In SoCs, software is usually attached to the hardware, and the two have a matching relationship. We can customize specific software based on the hardware’s architecture/interfaces, or adapt standard software and different hardware interfaces through the HAL layer.

HPUs require much higher standards. The software on HPUs has no direct relationship with the hardware. Software can run on hardware A, B, or any other hardware. Conversely, hardware can run any possible software. The execution and migration of software across different hardware resources are completely dynamic and, from a macro perspective, very frequent.

Typically, virtualization can be used to mask hardware interfaces, providing standardized hardware for software, facilitating the migration of software VMs/containers. However, as performance requirements increase, virtualization is gradually offloaded to hardware. VMs/containers need to directly interface with hardware interfaces through passthrough. When virtualization is fully offloaded to hardware acceleration, the hardware must provide completely consistent interfaces/architectures.

5 Difference 4: Control-Driven vs. Data-Driven

Previously, more computational load with less data volume meant that the architecture driven by data flow represented by CPU units was the primary model. Nowadays, in the era of big data computing, the characteristics of computation have shifted to a model of “large data volume with small computation load” (the term small computation load is relative), thus the data flow-driven computing architecture has become the mainstay of computational power.

In SoC, the architecture is CPU-centered, primarily relying on the software within embedded CPU cores to drive the operation of the entire SoC. In HPU, however, the architecture is data-centered, mainly relying on the flow of data to drive computational operations.

It is important to emphasize that some viewpoints suggest that a DPU-centered computing architecture is inherently data-driven. This assertion is not entirely correct. A DPU-centered architecture still relies on the CPU to control the operation of the entire board-level system, which means it is still a CPU-controlled architecture.

To truly achieve a data-driven computing architecture, significant adjustments must be made at the underlying hardware and software levels. This is challenging and labor-intensive.

6 Difference 5: Multi-Heterogeneous Software Collaboration vs. Multi-Heterogeneous Hardware Fusion

Both SoC and HPU are hybrid computing made up of multiple heterogeneous components, but the difference lies in that SoC is merely heterogeneous integration, while HPU requires heterogeneous fusion.

In SoC systems, each acceleration unit can be seen as a heterogeneous subsystem composed of CPU + acceleration unit; different heterogeneous subsystems are not associated with each other at the hardware level, requiring software to build interactions and collaborations between heterogeneous subsystems. In the current context of CPU performance gradually reaching its limits, this often means performance constraints.

In HPU, however, direct and efficient data interaction between different acceleration units at the hardware level is required, without the involvement of embedded CPUs. This way, at the hardware level, deep interactions, collaborations, and fusions between CPUs, GPUs, and various other acceleration units are achieved.

7 Difference 6: Software Programmable vs. Multi-Level Programmable

In SoC, other accelerators are usually at the ASIC level, allowing only simple control, while the business logic functions of the entire data plane are completely determined and cannot be programmed through software. In SoC, typically only the embedded CPU supports software programming.

In HPU, however, the programmability is richer:

• DSA Programmable: At each computing node, tasks that are performance-sensitive and have infrequently changing functional logic can be classified at the infrastructure level. Suitable for DSA acceleration processing.

• GPU Programmable: Here, the GPU refers specifically to GPGPU that supports parallel computing programming, and even AI programming; not just the classic GPU with graphics acceleration functionality.

• CPU Programmable: Similar to the CPU in SoC, fully software programmable.

8 Difference 7: Resource Determined vs. Resource Elastic Scalability

CPUs support resource (dynamic) scalability: by slicing a single CPU core into thousands of parts; then connecting dozens of CPU cores into a large CPU computing resource group through homogeneous parallelism; and resource expansion of multiple CPU chips can be achieved through UPI or even consistency networks.

Within SoCs, apart from CPU supporting scalability, other modules are generally designed with performance that is determined and cannot support resource expansion capabilities.

In HPU, every computing resource, even I/O resources, must support multiple levels of resource scalability, similar to CPUs. It can even achieve “almost infinite” resource expansion capabilities for clusters of thousands of chips.

9 Difference 8: Custom Chiplet vs. Native Support for Chiplet

When designing SoCs, if optimization through Chiplet is required, it usually necessitates designing several small chiplets specifically. These chiplets have different functions and are then connected and packaged into a system chip through Chiplet. This approach has some problems:

• Limited value of optimization. Some chiplets can use non-advanced processes to reduce costs; another is to reduce the area of a single DIE, optimizing yield. The value of this enhancement is merely percentage-based.

• Low reusability. Different models of chiplet DIEs are non-standard devices, which essentially increase the difficulty of chip integration.

HPUs adopt a design of resource elastic expansion, allowing for: single DIE HPU chips and HPU chips of different specifications encapsulated by different numbers of DIEs through Chiplet.

10 Difference 9: Small Design Scale vs. Orders of Magnitude Increase in Design Scale

Currently, as system scales increase, the design scale of single chips is also growing, but the traditional SoC architecture’s ability to support system scale is gradually approaching its limit. The scale limit means that although processes and packaging support larger-scale chip designs, under traditional architectures, exceeding the scale limit will lead to a dramatic increase in the complexity of the entire system, making it difficult to manage; and the utilization of various resources and performance will sharply decline, leading to severe waste and low economic efficiency.

With continuous optimization of processes and advancements in Chiplet packaging, the design scale of single chips is increasing by orders of magnitude. An innovative architecture is urgently needed to quickly enhance the “manageable conditions” of system scale.

SoC supports a single system, while HPU supports multiple systems. Moreover, HPU adopts a scalable distributed system architecture design, where each subsystem is equivalent to a SoC system. The manageable system scale of HPU can reach 10 times or even 100 times that of SoC.

11 Difference 10: Dedicated vs. General-purpose

SoC is a chip developed for a specific scenario.

HPU targets general, complex computing scenarios, positioned to respond to changes. Currently, there are three main architectures for general computing:

• CPU. Also known as GP-CPU, the general-purpose central processing unit can be used in almost any scenario.

• CPU + GPU. When CPU performance is insufficient, GPUs serve as many-core parallel acceleration platforms, offering performance enhancements by orders of magnitude compared to CPUs. Therefore, some performance-sensitive tasks suitable for GPU parallel computation can be handled by GPUs, while others continue to be processed by CPUs.

• HPU (fusion of CPU + GPU + DSA). DSA provides higher performance efficiency than GPUs; with the same transistor resources, DSA can achieve performance improvements by orders of magnitude compared to GPUs. But DSA is only suitable for relatively deterministic tasks. According to the 80/20 rule, 80% of computations are handled by DSA, while GPUs accelerate the remaining performance-sensitive tasks suitable for parallel computation. The remaining tasks not suitable for acceleration continue to be processed by CPUs.

12 Summary

The differences summarized in the above ten aspects are shown in the table below.

Of course, these are the important aspects we currently think of. There are many other differences that cannot be listed one by one. Functionally, both HPU and SoC integrate many functions together, showing many similarities. However, essentially, the two are completely different product positions and development directions.

As the demand for computing power continues to rise and the computing power network continues to expand, in the future, almost all processors will become computing chips with ultra-heterogeneous architectures, and ultra-heterogeneous processors will become the core foundation supporting macro computing power.

(End of text)

Reproduced content only represents the author’s views

Does not represent the position of the Institute of Semiconductors, Chinese Academy of Sciences

Editor: Chidori