With the rapid development of NPU, is it still necessary to study the architecture related to CPU? In fact, this is a matter of passing down knowledge, and I personally believe it is still very necessary. Through a comparison of CPU and NPU, we can see that NPU can be regarded as an extension and development of CPU, with the core idea being parallelism and efficiency.
The complex technologies in CPU, such as superscalar (Superscalar), out of order execution (Out of Order), branch prediction (Branch Prediction), have not become obsolete in the era of AI chips/NPU. In fact, they still exist but in a different form, because ultimately they are all aimed at solving a core contradiction: how to maximize the utilization of hardware resources (computing units, memory bandwidth) while executing instructions in parallel as much as possible in the presence of control dependencies (such as branches) and data dependencies (such as WAR/WAW), thereby improving performance. This core contradiction—the game between dependency and parallelism—exists not only in AI chips but is also amplified to the extreme. The battlefield and weapons have simply changed. The table below compares these concepts in CPU and NPU.
|
Core Concept |
Role in Traditional CPU |
Reincarnation and Evolution in NPU/AI Chips |
Is There Significance? |
|
Superscalar (Superscalar) |
Issuing multiple instructions to different execution units (such as ALU, FPU, LSU) within one clock cycle. |
Large-scale parallel arrays. NPU does not issue a few instructions but directly deploys hundreds or thousands of processing elements (PE) to compute large amounts of data (SIMD/SIMT) simultaneously under one instruction. This is ultra-large-scale “data parallelism”. |
Yes, the idea has been upgraded, and the granularity of parallelism has greatly increased. |
|
Out of Order Execution (Out of Order) & ROB |
Dynamic discovery of parts of instructions that can be executed in parallel, reordering instructions to avoid stalls. This trades hardware complexity for generality. |
Data flow driven/static scheduling. The AI computation graph (such as CNN) is known before execution, and data dependencies are determined at compile time. The compiler performs static scheduling to generate an optimal execution sequence (such as optimizing the graph with some optimization algorithms or the loss function of computational resources), allowing hardware to execute in order to maximize utilization without complex dynamic scheduling hardware. |
Yes, but the implementation method is reversed; dynamic scheduling executed by hardware is transformed into static scheduling executed by the compiler. |
|
Branch Prediction (Branch Prediction) |
Predicting program flow, pre-fetching and executing to avoid pipeline stalls. Critical for control-intensive tasks. |
Control flow simplification. AI computation is mainly compute-intensive, dominated by loops and regular data flows, and almost does not require complex control flow branches. Chip design tends to eliminate such complex units, using deterministic hardware pipelines to achieve extreme energy efficiency. |
Weaker, as the demands differ; MoE and other sparse activations may require routing prediction/sparse scheduling, but not traditional branch prediction. |
|
Register Renaming & WAR/WAW Handling |
Solving false data dependencies (name dependencies) between instructions, generally resolved through mapping physical registers to architectural registers, supporting out-of-order execution. |
Explicit data movement and synchronization. In NPU, data dependencies are managed through explicit movement instructions between different levels of memory (global DRAM, shared SRAM, registers) (such as DMA). Dependencies are resolved by the compiler during scheduling, while hardware is responsible for executing movement and computation instructions. |
Yes, similar to branch prediction, its function has shifted from hardware to the compiler; MoE and other sparse activations may require routing prediction/sparse scheduling, but not traditional branch prediction. |
|
Memory Hierarchy (Cache) |
Reducing access latency through a hardware-managed cache hierarchy is crucial for CPU performance. Cache is not perceived by users and is updated by the CPU through hardware scheduling policies. |
Software-managed storage hierarchy. NPU typically has multiple levels of on-chip SRAM (global shared/local registers), with allocation and usage explicitly controlled by the compiler. The compiler knows the global lifecycle of data and can make better scheduling decisions than hardware algorithms like LRU, maximizing data reuse (such as inputs/weights/outputs in convolution). |
Yes, but the management strategy has revolutionized. |
Learning CPU architecture is still crucial.
It lays the foundation for core thinking patterns: learning CPU architecture allows for a profound understanding of fundamental computing concepts such as latency, bandwidth, parallelism, locality, and throughput. These are the golden criteria for evaluating any computing system (including NPU) and help understand why NPU is designed that way. Understanding the “why” behind changes: understanding the costs that CPU incurs for generality (complex control logic, huge power overhead, latency caused by cache misses) allows for a true understanding of NPU’s design philosophy: “sacrificing generality for extreme efficiency.” Knowing what NPU “omits” and why omitting these things becomes its advantage. Analogous learning: many ideas are interconnected. For example, CPU’s vector instructions (AVX series) are the prototype of data parallelism and form the basis for understanding NPU’s large-scale arrays. CPU’s prefetching techniques and NPU’s compiler data movement scheduling aim to hide memory latency. The collaboration of multi-core CPUs and the design of multi-core clusters in NPU have many similarities. NPU shifts the focus of innovation in computer architecture from “how to smartly execute random instruction streams” in CPU to “how to efficiently execute a known computation graph” in NPU. The core challenge becomes: balancing computation and storage (the von Neumann bottleneck): how to reduce data movement and keep computing units “well-fed.” This has given rise to new paradigms such as pulsed arrays, data flow architectures, and in-memory computing. Data Reuse: how to maximize the retention of data to be reused (such as weights and inputs in convolution) across various levels of storage is key to energy efficiency. Sparsity: how to leverage sparsity in neural networks (many values are zero) to skip unnecessary computations and further enhance performance.
TGIF