It is well known that artificial intelligence (AI) is driving technological innovation, especially in the field of large-scale data processing. Machine learning (ML) and deep learning (DL) algorithms, designed to solve complex problems and learn autonomously, are continuously pushing the boundaries of computer capabilities.
As the complexity of problems requiring computer processing increases, the number of processes running has also exploded. This growth has led to the emergence of dedicated processors and a plethora of new terminology.
In addition to the widely recognized central processing unit (CPU), neural network processors (NPU), graphics processing units (GPU), and tensor processing units (TPU) have also joined the ranks of processors.
Next, we will specifically analyze the working principles of these dedicated processors and their differences. If you are interested, take a moment to listen to a lesson in IT history—I will delve into more technical concepts that have emerged from the integration of hardware and software over the past century.
Central Processing Unit (CPU): The Veteran of Computation
The CPU can be viewed as the general of a computer. It mainly consists of an arithmetic logic unit (ALU) and a control unit: the ALU is responsible for executing arithmetic operations (addition, subtraction, etc.) and logical operations (AND, OR, NOT, etc.); the control unit regulates the ALU, memory, and input/output functions, instructing them on how to respond to the programs just read from memory.
The most intuitive way to understand the CPU’s operation is to trace its input/output flow: the CPU receives requests (input), accesses the computer’s memory to obtain task instructions, delegates execution to its own ALU or other dedicated processors, and then integrates all data in the control unit to take unified action (output).
The following diagram is the circuit structure of an ALU from 1970:
More importantly, the following schematic clearly illustrates the operational logic of the CPU:
As we have transitioned from single-core processors to multi-core processors, the performance of CPUs has become increasingly powerful. The core of this improvement lies in the collaborative execution of tasks by multiple arithmetic logic units (ALUs), which are scheduled by the CPU’s control unit to achieve parallel processing. This architecture allows the CPU to work efficiently alongside dedicated AI processors like GPUs.
The Rise of Dedicated Processors
When a computer receives a task, the processor first needs to communicate with memory—this includes the program memory (ROM) for fixed tasks like booting, and the data memory (RAM) for frequently changing operations like loading applications, editing documents, and browsing the web. The bus connecting these components is the data transfer channel, but it can only access one type of memory at a time.
In the past, processors operated slower than memory access, but with the evolution of processor technology, this situation has reversed. Nowadays, when processing massive amounts of data, CPUs often find themselves waiting for memory access due to bus transfer congestion, which not only reduces efficiency but also leads to increased energy consumption. This phenomenon is referred to as the “von Neumann bottleneck” in the computing field. With the emergence of complex tasks like AI, breaking through this bottleneck has become urgent.
One solution is to develop chips optimized for specific tasks. Dedicated chips can effectively alleviate the processing pressure that machine learning algorithms place on CPUs. In the competition to create the best AI processors, giants like Google, IBM, Microsoft, NVIDIA, and Huawei have achieved more complex logical queries through dedicated processors, each with its unique technological path. Let’s analyze the characteristics of GPUs, TPUs, and NPUs:
Graphics Processing Unit (GPU)
The GPU was originally a specialized graphics processor, often mistakenly thought to be a graphics card with more hardware. Its design intent was to support massive parallel processing, working in conjunction with the CPU (fully integrated on the motherboard or as standalone hardware to handle high-load tasks), but the heat generated by high energy consumption is a significant issue.
GPUs have long been used in the gaming field until NVIDIA pushed for their transition to general computing in the early 21st century. In addition to chip design, NVIDIA’s proprietary CUDA platform allows programmers to directly invoke the GPU’s virtual instruction set and parallel computing units, meaning that computational kernels (clusters of processors working together) can be built to specialize in specific tasks without overusing other resources. The following diagram intuitively demonstrates its workflow:
This enables GPUs to be widely used in machine learning tasks, leveraging existing mature technology systems. In other words, when searching for solutions, the ultimately successful option is not always the one with pure performance superiority. If a technology requires a complete change in user behavior or necessitates everyone to relearn skills, it often faces resistance to adoption. Therefore, GPUs are well compatible with existing systems, programming languages, and other features, greatly facilitating their popularity. While not achieving plug-and-play status, their core advantages are evident.
With technological advancements, there are now open-source GPU platforms supported by industry giants (including NVIDIA), with OpenCL being the largest in scale. Additionally, tensor cores have been introduced (the principles of which will be elaborated on later in this article).
Tensor Processing Unit (TPU)
The good news is: this term simply refers to Google’s proprietary AI processor. Google has been using it in its own data centers since 2015 and publicly released it in 2016, with several commercial models now available. They are based on ASICs (Application-Specific Integrated Circuits, which will be discussed later) and run on Google’s TensorFlow software.
Compared to GPUs, the uniqueness of TPUs lies in their use of slightly lower computational precision, allowing them to flexibly handle different types of workloads. Google’s explanation is particularly insightful:
“If it rains outside, you don’t need to know the exact number of raindrops per second—you just need to understand the intensity of the rain. Similarly, neural network predictions typically do not require the precision of 32-bit or even 16-bit floating-point numbers. Through optimization, using 8-bit integers for neural network predictions can still maintain adequate precision.” (Excerpt from Google Cloud Blog)
While GPUs were originally designed for graphics processing and rendering, their effectiveness relies on the precise relationships between each pixel. If the precision of point locations is insufficient, vector calculations can lead to error magnification, resulting in visual quality differences akin to those between the PS2 and PS4 versions of Spyro.
Another key design difference from CPUs and GPUs is that TPUs adopt a pulsed array architecture. This architecture constructs a network of processors, where each node performs part of the computational task and then passes the results to the next node until the entire processing flow is completed. The nodes are typically fixed and identical, but the programs running between nodes can be customized. This structure is referred to as a data processing unit (DPU).
Neural Network Processor (NPU)
“NPU” is sometimes used as a general term for all dedicated AI processors, but it more specifically refers to processors designed for mobile devices.
NPUs contain all the components necessary for AI processing, operating based on the principle of synaptic weights. This concept, borrowed from biology, describes the strength of the connection between two neurons: when two neurons frequently transmit information, their connection is strengthened, making energy transfer more efficient—this is the scientific principle behind habit formation. Many neural networks simulate this mechanism.
The learning process of AI algorithms is reflected here: the system continuously tracks potential possibilities and strengthens the connection weights of relevant nodes. This is significant for power consumption control. While parallel processing can execute multiple tasks simultaneously, there are shortcomings in coordinating task completion, especially when architectural expansion leads to the dispersion of processing units.
Neural Networks and Decision Mechanisms in Computers
The human decision-making process is essentially composed of a series of continuous judgments, with the final decision influenced by all prior considerations. Since computers operate on strict binary logic, they do not inherently possess the ability to optimize decisions through contextualizing information. Neural networks are the solution to this problem—they are built on matrix mathematics, as illustrated in the following diagram:
Essentially, this allows computers to evaluate the global possibilities of each potential decision, weigh the outcomes, and learn from their own experiences and perceived information. All of this means that more computations need to be run simultaneously.
Summary of Core Differences
With a wealth of information, here are the key points:
-
Functional Positioning: GPUs were originally developed for graphics rendering, while TPUs and NPUs are built specifically for AI/machine learning workloads.
-
Parallel Processing: GPUs are designed for parallel processing, suitable for training complex neural networks. TPUs specialize this feature, focusing on tensor operations for higher speed and energy efficiency.
-
Customization Level: TPUs and NPUs are highly customized for AI tasks, while GPUs provide a general solution applicable to various computational tasks.
-
Application Scenarios: GPUs are commonly used in data centers and workstations for AI research and training; TPUs are widely used in Google Cloud infrastructure; NPUs are prevalent in AI-enabled devices such as smartphones and IoT devices.
-
Availability: GPUs are widely available from multiple manufacturers for researchers, developers, and enthusiasts; TPUs are exclusive resources of Google Cloud services; NPUs are integrated into specific devices.
Are These Differences Important?
At first glance, the definitions of different processors may seem similar. Multi-core processors integrate multiple ALUs under a central control unit; GPUs integrate more ALUs under dedicated processors; TPUs integrate multiple computing nodes under a DPU similar to a CPU.
Ultimately, the design choices between processors have subtle differences, but their real impact is seen at the large-scale application level rather than at the consumer level. Dedicated processors can handle large datasets more efficiently, meaning faster processing with less power (though total power consumption may rise as AI tools are used more frequently).
Equally important is that these terms themselves are in a rapidly evolving emerging field. Google TPU was released in 2015, just ten years ago. I cannot count how many discussions have ended with exaggerated imaginations of how AI will change the world, largely stemming from the belief that its capabilities are limitless.
However, the innovations that make AI possible are created by real human beings. (Although in the future, AI may be able to program itself, who knows?) The chips that drive AI are real physical entities—derived from silicon materials and processed in laboratories. Understanding these physical realities, the challenges we have had to overcome, and their solutions can help us utilize these tools more efficiently—and create cooler things in the future.
A Brief History of Hardware Development
This leads us to our history lesson. To better understand today’s topic, one must have a basic understanding of how computers are physically constructed. The most fundamental language of computers is binary code, represented by sequences of 0s and 1s. These values correspond to the on/off states of circuits: when the circuit is closed, current cannot pass through; when it is open, current can pass through. Transistors are used to regulate current, generate electrical signals, and act as switches or gates. Connecting a large number of transistors with circuits forms integrated circuit chips.
The combination patterns of the on/off states of transistors can be read by computers. As the number of transistors increases, the range of numbers that can be represented by binary code also expands. This affects our basic calculations for measuring bits and bytes: eight transistors store one byte of data (each transistor has two states, and all possible combinations (2^8) yield 256 combinations of open/closed gates (bits)). Therefore, 8 bits = 1 byte, which can represent any number between 0 and 255.
The miniaturization of transistors and the increase in transistor density on a single chip have driven advancements in computing power, speed, and energy efficiency. This has been primarily due to breakthroughs in semiconductor material purification technology, the application of precision chemical etching, and improvements in cleanroom technology. All of this began with the birth of integrated circuit chips.
Around 1958, several scientists simultaneously overcame different technical challenges, collectively leading to the invention of the integrated circuit. Jack Kilby of Texas Instruments developed a hybrid integrated circuit, measuring approximately 7/16×1/16 inches (11.1×1.6 mm). Later, Robert Noyce, co-founder of Intel, pioneered the monolithic integrated circuit (integrating all circuits onto the same chip), which was similar in size.
It is noteworthy that the original chips could only accommodate about 60 transistors. Today’s microchips can etch billions of transistors in much smaller sizes. The following image shows a physical picture of a disassembled integrated circuit:
Today’s chips are smaller than a coin.
Friends, this is one of the important reasons why modern smartphones can transform into pocket computers. It is easy to imagine that modern laptops or rack servers, with their larger sizes, can more efficiently integrate more of these components, thus giving rise to the rise of artificial intelligence.
What is FPGA?
Everything mentioned above pertains to fixed physical units on chips, but chip performance is also influenced by software. Software defines the logic and instructions for all components to work together. Therefore, there are two choices when manufacturing chips: either customize dedicated chips for specific software or use reconfigurable “blank chips”.
The first is called Application-Specific Integrated Circuits (ASICs). However, like all proprietary manufacturing processes, they must be produced on a large scale to be profitable, and the production cycle is relatively long. CPUs and GPUs typically operate based on this type of hardwired chip.
Reconfigurable chips are known as Field-Programmable Gate Arrays (FPGAs). They offer high flexibility and provide developers with various standard interfaces. This is highly valuable for AI applications (especially deep learning algorithms)—as technology iterates rapidly, FPGAs can achieve multifunctionality through reprogramming on the same chip, allowing developers to quickly complete testing, iteration, and market launch. Their flexibility is most prominently reflected in reprogrammable input/output interfaces, thus reducing latency and breaking through bottlenecks. For this reason, the industry often compares the performance of entire ASIC processor categories (CPU/GPU/NPU/TPU) with FPGAs, which has also led to the emergence of hybrid solutions.
Conclusion: The Astonishing Technology of Chips
Advancements in materials science and microchip manufacturing have laid the foundation for the processing power required by AI, with industry giants (NVIDIA, Huawei, Intel, Google, Microsoft, etc.) utilizing these chips to develop dedicated processors.
Meanwhile, software technology has enabled multi-processing core networking, enhancing computational speed through load distribution. All these advancements have collectively given rise to dedicated chips that meet the enormous demands of AI.