When you unlock your phone with facial recognition…
When ChatGPT responds to complex questions in seconds…
When autonomous vehicles recognize road conditions…
Behind all these actions, there is an AI chip “thinking”.
But what exactly are the commonly heard GPU, TPU, and NPU? What are these “black technologies”?
Why can some chips support the training of ChatGPT while others can run efficiently on a phone?
Today, we will break down this battle of AI chips in simple terms.
Understanding the Core Logic: The “Specialties” of AI Chips Are Task-Dependent
AI chips are essentially designed to serve “machine learning”, but the scenarios for machine learning vary widely: training ChatGPT requires processing massive amounts of data and performing high-precision calculations;
facial recognition on mobile phones needs to respond quickly without consuming too much power;
Google search recommendations need to stably handle billions of requests…
Different needs have given rise to three types of chips, each with its strengths—like a kitchen’s chef’s knife, paring knife, and carving knife, each designed for different purposes.
The core contradiction is one:the balance between generality and specificity.
General chips can be used in a wider range of scenarios, but their efficiency may not be high;
the more specialized the chip, the faster it runs on specific tasks, but it struggles to accommodate other needs.
The differences between GPU, TPU, and NPU are all hidden in this balance.
Breaking Down: The Unique Skills of Three Types of Chips
1. GPU: The Versatile Workhorse, From Gaming Graphics Cards to AI Training Powerhouse
GPUs were originally designed for gaming—rendering 3D graphics requires calculating the color of millions of pixels simultaneously, which demands “multi-threaded parallel computing”.
Unexpectedly, this capability perfectly aligns with the needs of AI training: deep learning essentially involves a large number of repetitive matrix operations, akin to solving ten thousand similar math problems at once, and the multi-core architecture of GPUs is tailor-made for this.
Today’s AI GPUs are far from ordinary graphics cards.
Take NVIDIA’s A100 as an example; it has thousands of CUDA cores and can handle over 60,000 threads simultaneously, paired with TB/s-level high-bandwidth memory, akin to providing a “high-speed express lane” for computational units, ensuring smooth data transfer. Its killer feature isflexibility: it supports dynamic adjustments from high precision (FP32) to low precision (BF16), compatible with all major AI frameworks like TensorFlow and PyTorch, whether training image recognition models or large language models, it can handle it all.
Typical scenarios: When OpenAI trained GPT-3, it used thousands of NVIDIA V100 GPUs grouped together to process massive amounts of data; video analysis and recommendation systems in data centers also largely rely on GPUs. The downside is “high power consumption”; high-end GPUs can consume over 300W, requiring dedicated cooling systems.
2. TPU: Google’s Exclusive “Special Forces”, Born for Large Models
When Google was training its own AI models, it found that while GPUs are versatile, there is always “performance waste”—many hardware units are not utilized for the core tasks of deep learning. So Google simply created the TPU (Tensor Processing Unit), optimized specifically for the “tensor operations” of deep learning, concentrating all resources on “tackling tough challenges”.
The core secret of TPU is its “pulsed array” architecture, akin to an efficient assembly line factory: data enters from one end, and each computing unit processes it sequentially without wasting a second. For example, a single TPU v4 chip can achieve a peak performance of 275 TFLOPS at BF16 precision, significantly higher than comparable GPUs; if 1024 TPUs are connected in a cluster, the total computing power can exceed 100 PFLOPS, making training trillion-parameter large models 70% faster than GPUs.
Its advantages arehigh throughput + low energy consumption, which can improve energy efficiency by 30-80 times in large-scale inference tasks in the cloud. Google Search ranking, YouTube recommendations, and DeepMind’s AlphaGo training all rely on TPU clusters. However, the downsides are also evident: extremely poor generality, mainly compatible with Google’s own TensorFlow framework, making it cumbersome to use in other scenarios.
3. NPU: The “Energy-Saving Expert” for Edge Devices, The AI Brain Hidden in Phones
As AI moves from the cloud into phones, smartwatches, and intelligent cameras, “energy efficiency” and “low latency” become the primary demands—no one wants to wait 3 seconds for facial recognition or have their phone die from AI computations after three charges a day. The NPU (Neural Processing Unit) is designed specifically for these “edge scenarios”.
The design philosophy of NPU is “minimalist and efficient”: it does not need to support complex computations like GPUs, but instead focuses on convolution, activation functions, and other commonly used AI operations, designing a dedicated instruction set where a single instruction can complete multi-layer neuron calculations. It also employs techniques like “sparse computation” and “weight sharing” to only compute useful data, reducing resource waste. For instance, the Cambrian MLU270 chip can achieve an energy efficiency of 10 TOPS/W, over ten times that of GPUs, with power consumption as low as 1W.
Today, flagship phones almost all come with NPUs: Apple’s A15 chip’s NPU can process 15 trillion operations per second, supporting real-time 4K video segmentation; Qualcomm’s Snapdragon AI Engine enables real-time translation and beautification; Tesla’s FSD chip’s NPU can quickly process data from multiple cameras, achieving millisecond-level road condition decisions. Its shortcoming is limited computing power, only able to handle small to medium models, and cannot undertake large model training.
Understanding Core Differences at a Glance, No More Confusion
|
Comparison Dimension |
GPU (Graphics Processing Unit) |
TPU (Tensor Processing Unit) |
NPU (Neural Processing Unit) |
|---|---|---|---|
|
Core Advantages |
Strong versatility, flexible adaptation to all scenarios |
High efficiency in tensor operations, extremely high throughput |
Low power consumption, low latency, real-time response |
|
Typical Computing Power (Reference) |
FP32 precision about 30 TFLOPS |
BF16 precision about 275 TFLOPS |
INT8 precision 1-10 TOPS |
|
Representative Products |
NVIDIA A100, RTX 4090 |
Google TPU v4, TPU Pod |
Apple A series NPU, Cambrian MLU |
|
Applicable Scenarios |
Large model training, data center inference |
Cloud large-scale AI services (search, recommendations) |
Mobile phones, autonomous driving, smart hardware |
|
Core Shortcomings |
High power consumption, low efficiency in specialized scenarios |
Poor generality, dependent on specific ecosystems |
Limited computing power, unable to undertake large model training |
Future Trends: No Absolute King, Only Collaborative Combat
The AI chip industry is no longer an era of “going solo”; more and more scenarios require the three types of chips to work together:
For example, using TPU to quickly complete the basic training of large models, then using GPU for fine-tuning, and finally deploying the optimized model to NPU for real-time applications on mobile devices.
Moreover, architectural integration has also become a new trend:
GPUs have incorporated dedicated Tensor Cores to enhance matrix operation efficiency, NPUs have borrowed the pulsed array design from TPUs, and even mobile chips are starting to support “lightweight training”.
In the future, we may not see purely GPU, TPU, or NPU, but rather “heterogeneous computing platforms” that can automatically allocate resources based on tasks.
Finally, a small tip:
To determine the purpose of a certain type of AI chip, look at its “core demand”—choose GPU for generality, TPU for large-scale cloud tasks, and NPU for energy efficiency in edge devices. Next time you see chip news, you’ll easily grasp its positioning!
What other details about AI chips would you like to know? For example, how NPU optimizes photography on phones, or the specific process of GPU training large models, feel free to leave a comment!
Scan to add the course assistanteKnow AI MemberLearn what I love, create what I think, and let every learning experience become a starting point for creation!👉 [Click here to activate your membership privileges now!]