GPU, NPU, TPU: The Battle for AI Computing Power

Introduction: The Debate on the “Cornerstone of Computing Power” in the AI Wave

Since the end of 2022, a technological revolution represented by large language models (LLMs) and generative AI has swept the globe, driven by an almost insatiable demand for computing power. Computing power has become the core engine driving the development of artificial intelligence, akin to the “new oil” of the digital economy era. In this arms race for computing power, AI chips serve as the physical foundation, with their architecture, performance, and ecosystem directly determining the speed and boundaries of AI technology evolution.

This article will focus on the three main players in the current AI chip field: GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), and NPU (Neural Processing Unit). They are not simply substitutes for one another but are key solutions evolved to address different AI computing challenges. We will systematically analyze the following core questions: What are the fundamental differences in the technical essence of these three types of chips? Who are the dominant players in the global and Chinese markets, and what is the competitive landscape? Looking to the future, what will be their respective development paths? By answering these questions, this article aims to present readers with a panoramic view of the competition in AI computing power.

Part One: In-Depth Analysis of Technical Architectures and Application Scenarios: The “Genetic” Differences Between GPU, NPU, and TPU

To understand the landscape of AI chips, we must first return to their technical origins. The design philosophies, core architectures, and application scenarios of GPU, TPU, and NPU exhibit essential differences, which determine their different positions in the AI computing power map.

1. GPU (Graphics Processing Unit): The King of General Parallel Computing

Architectural Principles: The birth of the GPU stemmed from the need to render complex 3D graphics. Its core architecture is “Single Instruction, Multiple Threads” (SIMT), which drives hundreds or thousands of computing cores (ALUs) to perform the same operation simultaneously with a single instruction. This design makes it inherently adept at handling large-scale, data-parallel, and compute-intensive tasks, such as the ubiquitous matrix multiplications in neural network training.

Core Advantages:

• Versatility and Flexibility: The strength of GPUs lies in their high programmability. With NVIDIA’s CUDA (Compute Unified Device Architecture) platform as a representative, developers can use high-level languages (such as C++, Python) to precisely control the GPU’s computing processes. This allows them to not only excel in AI training but also to be widely applied in various high-performance computing (HPC) scenarios such as scientific computing, data analysis, and physical simulations.
• Mature Ecosystem: After more than a decade of development, NVIDIA has built an unparalleled software ecosystem around CUDA. From low-level drivers, compilers, and mathematical libraries (cuDNN) to top-level application frameworks (deep support for TensorFlow and PyTorch), along with a vast developer community and academic recognition, it forms an unassailable “moat”.

Core Disadvantages:

• Energy Efficiency Bottleneck: As a general-purpose processor, GPUs contain a large amount of complex logic for graphics processing, thread scheduling, and cache management, which are redundant overheads for pure AI computing. Therefore, when executing specific neural network operations, their computational performance per unit power consumption (energy efficiency ratio) is often lower than that of dedicated chips.
• High Costs: Top-tier GPUs (such as NVIDIA H100/B200) used for data center training are extremely expensive due to their complex architecture and cutting-edge manufacturing processes, becoming one of the main costs of AI development.

Typical Scenarios: Large-scale model training in cloud data centers (especially LLMs), complex AI model research and development, high-performance computing clusters.

2. TPU (Tensor Processing Unit): The “Specialist” Born for Deep Learning

Architectural Principles: The TPU is a dedicated integrated circuit (ASIC) designed by Google to accelerate its internal deep learning workloads. Its architectural essence lies in the “Systolic Array”. It can be imagined as an efficient data processing pipeline: data is input from one end of the array and flows through a row of processing units (multiply-accumulators) in a heartbeat-like manner, performing calculations with pre-loaded weights and outputting results from the other end. This design greatly reduces the need for access to main memory, achieving extremely high data reuse rates and computational efficiency.

Core Advantages:

• Extreme Performance and Energy Efficiency: Because it is tailored for the core operation in neural networks—large-scale matrix multiplication—TPUs strip away all non-essential hardware logic. When executing such specific tasks, their performance (TOPS) and energy efficiency ratio (TOPS/Watt) can far exceed that of contemporary general-purpose GPUs.
• Software-Hardware Co-Optimization: TPUs are deeply integrated with Google’s own deep learning frameworks TensorFlow and JAX, achieving end-to-end optimization from compiler to hardware, allowing efficient mapping of high-level algorithm descriptions to the systolic array for execution.

Core Disadvantages:

• High Specialization, Low Flexibility: The outstanding performance of TPUs comes from their specialization, which also means they are inefficient at handling non-tensor operations or irregular computational tasks. Their ecosystem is relatively closed, primarily serving Google’s frameworks.
• Limited Access Channels: Google does not sell TPU hardware externally but offers it as a core competitive advantage through its Google Cloud Platform (GCP) as a cloud service to external users.

Typical Scenarios: Large-scale AI applications within Google (search ranking, YouTube video recommendations, Google Translate, etc.), AI model training and inference services on the GCP cloud platform.

3. NPU (Neural Processing Unit): The “Energy Pioneer” for End Devices and Edge Computing

Architectural Principles: NPU is a broad concept that refers to various processors designed to accelerate neural network computations, typically also an ASIC. Unlike TPUs, which focus on large-scale cloud computing, NPUs are designed to efficiently execute AI inference tasks on end devices or edge devices with constraints on power consumption, cost, and size. Their architecture often adopts a “Dataflow” model, with hardware structures customized according to specific network models (such as CNNs, Transformers) to maximize data reuse in on-chip caches (SRAM) and minimize slow data exchanges with external DRAM, thus achieving ultra-low power consumption.

Core Advantages:

• Top Energy Efficiency Ratio: NPUs have the highest energy efficiency ratio among the three. They can complete complex AI inference tasks at power levels of just a few watts or even milliwatts, which is crucial for battery-powered mobile devices.
• Low Latency: Dedicated hardware computation paths ensure rapid responses for AI tasks, making them very suitable for real-time processing scenarios, such as computational photography in smartphones and obstacle detection in autonomous driving.

Core Disadvantages:

• Extremely Low Flexibility: Most NPUs are deeply optimized for a limited number of fixed neural network models. When new algorithms with significantly different structures emerge, NPUs may not run efficiently or may not run at all.
• Fragmented Ecosystem: Almost every chip manufacturer, including Apple, Qualcomm, MediaTek, and Huawei HiSilicon, has its own NPU architecture and accompanying software development kits (SDKs). The lack of a unified programming standard leads to high development costs as application developers need to adapt to different platforms.

Typical Scenarios: Smartphones (photo enhancement, facial recognition, voice assistants), smart driving (perception and decision-making), smart home (device interconnection), security cameras (human detection), and various edge and end devices.

Key Points Summary

• GPU is the “general-purpose knife”, dominating the AI training field with its CUDA ecosystem, but energy efficiency is its shortcoming.
• TPU is the “surgical scalpel”, providing extreme performance for specific deep learning tasks within the Google ecosystem, but its ecosystem is closed and applications are limited.
• NPU is the “energy expert”, focusing on end and edge inference, with extreme energy efficiency as its core advantage, but it has low flexibility and a fragmented ecosystem.

4. Summary Comparison: Understanding the Differences Among the Three with One Diagram

Comparison Dimension	GPU (Graphics Processing Unit)	TPU (Tensor Processing Unit)	NPU (Neural Processing Unit)
Core Architecture	Single Instruction, Multiple Threads (SIMT)	Systolic Array	Dataflow / Dedicated ASIC
Design Philosophy	General Parallel Computing	Extreme Acceleration for Tensor Operations	Extreme Energy Efficiency for Specific Network Inference
Generality/Flexibility	Very High	Low	Very Low
Performance Focus	Training & Inference (Especially Good at Training)	Training & Inference (Specific Tasks)	Inference
Energy Efficiency Ratio	Medium	High	Very High
Ecosystem	Extremely Mature, Open (CUDA)	Closed (Google Ecosystem)	Highly Fragmented (Each Manufacturer Builds Its Own)
Typical Application Scenarios	Cloud Training, HPC	Ultra-Large-Scale Cloud Services	End Devices, Edge Computing
Cost	High (Especially Training Cards)	Reflected in Cloud Services, Hardware Cost Unknown	Integrated into SoC, Lower Individual Cost

Part Two: Global Market Landscape: The “Moats” and Strategic Layouts of Giants

The differences in technical architectures directly shape the distinct market landscapes of the three tracks. Giants have built deep “moats” around their respective advantages.

1. GPU Dominance: NVIDIA’s Ecological Empire

Main Players: NVIDIA (absolute dominance), AMD, Intel (challenging followers).

NVIDIA Competitive Analysis: NVIDIA’s hegemony in the AI chip market goes far beyond the hardware itself. Its core competitiveness is a solid structure composed of three parts:

1. Leading Hardware: From A100, H100 to the latest B200 platform, NVIDIA has defined the “gold standard” for AI training performance through continuous architectural innovation and performance iteration. Its products have become the benchmark against which all other solutions are measured.
2. CUDA Moat: This is NVIDIA’s true barrier. The CUDA platform abstracts the complexity of underlying hardware, providing developers with a stable, efficient, and feature-rich programming environment. Millions of developers worldwide, along with nearly all top universities and research institutions, base their AI research and development on CUDA, creating a powerful network effect. Any new entrant must not only produce hardware with comparable performance but also persuade the entire ecosystem to migrate, which is a daunting cost and challenge.
3. Full-Stack Layout: NVIDIA’s strategy has long surpassed the chip itself. It offers integrated servers (DGX/HGX systems) that combine GPUs, networking (NVLink/InfiniBand), and software, and has built the NVIDIA AI Enterprise software platform and cloud services on this foundation, providing end-to-end solutions from hardware to applications, further locking in customer value.

AMD/Intel Strategic Analysis: AMD, with its ROCm platform, and Intel, promoting oneAPI, are both attempting to challenge CUDA’s closed ecosystem through an “open standards” strategy. They hope to build a software stack compatible with multiple hardware vendors to attract developers. However, due to the significant gap in ecosystem maturity, toolchain completeness, and market first-mover advantage, they still find it difficult to shake NVIDIA’s position in the high-end AI training market, focusing more on finding opportunities in specific markets or mid-to-low-end fields.

2. TPU’s Exclusive Track: Google’s “Self-Manufactured and Self-Sold” Model

Main Player: Google.

Competitive Analysis: Google’s TPU strategy is unique, with its core serving its own needs.

• Extreme Internal Synergy: Google has the largest and most complex AI workloads globally, such as search ranking, YouTube video recommendations, and Google Translate. The primary goal of self-researching TPUs is to reduce costs and improve efficiency for these massive businesses, freeing itself from reliance on external suppliers and ensuring the security of computing power supply for its core businesses.
• Cloud Service Differentiation: In the public cloud market, AWS and Azure occupy leading positions due to their first-mover advantage. Google uses TPUs as a core differentiating weapon for GCP, providing an attractive alternative for customers (such as AI startups) who are extremely sensitive to AI training performance and costs, thus carving out a niche market in the fierce cloud competition.

Strategic Analysis: The closed model of “not selling hardware externally, only providing cloud services” is a double-edged sword. The advantage is that it maintains the exclusivity and leading nature of the technology, making it difficult for competitors to imitate. The downside is that it limits the market ceiling for TPUs, with its influence primarily confined to the GCP ecosystem, unable to cover the entire industry like NVIDIA.

3. The Fragmented Battlefield of NPU: Kings of Mobile and Edge Computing

Main Players: Apple (Neural Engine in A/M series chips), Qualcomm (AI Engine in Snapdragon chips), MediaTek (APU in Dimensity chips).

Competitive Analysis: The NPU market is highly fragmented, with the core of competition not being single-point computing power but rather scenarios and experiences.

• SoC Integration Advantage: NPUs are not standalone chips but are integrated as core IP modules within the SoCs (System on Chip) of smartphones or automotive cockpits. Their competitiveness lies in the efficient collaboration with other modules such as CPUs, GPUs, and ISPs (Image Signal Processors).
• Scenario Definition Capability: The true winners are those companies that can deeply understand end application scenarios. For example, Apple’s Neural Engine is deeply integrated with its iOS system and camera algorithms, achieving smooth “killer applications” such as portrait mode and cinematic mode, greatly enhancing user experience. Qualcomm also leverages its AI Engine in areas like connectivity and audio processing to establish advantages.

Strategic Analysis: This is a vast but highly fragmented market. Each company’s strategic core is to establish product barriers and brand loyalty through unique features and experiences enabled by NPUs within their dominant ecosystems (such as iOS vs. Android) and device domains (smartphones, automobiles, IoT).

Part Three: The Rise of Chinese Power: The Pursuit and Breakthrough of Domestic AI Chips

In the face of fierce competition in the global AI chip market and external environmental challenges, Chinese companies are actively laying out in three directions: GPU, NPU, and TPU-like chips, carving out a challenging yet opportunity-filled path of pursuit.

1. Domestic GPU Track: Struggling to Break Through the “CUDA High Wall”

Main Players: Biren Technology, Moore Threads, Jingjia Micro, etc.

Competitive Assessment:

• Hardware Level: In recent years, some domestic GPU startups have released products whose theoretical peak computing power (FP32, FP16/BF16) can match the previous generation of international mainstream manufacturers. This is a significant improvement. However, there is a huge gap between theoretical peak computing power and actual application performance, involving a series of engineering challenges such as compiler efficiency, driver stability, performance under large-scale clusters, and chip manufacturing yield. Domestic GPUs still have a long way to go in these areas.
• Software Ecosystem Level: This is the biggest and most fundamental shortcoming faced by domestic GPUs. NVIDIA’s CUDA ecosystem is its most solid “high wall”. Domestic manufacturers generally choose a strategy compatible with CUDA while building their own software stacks (such as Biren’s BIRENSUPA, Moore Threads’ MUSA). However, achieving high compatibility with the CUDA ecosystem, attracting developers to migrate, and establishing a complete toolchain and community requires long-term, substantial investment, which is not an overnight task.

Challenges and Opportunities: Domestic GPUs face dual challenges of limited access to advanced processes and the suppression of the CUDA ecosystem. However, their greatest opportunity lies in China’s vast domestic market and national policy support. By forming application demonstrations in specific industries (such as intelligent computing centers and government-enterprise innovation), they can build a base of “exchanging applications for ecosystems”, gradually refining products and improving software, which is a realistic path to break through the “CUDA high wall”.

2. Domestic NPU/AI-Specific Chip Track: A Hundred Flowers Blooming, with Scenarios as Kings

This is the area where domestic AI chips have achieved the most significant results and competitiveness, showing a dual-winged approach of cloud and edge.

Cloud Players: Huawei Ascend, Cambricon (MLU), Suiyuan Technology, Tensynse.

• Competitive Assessment: Represented by Huawei Ascend, China’s cloud AI chips have formed a strong systemic competitiveness. Huawei not only provides Ascend series chips (such as Ascend 910) but also builds a full-stack solution from the underlying chip enabling library CANN, AI framework MindSpore, to upper-layer application enabling platforms. This “software-hardware synergy” capability has enabled large-scale deployments in government and enterprise markets such as telecommunications, finance, energy, and transportation, forming a relatively independent ecological cycle. Cambricon, Suiyuan Technology, and others are also deeply cultivating their respective niches (such as inference and training) and have accumulated a considerable customer base.
• Challenges: The main challenge lies in how to compete with GPUs in terms of generality and how to break the inertia of industry customers’ long-term reliance on NVIDIA solutions.

Edge Players: Horizon Robotics (Journey series), Black Sesame Intelligence (Huashan series), Rockchip, etc.

• Competitive Assessment: Domestic edge NPU manufacturers have achieved rapid development through deep binding with vertical industries, especially in the smart automotive sector. For example, Horizon Robotics not only sells its Journey series chips but also provides accompanying algorithms, toolchains, and development support, helping automotive companies quickly realize advanced driver-assistance systems (ADAS) functions. This “chip + solution” model greatly lowers the development threshold for customers, allowing it to occupy a leading market share in the smart driving pre-installation market.
• Opportunities: The demand for edge AI scenarios is extremely diverse (automotive, security, industrial, consumer electronics), making it difficult for any one company to dominate. This provides numerous opportunities for domestic manufacturers to enter niche markets and establish competitive advantages by defining scenarios and optimizing solutions.

3. The Self-Research Path of Chinese Internet Giants: China’s “TPUs”

Main Players: Alibaba (Hanguang series), Baidu (Kunlun chip), Tencent (Zixiao series).

Strategic Motivation Analysis: Similar to Google’s logic, the primary motivation for Chinese internet giants to self-research AI chips is to serve their internal needs. Facing the massive and high-concurrency AI computing demands brought by their own businesses (e-commerce recommendations, search ranking, advertising placement, content review), self-researching chips can achieve:

1. Cost Reduction and Efficiency Improvement: Optimizing hardware for their core algorithms to achieve better cost-effectiveness and energy efficiency than general-purpose GPUs.
2. Supply Chain Security: Ensuring autonomous control and stable supply of core computing power in the current international situation.
3. Technological Iteration: Rapidly feeding back business needs into chip design to achieve collaborative evolution of algorithms and hardware.

Competitive Assessment: The greatest advantage of these giants lies in having the most authentic and vast application scenarios to “feed” and iterate their chips. Currently, these chips support internal businesses while gradually providing computing power services externally through their respective public cloud platforms (such as Alibaba Cloud, Baidu Smart Cloud), forming a model of “internal digestion + external empowerment”. Their competitiveness is mainly reflected within their cloud ecosystems and is an important part of their cloud service differentiation.

Part Four: Future Outlook: Integration, Differentiation, and New Paradigms

Looking ahead, the technological evolution and market landscape of AI chips will present a complex situation of coexistence between integration and differentiation.

1. Technical Trends: From “Single Champion” to “Heterogeneous Integration”

Chiplet Technology: As Moore’s Law slows, the design and manufacturing costs of large monolithic chips are becoming increasingly high. Chiplet technology has become the key to breaking through. It allows different processes and functions of “chiplets” (such as CPU cores, GPU cores, NPU cores, I/O modules) to be packaged together like Lego blocks, forming a system-level chip. This means that future AI chips will no longer be purely GPUs or NPUs but rather a highly integrated heterogeneous computing platform. For example, a chip could integrate a powerful GPU die for training and multiple high-efficiency NPU dies for inference, thus achieving the best balance of performance, cost, and flexibility.

Software-Defined Hardware: In the heterogeneous era, the importance of software will be unprecedentedly highlighted. How to seamlessly and efficiently deploy upper-layer applications onto diverse hardware (GPU, NPU, CPU, etc.) through advanced compilers and unified software stacks will become core competitiveness. Future competition will shift from a simple hardware performance contest to an ecological contest of “hardware + software” collaborative optimization.

2. Market Trends: The Division and Cooperation of Cloud Training and Edge Inference

Cloud Training Market: In the foreseeable future, due to its unparalleled versatility and mature CUDA ecosystem, GPUs will continue to dominate the large model training market. However, as models and application scenarios solidify, Google’s TPUs and various cloud-specific ASICs (including products from Chinese manufacturers) will continue to erode GPU’s market share in ultra-large-scale scenarios with high demands for cost and energy efficiency.

Inference Market (Cloud + Edge): This is the largest incremental market for future AI chips and the true “battleground”. The market will be highly differentiated:

• Edge Side: NPUs and various dedicated ASICs will dominate due to their extreme energy efficiency advantages. From smartphones to automobiles to IoT devices, everything needs AI, and everything needs low-power NPUs.
• Cloud Side: The diversity of inference tasks (from latency-sensitive to throughput-intensive) will provide coexistence space for various architectures. GPUs, TPUs, and specialized inference chips (such as NVIDIA’s L4/T4 series and domestic inference chips) will jointly share the market based on cost-effectiveness and business needs.

3. Exploration of New Paradigms: Beyond the Von Neumann Vision

As a long-term outlook for the future, academia and industry are exploring new paradigms that could disrupt the current computing architecture to fundamentally address the “Von Neumann bottleneck” (the “storage wall” caused by the separation of processors and memory) and power consumption issues.

Overview of Cutting-Edge Technologies:– Processing In Memory (PIM)/Compute In Memory (CIM): Integrating computing units directly into memory to achieve in-place computation at the storage location, completely eliminating the overhead of data movement. – Neuromorphic Computing: Mimicking the structure and information processing methods of biological brains (such as spiking neural networks) to build event-driven, ultra-low-power computing systems. – Analog Computing: Utilizing the physical properties of transistors (such as voltage and current) to perform mathematical operations, potentially achieving orders of magnitude improvement in energy efficiency for specific computations.

Most of these technologies are currently in research or early prototype stages, but they represent the long-term evolutionary direction of AI chips and the ultimate hope for achieving artificial general intelligence (AGI) at the computing power level.

Conclusion: The Changes and Constants in the Chess Game of Computing Power

Through a systematic analysis of GPUs, NPUs, and TPUs, we can see a dynamically evolving chess game of AI computing power. In this game, there are both unchanging iron laws and changing patterns.

The unchanging fact is the insatiable thirst for computing power in artificial intelligence. Whether it is the scale of models, the dimensions of data, or the diversity of applications, the demand for stronger, more efficient, and more economical computing capabilities continues to drive the development of the entire AI chip industry.

The changing aspect is the paths to achieving computing power and the market landscape. It has been proven that no single chip architecture can “dominate” all scenarios. The versatility of GPUs, the specialization of TPUs, and the energy efficiency of NPUs determine that they will each play indispensable key roles at different levels in the cloud, edge, and end, in both training and inference processes. The future will inevitably be an era of heterogeneous computing with multiple architectures coexisting and collaborating.

From a global perspective, NVIDIA’s dominance built on its GPU hardware and CUDA ecosystem is unlikely to be shaken in the short term, especially in the fields of AI research and large-scale training. However, the vast inference market and diverse dedicated scenarios leave ample room for innovation and competition for its competitors, including Google, AMD, Intel, and numerous ASIC companies.

From a Chinese perspective, the path of domestic AI chip breakthroughs shows a clear asymmetric characteristic. In the fields of NPUs and dedicated ASICs, especially when deeply bound to vertical application scenarios such as smart vehicles, security, and government-enterprise, Chinese manufacturers have demonstrated strong product definition capabilities and market competitiveness. However, in the most challenging GPU track, while hardware catch-up is important, building a software ecosystem is the decisive “Long March” to success.

Ultimately, computing power is the oil of the new era, and the innovation of chip architecture and the competition of ecosystems will continue to define the boundaries and depth of this grand intelligent revolution.

(Note: This article was generated by the Tiangong Super Intelligence System, and the views are for reference only and do not constitute any investment advice.)