The “New Normal” of the Chip War: The Rise of Google’s TPU and NVIDIA’s “Anxiety”
Introduction: A Trillion-Dollar Market Shock Triggered by “Rumors”
Recently, a report surfaced that tech giant Meta is in talks with Google to deploy Google’s self-developed Tensor Processing Units (TPUs) in its data centers starting in 2027. This news struck the capital market like a thunderclap, causing a significant shock. The global AI chip leader NVIDIA saw its stock price plummet, with its market value evaporating by nearly a trillion dollars at one point. Of course, the reasons behind NVIDIA’s stock decline are multifaceted, including fears of an AI bubble. Even after NVIDIA released an impressive earnings report, some major players warned that profitability does not guarantee a clear future outlook, cautioning against repeating the painful lessons of Cisco during the 2000 tech bubble. Today, we will not discuss the reasons from a capital market perspective but will analyze the current state and landscape of the computing power market from a product perspective.
As of today, with the concept of artificial intelligence gaining traction, the competition in the AI computing power field has entered a new and more brutal phase. When super buyers like Meta start comparing options and even consider sourcing custom chips from Google, a close ally of NVIDIA, it forces everyone to rethink: What is the magic of Google’s TPU? Can it truly shake NVIDIA’s throne of GPUs?
Below, I will compare Google’s latest TPU v6e (codenamed Trillium) with NVIDIA’s flagship product H100 GPU from three dimensions: technical architecture, economic benefits, and ecosystem, and attempt to reveal the underlying logic and future trends behind this event.
1. Technical Deep Dive: The Specialization of ASICs vs. the Generality of GPUs
As previously analyzed, part of NVIDIA’s stock price crash stems from market considerations of the performance and cost advantages of its competitor—Google’s TPU. Let’s start by analyzing the differences between the two chips from the underlying architecture and the advantages of TPU.
1. Core Differences: Systolic Array vs. General Parallelism
The core of Google’s TPU is the Systolic Array. This is a hardware structure tailored for matrix multiplication operations (which can be understood as the core operation in deep learning). Data flows through the array like blood, with each Processing Element (PE) continuously performing multiply-accumulate operations. This design brings two key advantages:
- Extreme Computational Density: Unlike general GPUs that need to reserve a large number of transistors for graphics rendering and general computing tasks, TPUs allocate almost all resources to matrix operations, achieving an extremely high TOPS/Watt (trillions of operations per watt) efficiency ratio.
- Efficient Interconnect Topology: TPU Pods utilize custom Optical Circuit Switching (OCS) technology and 2D Torus interconnect topology, allowing thousands of TPU chips to be connected into a massive computing cluster, with an All-Reduce operation speed that is 10 times faster than Ethernet-based GPU clusters.
TPU Architecture Diagram
In contrast, NVIDIA’s GPUs, such as the H100, while accelerating AI computations through their Tensor Cores, are fundamentally still general-purpose parallel processors. Their advantages lie in:
- Fungibility: The H100 can run almost all AI models and also handle high-performance computing (HPC) and graphics rendering tasks. This flexibility is unmatched by ASIC chips.
- Ecological System (CUDA): NVIDIA has built a large and robust developer ecosystem with its mature CUDA software stack. Almost any new AI model or framework will choose CUDA as the preferred runtime environment.
| Feature | Google TPU v6e (Trillium) | NVIDIA H100 GPU | Deep Analysis |
|---|---|---|---|
| Core Architecture | Systolic Array | Tensor Core (General Parallel) | Specialization vs. Generality determines the applicable range and efficiency ratio. |
| Interconnect Technology | OCS + 2D Torus | NVLink + InfiniBand | TPU has advantages in ultra-large-scale cluster communication. |
| Software Ecosystem | JAX, TensorFlow, PyTorch/XLA | CUDA, PyTorch, TensorFlow | CUDA ecosystem maturity far exceeds that of TPU. |
| Applicable Scenarios | LLM training, recommendation systems, specific AI inference | Almost all AI tasks, HPC, graphics rendering | TPU is suitable for specific workloads within the Google ecosystem. |
2. Performance and Cost: The Temptation of 4x Cost-Effectiveness
In this market with high demand for computing power, facing the “burning money” style of energy consumption, reducing computing power costs is increasingly important. The TPU seems to provide an answer in this regard. According to public data from Google and its partners, the TPU v6e can offer up to 4 times the performance per dollar compared to the H100 under specific workloads.
- Training Speed: The training speed of the BERT model on TPU is 2.8 times faster than on A100; the training time for the T5-3B model has been reduced from 31 hours to 12 hours.
- Inference Cost: After migrating to TPU, Midjourney’s inference costs decreased by 65%.
This enormous cost advantage is clearly irresistible for giants like Meta, which have vast user bases and extensive AI infrastructure. If Meta can migrate tens of thousands of inference workloads to the more cost-effective TPU, the annual savings will be astronomical.
2. Market Attribution Analysis: The Game of Tech Giants from the Buyer’s Market Perspective
NVIDIA’s stock price crash is superficially a technical competition, but at a deeper level, it signals the shift of the AI computing power market from a seller’s market to a buyer’s market.
Self-Rescue and Checks Among Tech Giants
In recent years, NVIDIA has leveraged its monopoly on GPUs to control the new oil of the AI era—computing power. However, this monopoly is essentially fatal for tech giants:
- Cost Control: The costs of purchasing and leasing NVIDIA GPUs continue to rise, becoming one of the largest expenses for giants.
- Supply Chain Risks: Computing power supply is constrained; if NVIDIA’s production capacity is limited, all AI businesses will come to a halt.
Therefore, giants like Google, Meta, and Amazon (Trainium) are investing heavily in developing their own ASIC chips, which is less about surpassing NVIDIA and more about self-rescue and checks and balances.
In my view, Meta’s consideration of adopting TPU is not a complete abandonment of NVIDIA but rather a strategic hedge. It sends a clear signal to the market and NVIDIA: We have alternatives; you are no longer the only choice. Having options means less constraint and a balance of power in the supply-demand relationship. This game will ultimately lead to a healthier and more diversified AI computing power market.