As the AI wave sweeps across the globe, NVIDIA, as the dominant player in the chip market, currently holds an 80% market share in AI chips. However, its main competitors, such as AMD HW and other chip manufacturers, are rapidly rising and capturing NVIDIA’s AI chip market. In addition to these mainstream chip manufacturers competing for the AI chip market, companies like Microsoft and Google are also launching their own self-developed ASIC chips.Google’s seventh-generation TPU, Ironwood, matches the performance of NVIDIA’s B200, while the power consumption of the chip is also soaring. This year, Google’s TPU has exceeded 600W per chip, with a cluster power reaching up to 10MW.
In terms of liquid cooling, after multiple experiments and iterations, Google officially entered the liquid cooling TPU field in 2018. Since then, the company has continued to develop and improve its cooling designs. Their current liquid cooling solutions are designed for data center scale, with cooling loops spanning across racks rather than being limited to inside servers.
1. Cold Plate DesignSplit-flow cold plates: These divide the fluid into multiple paths and spray it evenly onto hot spots, avoiding preheating and uneven cooling caused by the “first in, last out” effect. The advantage is that hot spots are cooled more precisely, with lower thermal resistance and smaller temperature differences; the downside is that the design and manufacturing are more complex, requiring high standards for distributors and valves.Bare-die direct cooling: This removes the packaging top cover and places the cold plate directly against the chip. This significantly reduces thermal resistance but introduces challenges in assembly and protection (mechanical stress, electrical isolation, maintenance, and replacement costs).
2. CDU and Pumps: System-level design emphasizing hot-swappability and redundancyGoogle has designed several CDUs to be “hot-swappable” during operation, typically in an N+1 or 5+1 configuration, ensuring that the system remains online during maintenance of any unit. The selection of pumps, head, and cavitation prevention are all emphasized. The industry trend is to choose shielded pumps / non-mechanical seal pumps (to reduce leakage risks) and implement closed-loop control for flow and pressure.
3. Connectors and MaintenanceOn-site maintenance requires that “disconnection is clean,” so quick connectors must achieve dry-break or at least minimize leakage. Additionally, a complete set of liquid leak sensors, automatic isolation valves, trays, and leak path designs are implemented to enable immediate detection and localized isolation.4. Control SystemTemperature-pressure-flow closed-loop coordination is essential; liquid cooling does not work like air cooling where “turning on the fan creates airflow.” The temperatures at both ends (chip side and CDU side), supply and return water differences, flow rates, pump speeds, and valve positions must be coordinated, especially during load transients (when hundreds of chips simultaneously increase/decrease load), to ensure the system does not experience dry-out or cavitation. Google has integrated these into automated control, along with strategies (such as bypass, throttling, and splitting) to buffer sudden changes.
Follow us for more exciting content