Translator’s Note

In today’s world, where the demand for AI computing power is growing exponentially, the bottleneck of computing power has gradually shifted from the pure computing capability of chips to the communication efficiency between chips. NVIDIA has built a strong moat with its NVLink and InfiniBand technologies. Huawei’s UB-Mesh, showcased at Hot Chips, is an ambitious attempt to fundamentally challenge the existing architecture. The “Unified Bus” concept directly addresses the current chaos of multiple protocols such as PCIe, Ethernet, and CXL coexisting within data centers, aiming to reduce latency and complexity. More critically, its proposed “sub-linear” cost growth model precisely targets one of the biggest pain points in building large-scale AI clusters—high network costs. This is not just a technology release but also a profound reflection and layout by Huawei on the future shape of AI data centers. Understand Google’s Ironwood TPU: How powerful is the “performance monster” with 9216 chips?

Huawei Unveils UB-Mesh Interconnect Technology for Large AI Supernodes at Hot Chips 2025

Author: Ryan Smith – August 26, 2025
Original link: https://www.servethehome.com/huawei-presents-ub-mesh-interconnect-for-large-ai-supernodes-at-hot-chips-2025/

The third and final machine learning-related presentation before lunch came from Huawei. Unlike many other machine learning vendors who came to promote products, Huawei’s presentation focused more on foundational technology. Specifically, it discussed how to efficiently utilize a mesh architecture to interconnect chips in large AI systems.

Focusing on the so-called “SuperNodes”—single supercomputing clusters with over a million chips—Huawei is showcasing its United Bus mesh (UB-mesh) technology. The core challenge currently faced is how to scale the network to provide low-latency connections for all chips within the supernode while avoiding network equipment costs exceeding those of the accelerator chips themselves, all while maintaining extremely high reliability in interconnections. Hot Chips 2025’s biggest dark horse: d-Matrix deep dive, is the future of AI inference not in video memory, but in memory?

This presentation deviated from the norm at Hot Chips, being an online virtual presentation. Dr. Liao delivered this talk remotely from China.

This presentation leaned more towards architecture and computer science rather than pure engineering technology. Huawei aims to build a mesh network that can scale to millions of processors. At such a scale, the size of a supernode could be equivalent to an entire data center.

The enormous scale of the system means that it is not just about connecting GPUs; memory pools, SSDs, network cards, and switches all become part of this unified node.

Huawei advocates for a unified bus that replaces the current mishmash of various specific technology protocols with a single protocol. A general protocol means that any port can connect to any other port without the need for protocol conversion. This effectively reduces latency by eliminating those steps that could introduce delays.

The unified protocol will also allow for a more simplified architecture. RISC-V strikes back! A 50-person team creates the Cuzco god chip, aiming to dethrone Intel and ARM?

Even with these grand goals, the UB (Unified Bus) can still operate over Ethernet.

However, to achieve these goals, several challenges must be overcome. One particularly noteworthy issue is that physical links become longer—the network needs to span the entire data center—meaning fiber connections must be used. The error rate of optical networks is 2 to 3 orders of magnitude higher than that of electrical connections. This necessitates the development of superior error recovery technologies at higher layers.

At the same time, the enormous scale means that the entire node must be able to withstand node failures. At such a large scale, the failure of a single server is no longer a question of “if it will happen” but rather “when it will happen.”

How to build a physical network that increases bandwidth by 100 times while keeping costs far below 100 times?

Huawei believes this requires a completely new topology. It can be described as a hybrid topology that integrates the advantages of various network types at different levels.

One possibility that Huawei is exploring is the integration of three technologies: using a CLOS architecture at the highest level, an n-dimensional mesh suitable for scales from a single rack to dozens of nodes below it, and then using a lower-cost option of an n-dimensional sparse mesh.

The training of large language models (LLMs) has reached five-dimensional parallelism.

This is a conceptual diagram of the UB-mesh topology. It is implemented across multiple dimensions. Any node within each dimension has full connectivity. Higher dimensions are then responsible for connecting lower dimensions.

All these designs need to be balanced against costs. You definitely do not want the cost of network equipment to exceed that of the computing devices that actually perform the calculations.

As the scale of the network expands, the costs of traditional networks exhibit super-linear growth. However, the cost growth of UB-mesh is sub-linear, meaning that even with a significant increase in the number of computing nodes, there will only be a moderate increase in costs.

This is a real-world example. A system with 64 nodes adopted a CLOS + 2D-Mesh configuration. 🤣⚡ Linus drops another golden quote: VIBE coding=Very Inefficient But Entertaining!

However, how can the reliability of optical links be sufficient to meet the demands of supernodes?

It is necessary to enhance the resilience of the optical links themselves. First, support link-level retries on backup optical links within the same optical module to ensure that packets do not traverse problematic paths again.

A second solution for more severe failures is to cross-connect the MAC (Media Access Control layer) to multiple optical modules, so that even if one optical module fails, there is still a good module available.

Huawei’s goal is to increase the mean time between failures (MTBF) by 100 times. One way to achieve this is to provide hot spare racks that can take over the work of a node when it fails. The failed rack is then repaired and reintroduced as a new hot spare rack. Additionally, if the rack itself has extra spare chips built in, it can also have a certain level of resilience; in this case, it can return to the system as a “weak hot spare.”

In summary, by shifting to a unified protocol and making multiple improvements to network topology and hardware redundancy, UB-mesh will make it possible to build and deploy reliable supernodes at data center scale. So, who wants a 1GW (gigawatt) level AI data center? Intel’s bold gamble: teaming up with Samsung to take on TSMC, what do you think?

Colliding Sparks of Ideas

After viewing Huawei’s vision for the future AI supernode network architecture, I’m sure you have many thoughts. Feel free to share your views below, and let’s discuss the future of technology! In 2025, who will still be coding for floppy disks? The “living fossil” sentiment of the open-source world.

1. 1. UB-Mesh vs. NVLink/InfiniBand: In your opinion, what are the core advantages and potential challenges of Huawei’s UB-Mesh solution compared to NVIDIA’s NVLink/NVSwitch or the industry’s mainstream InfiniBand/RoCE solutions?
2. 2. The Future of the “Unified Bus”: The concept of a “Unified Bus” is very appealing; will it become the mainstream architecture for future data centers? What do you think will be the biggest obstacles to achieving this goal? (e.g., industry standards, ecosystem support, etc.)
3. 3. Trade-offs Between Cost and Reliability: In your experience building or managing large-scale computing clusters, have network costs and reliability been major pain points? What do you think is the likelihood of achieving Huawei’s goals of “sub-linear costs” and “hundredfold reliability” in the real world?