“Training a large model costs tens of millions of dollars in computing power,” “GPU server prices have doubled and are still hard to obtain,” “Half of the GPUs in the cluster are idle but cannot be scheduled”—this reflects the most genuine computing power anxiety in the current AI industry.
As the industry struggles with computing power bottlenecks, three major technological directions have proposed significant solutions: Huawei’s newly released Flex:ai AI container software breaks resource barriers with a 10% precision computing power partition, increasing NPU/GPU resource utilization by 30%; the Aegaeon multi-model service system proposed by Peking University and Alibaba’s team achieves an 82% savings in GPU resources; while the supernode technology represented by NVIDIA DGX creates a “performance aircraft carrier” for large model training through hardware aggregation.
On one side is the supernode’s “hardware performance stacking” to the right, and on the other is virtualization’s “software resource partitioning” to the left, while Huawei’s Flex:ai and Alibaba’s Aegaeon have found a path of integrated innovation in between—they not only reconstruct the logic of AI computing power usage but also provide new possibilities for cost reduction and efficiency improvement for enterprises. This article will deeply analyze these three major technologies to clarify the optimal solution for computing power management under the paradigm of “virtualization to the left, supernodes to the right.”
Before diving into the main content, let’s briefly introduce the two core technologies: GPU virtualization and supernodes:
GPU Virtualization: A processing technology that allows multiple virtual machines to share physical GPU resources through virtualization technology. NVIDIA’s MIG solution can partition a single GPU into up to 7 separate GPUs, allowing 7 users to simultaneously utilize the resources of one GPU card; AMD’s SRIOV can partition a single GPU into up to 8 independent GPUs.
Supernodes: Scalable computing units formed by integrating multiple computing chips (such as GPUs or NPUs) through high-speed interconnect technology, primarily aimed at solving the issues of computing power collaboration and efficiency in training large AI models. Vertical scaling maximizes communication bandwidth within the cluster and minimizes latency between dense computing resources (e.g., GPUs), while horizontal scaling enables longer-distance communication within data center infrastructure.
1. Core Breakthroughs of Flex:ai, Aegaeon, and Supernodes
Whether it is the hardware aggregation of supernodes, the computing power pooling of Flex:ai, or the fine scheduling of Aegaeon, the essence is to solve the problem of “low GPU resource utilization,” but the technical paths of the three show a clear distinction of “rightward, integration, and leftward refinement.”
1. Huawei Flex:ai: Cross-architecture “Computing Power Pooling Engine”
Flex:ai is not just a simple NPU/GPU management tool, but a “XPU computing power scheduling operating system” built on Kubernetes, with its core breakthrough being the dual capability of “computing power partitioning + global pooling,” completely breaking the boundaries of hardware and nodes.
Two Core Technologies of Flex:ai
1. Fine Computing Power Partitioning: Partitions a single GPU/NPU into virtual computing power units with 10% precision, allowing a single card to simultaneously carry multiple AI workloads, with the computing power ratio dynamically adjustable, solving the waste issue of “a single card can only run one model.”
2. Global Computing Power Aggregation: Summarizes the idle XPU computing power of all nodes in the cluster into a “shared computing power pool,” achieving global scheduling across nodes and architectures, allowing scattered idle computing power to be efficiently utilized.
Notably, its differentiated advantages include: first, full architecture compatibility, capable of scheduling NVIDIA GPUs, adapting to Ascend NPUs, and third-party chips, breaking the ecological monopoly of hardware vendors; second, an open-source strategy, with releases synchronized to the Model Engine community (https://gitcode.com/ModelEngine), significantly reducing enterprise migration costs, making Flex:ai highly competitive in mixed computing environments.
2. Alibaba Aegaeon: Token-level “Model Scheduling Tool”
The core value of Aegaeon is reflected in its published technical papers—achieving extreme efficiency in sharing GPU resources among multiple models through “token-level dynamic scheduling.” This is a disruptive reconstruction of the AI model service process.
In traditional multi-model services, GPU resources are either monopolized by a single model or incur significant performance losses during model switching. Aegaeon creatively lowers the scheduling granularity from “model-level” to “token-level” (where a token is the basic unit of AI text processing), akin to upgrading a parcel sorting system from “batch sorting” to “real-time scheduling by individual parcels.”
Aegaeon addresses resource wastage in multi-model services through a layered design of “token-level scheduling, phased computation, cache reuse, and elastic scaling” to ultimately achieve extreme pooling of GPU resources (paper: https://doi.org/10.1145/3731569.3764815)
Core Data and Mechanisms of Aegaeon Paper
• Key Results: The number of GPUs required for 10 models was reduced from 1192 to 213, achieving a resource savings rate of 82%, far exceeding the industry average.
• Core Mechanism: Allocates GPU computing resources in real-time at the token level; includes an automatic scaling module for models that dynamically adjusts resource allocation based on request traffic, ensuring service quality while avoiding resource wastage.
• Performance Optimization: Reduces model switching overhead by 97% through techniques such as preloading model parameters and optimizing memory scheduling, addressing performance losses caused by fine scheduling.
3. Supernodes: Hardware-intensive “Computing Power Behemoth”
If virtualization is the leftward exploration of “software partitioning,” supernodes (SuperPod) represent a typical rightward practice of “hardware aggregation.” It embodies a technology route in computing power management that emphasizes “heavy hardware and strong interconnects,” with the core technology being “high-density integration + low-latency interconnects”: on the hardware level, customized servers (such as NVIDIA DGX GB200) are used, integrating multiple high-performance GPUs in a single node and achieving direct GPU connections; on the network level, high-speed interconnect technologies like InfiniBand are employed to reduce communication latency within and between nodes to microsecond levels, ensuring efficient data flow during large-scale parallel computing.
NVIDIA DGX GB200 NVL72 Rack System
This architecture determines that supernodes are inherently designed for large models—when training models with hundreds of billions of parameters, hundreds or thousands of GPUs need to work in coordination, and the hardware optimization of supernodes maximizes the aggregation performance of the computing power cluster. Taking the industry-leading NVIDIA DGX SuperPOD as an example, its technical details can be divided into three layers:
Three-layer Technical Architecture of Supernodes
• Hardware Layer: Uses customized server nodes, integrating 8 A100/H100 GPUs per node, achieving high-speed direct connections between GPUs within the node through NVLink technology, avoiding data transmission bottlenecks within a single node; at the cluster level, a high-density rack design is adopted, with every 4 nodes forming a “computing unit,” enhancing space utilization and heat dissipation efficiency.
• Network Layer: Equipped with InfiniBand HDR200 high-speed interconnect network, achieving inter-node bandwidth of 200GB/s and latency as low as 1.2 microseconds, more than 10 times that of ordinary Ethernet; through network topology optimization, ensures “consistent communication latency between any two nodes,” maintaining load balancing during large-scale parallel computing.
• Software Layer: Accompanied by the NVIDIA AI Enterprise suite, which includes optimization tools for Model Parallelism and Data Parallelism, automatically splitting large model parameters across different GPUs and distributing training data shards to maximize the utilization of cluster computing power.
Using a 72×1 NVLink topology, this topology includes 72 GPUs distributed within a single NVLink domain. The DGX GB200 rack system contains 18 computing nodes (also known as trays). Each computing tray includes two GB200 super chips, each equipped with two B200 GPUs and one Grace CPU. The computing tray integrates four ConnectX-7 (CX-7) network cards, supporting InfiniBand NDR (400Gbps) connections for cross-rack computing networks, as well as two BlueField-3 (BF3) network cards.
The performance advantages of supernodes are particularly significant in practical scenarios: OpenAI trained GPT-3 using a cluster with a similar supernode architecture, compressing the training cycle from “years” to “months”; a major domestic company built a supernode cluster based on DGX SuperPOD, achieving a 45% improvement in the efficiency of training large models with hundreds of billions of parameters compared to ordinary GPU clusters. However, the extreme performance comes with a high cost—a DGX SuperPOD with 512 A100 GPUs has a procurement cost exceeding 200 million yuan, and annual maintenance costs account for 15%-20% of the procurement cost.
However, the “rightward” characteristics of supernodes also bring significant limitations: first, “rigid resource allocation,” as supernode clusters typically allocate resources at the “whole cluster/whole node” level, meaning that even training a small to medium model requires occupying at least one node (8 GPUs), leading to severe resource wastage; second, “ecological closure,” as DGX SuperPOD only supports NVIDIA GPUs and cannot accommodate domestic chips like Ascend, requiring enterprises to reconstruct the entire supernode cluster if they change hardware systems; third, “limited scalability,” as the network topology and hardware configuration of supernodes are fixed at deployment, requiring nodes to be added according to fixed specifications for subsequent expansions, making it inflexible to adapt to slight increases in computing power demand.
Multi-dimensional Comparison of the Scenario Adaptation Boundaries of the Three Technologies
Combining GPU virtualization technology, we can clearly delineate the applicable scenarios of supernodes, Flex:ai, and Aegaeon based on core dimensions such as “technical route, cost, and efficiency,” clarifying the choice logic of “leftward/rightward/integration.”
“Virtualization to the Left”—pursuing flexible resource reuse through software/hardware partitioning; “Supernodes to the Right”—creating peak performance through hardware aggregation. Huawei’s Flex:ai and Alibaba’s Aegaeon have found an innovative integration path between these two extreme routes. In addition to Flex:ai and Aegaeon, GPU virtualization and supernodes (SuperPod) are also mainstream computing power management solutions, allowing for a comparison across 7 core dimensions to clarify the applicable boundaries of different technologies.
Differentiated Value and Combination Logic of the Three Technologies
When enterprises choose computing power management solutions, it is essentially a comprehensive trade-off of “performance needs, cost budget, and scenario complexity.” Supernodes, Flex:ai, and Aegaeon are not in competition but are complementary solutions covering the entire spectrum of “training-inference-mixed load.” We will analyze their differentiated value and combination usage logic from both technical paths and commercial adaptation perspectives.
1. Technical Path: Rightward for Performance, Leftward for Refinement, and Middle for Balance
The “rightward path” of supernodes is a hardware-driven extreme performance route—building technical barriers through dedicated servers and high-speed interconnect networks, with the core goal of solving the parallel computing bottleneck of models with hundreds of billions of parameters. The advantage of this path is that performance loss is <5%, but the cost is ecological closure and high expenses, making it unsuitable for flexible multi-load scenarios.
Aegaeon’s “leftward refinement” is a software-driven efficiency route—based on virtualization’s “resource partitioning,” it lowers the scheduling granularity from “card-level” to “token-level,” focusing on resource optimization in multi-model inference scenarios. Its advantage is extremely high efficiency in multi-model services, but it has weak generalization and cannot support large model training.
Flex:ai’s “middle balance” is an integration route—absorbing the concept of “computing power aggregation” from supernodes, it uses software to achieve cross-node resource pooling; borrowing from virtualization’s “flexible partitioning” approach, it adapts to multiple loads with 10% precision; while also being compatible with multiple architecture hardware, avoiding the closure and high costs of supernodes, and solving the generalization issues of Aegaeon, making it a “universal adapter” covering most scenarios.
2. Commercial Adaptation: Exclusive for Giants, Precision Empowerment, and Inclusive for All
Supernodes are an “exclusive solution for giants”: suitable only for well-funded companies like Google, Meta, and major domestic firms, with its core value being to seize market opportunities by shortening the training cycle of large models; the procurement cost of 200 million yuan is a “strategic investment” for such enterprises.
Aegaeon is a “precision empowerment solution”: focusing on cloud service providers’ AI API platforms and enterprises’ multi-model deployments, for example, Alibaba Cloud can reduce nearly 900 GPU investments for 10 commonly used models through Aegaeon, saving nearly 100 million yuan in annual maintenance costs, making it the “efficiency champion within the scenario.”
Flex:ai is an “inclusive solution for all”: its open-source strategy lowers the usage threshold for small and medium enterprises, and its cross-architecture features support the reuse of existing hardware; a manufacturing enterprise improved GPU cluster utilization from 35% to 82% through Flex:ai, directly saving 6 million yuan in hardware procurement costs, adapting to the needs of most enterprises from internet giants to traditional businesses.
Left or Right? The Answer Lies in “Demand”
The “rightward” supernodes are the “performance cornerstone” for large model training, Aegaeon’s “leftward refinement” is the “efficiency tool” for multi-model inference, and Flex:ai’s “middle balance” is the “universal adapter” for mixed loads. For enterprises, the optimal strategy is a “combination punch”: using supernodes to support core large model training, Aegaeon to optimize multi-model inference services, and Flex:ai to build a global computing power pool to manage all loads, achieving “performance without compromise, resources without waste” through technological integration.
With the rapid development of AI technology, computing power will become a fundamental resource like water and electricity. The explorations of Flex:ai and Aegaeon are making the vision of “computing power as a service” a reality—enterprises no longer need to worry about computing power, but can focus on their own AI business innovations, leaving the rest to computing power management tools. This may be the ultimate significance of the computing power revolution.
References (Add WeChat friends or send a private message to get)
-
NVIDIA Supernodes: NVIDIA DGX SuperPOD: Next Generation Scalable Infrastructure for AI Leadership.pdf
-
Alibaba Pooling: Aegaeon: Effective GPU Pooling for Concurrent LLM Serving on the Market.pdf
-
NVIDIA MIG: NVIDIA Multi-Instance GPU User Guide.pdf
Follow me!
Let’s
Explore
Technical Principles and Business Logic
One-click Triple Connection
Previous
Selected
Domestic AI Chips are Facing a “Breakthrough Moment” Interpretation of the 2025 Domestic AI Chip Industry White Paper
“Ascend + Kunpeng” Dual Engine Drive: Huawei’s AI Computing Power Strategy Stimulates Resonance Across the Entire Industry Chain
Google Gemini 3 Pro: The Ultimate Reflection of AI Applications and TPU Hardware-Software Collaborative Design
Major Model Giants are Developing AI Chips: The Hard Battle Behind the Computing Power Competition
Cambricon and Ascend Day 0 Adaptation of DeepSeek’s Latest Model, Domestic AI Software and Hardware Collaboration is Timely
Global AI Giants’ Capital Expenditure Soars: Understanding the Underlying Logic of Growth in the AI Computing Power Industry Chain