“Training a large model costs tens of millions of dollars in computing power,” “GPU server prices have doubled and are still hard to obtain,” “Half of the GPUs in the cluster are idle and cannot be scheduled”—this is the most genuine computing power anxiety in the current AI industry.
As the industry struggles with computing power bottlenecks, three major technological directions have proposed significant solutions: Huawei’s newly released Flex:ai AI container software breaks resource barriers with 10% precision computing power partitioning, increasing NPU/GPU resource utilization by 30%; the Aegaeon multi-model service system proposed by Peking University and Alibaba’s team achieves an 82% savings in GPU resources; while the supernode technology represented by NVIDIA DGX creates a “performance aircraft carrier” for large model training through hardware aggregation.
On one side is the supernode’s “hardware performance stacking” to the right, and on the other is virtualization’s “software resource partitioning” to the left, while Huawei Flex:ai and Alibaba Aegaeon have found a path of integrated innovation in between—restructuring the logic of AI computing power usage and providing new possibilities for cost reduction and efficiency improvement for enterprises. This article will deeply analyze these three major technologies to clarify the optimal solution for computing power management under the paradigm of “virtualization to the left, supernodes to the right.”
Before diving into the main content, let’s briefly introduce the two core technologies: GPU virtualization and supernodes:
GPU Virtualization: A processing technology that allows multiple virtual machines to share physical GPU resources through virtualization technology. NVIDIA’s MIG solution can partition a single GPU into up to 7 separate GPUs, allowing 7 users to simultaneously utilize the resources of one GPU card; AMD’s SRIOV can partition a single GPU into up to 8 independent GPUs.
Supernodes: Scalable computing units formed by integrating multiple computing chips (such as GPUs or NPUs) through high-speed interconnection technology, primarily aimed at solving the issues of computing power collaboration and efficiency in training large AI models. Vertical scaling maximizes communication bandwidth within the cluster and minimizes latency between dense computing resources (e.g., GPUs), while horizontal scaling enables longer-distance communication within the data center infrastructure.
1. Core Breakthroughs of Flex:ai, Aegaeon, and Supernodes
Whether it is the hardware aggregation of supernodes, the computing power pooling of Flex:ai, or the fine scheduling of Aegaeon, the essence is to solve the problem of “low GPU resource utilization,” but the technical paths of the three show a clear distinction of “to the right, integration, and to the left refinement.”
1. Huawei Flex:ai: Cross-Architecture “Computing Power Pooling Engine”
Flex:ai is not just a simple NPU/GPU management tool, but a “XPU computing power scheduling operating system” built on Kubernetes, with its core breakthrough being the dual capability of “computing power partitioning + global pooling,” completely breaking the boundaries of hardware and nodes.
Two Core Technologies of Flex:ai
1. Fine Computing Power Partitioning: Partitions a single GPU/NPU into virtual computing power units with 10% precision, allowing a single card to simultaneously carry multiple AI workloads, with the computing power ratio dynamically adjustable, solving the waste issue of “a single card can only run one model.”
2. Global Computing Power Aggregation: Aggregates all idle XPU computing power from all nodes in the cluster into a “shared computing power pool,” enabling global scheduling across nodes and architectures, allowing scattered idle computing power to be efficiently utilized.
Notably, its differentiated advantages include: first, full architecture compatibility, capable of scheduling NVIDIA GPUs as well as adapting to Ascend NPUs and third-party chips, breaking the ecological monopoly of hardware vendors; second, an open-source strategy, with releases synchronized to the Model Engine community (https://gitcode.com/ModelEngine), significantly reducing the migration costs for enterprises, making Flex:ai highly competitive in mixed computing environments.
2. Alibaba Aegaeon: Token-Level “Model Scheduling Artifact”
The core value of Aegaeon is reflected in its published technical paper—achieving extreme efficiency in sharing GPU resources among multiple models through “token-level dynamic scheduling.” This is a disruptive reconstruction of the AI model service process.
In traditional multi-model services, GPU resources are either monopolized by a single model or incur significant performance losses during model switching. Aegaeon creatively lowers the scheduling granularity from “model-level” to “token-level” (where a token is the basic unit of AI text processing), akin to upgrading a parcel sorting system from “batch sorting” to “real-time scheduling by individual parcels.”
Aegaeon addresses the resource waste issue in multi-model services through a layered design of “token-level scheduling, phased computation, cache reuse, and elastic scaling” to ultimately achieve extreme pooling of GPU resources (paper: https://doi.org/10.1145/3731569.3764815)
Core Data and Mechanisms from Aegaeon Paper
• Key Results: The number of GPUs required for 10 models was reduced from 1192 to 213, achieving a resource savings rate of 82%, far exceeding the industry average.
• Core Mechanism: Allocates GPU computing resources in real-time at the token level; includes an automatic scaling module for models that dynamically adjusts resource allocation based on request traffic, ensuring service quality while avoiding resource waste.
• Performance Optimization: Reduces model switching overhead by 97% through techniques such as preloading model parameters and optimizing memory scheduling, addressing performance losses caused by fine scheduling.
3. Supernodes: Hardware-Intensive “Computing Power Behemoth”
If virtualization is the leftward exploration of “software partitioning,” supernodes (SuperPod) represent a typical rightward practice of “hardware aggregation.” They embody a technology route in computing power management that emphasizes “heavy hardware and strong interconnection,” with the core technology being “high-density integration + low-latency interconnection”: on the hardware level, customized servers (such as NVIDIA DGX GB200) are used, integrating multiple high-performance GPUs within a single node and achieving direct GPU interconnection; on the network level, high-speed interconnection technologies like InfiniBand are employed to reduce communication latency within and between nodes to microsecond levels, ensuring efficient data flow during large-scale parallel computing.
NVIDIA DGX GB200 NVL72 Rack System
This architecture determines that supernodes are inherently designed for large models—when training models with hundreds of billions of parameters, hundreds of GPUs need to work in coordination, and the hardware optimization of supernodes maximizes the aggregation performance of the computing power cluster. Taking the industry-leading NVIDIA DGX SuperPOD as an example, its technical details can be divided into three layers:
Three-Layer Technical Architecture of Supernodes
• Hardware Layer: Utilizes customized server nodes, integrating 8 A100/H100 GPUs per node, achieving high-speed direct interconnection between GPUs within the node through NVLink technology, avoiding data transmission bottlenecks within a single node; at the cluster level, a high-density rack design is adopted, with every 4 nodes forming a “computing unit,” enhancing space utilization and heat dissipation efficiency.
• Network Layer: Equipped with InfiniBand HDR200 high-speed interconnection network, achieving inter-node bandwidth of 200GB/s and latency as low as 1.2 microseconds, more than 10 times that of ordinary Ethernet; through network topology optimization, ensures “consistent communication latency between any two nodes,” maintaining load balancing during large-scale parallel computing.
• Software Layer: Accompanied by the NVIDIA AI Enterprise suite, which includes Model Parallelism and Data Parallelism optimization tools, automatically splitting large model parameters across different GPUs and distributing training data shards to maximize the utilization of cluster computing power.
Using a 72×1 NVLink topology, this topology includes 72 GPUs distributed within a single NVLink domain. The DGX GB200 rack system contains 18 computing nodes (also known as trays). Each computing tray integrates two GB200 super chips, each equipped with two B200 GPUs and one Grace CPU. The computing tray integrates four ConnectX-7 (CX-7) network cards, supporting InfiniBand NDR (400Gbps) connections for cross-rack computing networks, as well as two BlueField-3 (BF3) network cards.
The performance advantages of supernodes are particularly significant in practical scenarios: OpenAI trained GPT-3 using a cluster with a similar supernode architecture, compressing the training cycle from “years” to “months”; a major domestic company built a supernode cluster based on DGX SuperPOD, achieving a 45% efficiency improvement in training large models with hundreds of billions of parameters compared to ordinary GPU clusters. However, the extreme performance comes with a high cost— a DGX SuperPOD with 512 A100 GPUs has a procurement cost exceeding 200 million yuan, with annual maintenance costs accounting for 15%-20% of the procurement cost.
However, the “rightward” characteristics of supernodes also bring significant limitations: first, “rigid resource allocation,” as supernode clusters typically allocate resources at the “whole cluster/whole node” level, meaning that even training a small to medium model requires occupying at least one node (8 GPUs), leading to severe resource waste; second, “closed ecosystem,” as DGX SuperPOD only supports NVIDIA GPUs and cannot accommodate domestic chips like Ascend, requiring enterprises to reconstruct the entire supernode cluster if they change hardware systems; third, “limited scalability,” as the network topology and hardware configuration of supernodes are fixed at deployment, requiring nodes to be added according to fixed specifications for subsequent expansions, making it inflexible to adapt to small increases in computing power demand.
For more information, please refer to Knowledge Planet
Multi-Dimensional Comparison of the Scene Adaptation Boundaries of Three Major Technologies
Combining GPU virtualization technology, we clearly delineate the applicable scenarios of supernodes, Flex:ai, and Aegaeon from core dimensions such as “technical route, cost, and efficiency,” clarifying the choice logic of “left/right/integration.”
“Virtualization to the Left”—pursuing flexible resource reuse through software/hardware partitioning; “Supernodes to the Right”—creating performance peaks through hardware aggregation. Huawei Flex:ai and Alibaba Aegaeon have found an innovative integration path between these two extreme routes. In addition to Flex:ai and Aegaeon, GPU virtualization and supernodes (SuperPod) are also mainstream computing power management solutions, allowing for a comparison across seven core dimensions to clarify the applicable boundaries of different technologies.
Differentiated Value and Combination Logic of Three Major Technologies
When enterprises choose computing power management solutions, it is essentially a comprehensive trade-off of “performance requirements, cost budget, and scene complexity.” Supernodes, Flex:ai, and Aegaeon are not in competition but are complementary solutions covering all scenarios from “training-inference-mixed load.” We will analyze their differentiated value and combination logic from both technical paths and commercial adaptation perspectives.
1. Technical Path: Rightward Performance Stacking, Leftward Refinement, Middle Ground Balance
The “rightward path” of supernodes is an extreme performance route driven by hardware—building technical barriers through dedicated servers and high-speed interconnection networks, with the core goal of solving the parallel computing bottleneck of models with hundreds of billions of parameters. The advantage of this path is that performance loss is <5%, but the cost is a closed ecosystem and high expenses, making it unsuitable for flexible multi-load scenarios.
Aegaeon’s “leftward refinement” is an efficiency route driven by software—on the basis of virtualization “partitioning resources,” it lowers the scheduling granularity from “card-level” to “token-level,” focusing on resource optimization in multi-model inference scenarios. Its advantage is extremely high efficiency in multi-model services, but it has weak generalization and cannot support large model training.
Flex:ai’s “middle ground balance” is an integration route—absorbing the concept of “computing power aggregation” from supernodes, it uses software to achieve cross-node resource pooling; borrowing the idea of “flexible partitioning” from virtualization, it adapts to multiple loads with 10% precision; while also being compatible with multiple architecture hardware, avoiding the closure and high costs of supernodes, and solving the generalization issues of Aegaeon, becoming a “universal adapter” covering most scenarios.
2. Commercial Adaptation: Exclusive for Giants, Precision Empowerment, Inclusive for All
Supernodes are an “exclusive solution for giants”: suitable only for well-funded companies like Google, Meta, and major domestic firms, whose core value lies in seizing market opportunities by shortening large model training cycles; the procurement cost of 200 million yuan is a “strategic investment” for these enterprises.
Aegaeon is a “precision empowerment solution”: focusing on cloud service providers’ AI API platforms and enterprises’ multi-model deployments, for example, Alibaba Cloud can reduce nearly 900 GPU investments for 10 commonly used models through Aegaeon, saving nearly 100 million yuan in annual maintenance costs, making it the “efficiency champion within the scene.”
Flex:ai is an “inclusive solution for all”: its open-source strategy lowers the usage threshold for small and medium enterprises, and its cross-architecture features support the reuse of existing hardware; a manufacturing enterprise improved GPU cluster utilization from 35% to 82% through Flex:ai, directly saving 6 million yuan in hardware procurement costs, adapting to the needs of most enterprises from internet giants to traditional businesses.
Left or Right? The Answer Lies in “Demand”
The “rightward” supernodes are the “performance cornerstone” for large model training, Aegaeon’s “leftward refinement” is the “efficiency tool” for multi-model inference, and Flex:ai’s “middle ground balance” is the “universal adapter” for mixed loads. For enterprises, the optimal strategy is a “combination punch”: using supernodes to support core large model training, using Aegaeon to optimize multi-model inference services, and using Flex:ai to build a global computing power pool to manage all loads, achieving “performance without compromise, resources without waste” through technological integration.
With the rapid development of AI technology, computing power will become a fundamental resource like water and electricity. The explorations of Flex:ai and Aegaeon are making the vision of “computing power as a service” a reality—enterprises no longer need to worry about computing power, but can focus on their own AI business innovations, leaving the rest to computing power management tools. This may be the ultimate significance of the computing power revolution.
References (Add WeChat friends or send a private message to get)
-
NVIDIA Supernodes: NVIDIA DGX SuperPOD: Next Generation Scalable Infrastructure for AI Leadership.pdf
-
Alibaba Pooling: Aegaeon: Effective GPU Pooling for Concurrent LLM Serving on the Market.pdf
-
NVIDIA MIG: NVIDIA Multi-Instance GPU User Guide.pdf
Follow me!
Let’s
Explore
Technical Principles and Business Logic
One-Click Triple Connection
Previous
Selected
Domestic AI Chips are Facing a “Breakthrough Moment”—Interpretation of the 2025 Domestic AI Chip Industry White Paper
“Ascend + Kunpeng” Dual-Engine Drive: Huawei’s AI Computing Power Strategy Stimulates Resonance Across the Entire Industry Chain
Google Gemini 3 Pro: The Ultimate Reflection of AI Applications and TPU Hardware-Software Collaborative Design
Major Model Giants are Developing Their Own AI Chips: The Tough Battle Behind the Computing Power Competition
Cambricon and Ascend Day 0 Adaptation of DeepSeek’s Latest Model, Domestic AI Software and Hardware Collaboration is Timely
Global AI Giants’ Capital Expenditure Soars: Understanding the Underlying Logic of Growth in the AI Computing Power Industry Chain