1. Background
In recent years, with the continuous development of AI technology, AI has been widely applied in fields such as finance and healthcare. The emergence of large language models has had a tremendous impact on various industries. However, these models have very high requirements for computing power and memory, leading to issues such as computing power shortages and increased energy consumption. Currently, many countries are actively formulating relevant policies to stimulate and guide the development of the computing power industry, such as the implementation opinions on the “East Data West Computing” project and the action plan for the high-quality development of computing power infrastructure. At the same time, major domestic and international companies are increasing their investment in AI cluster construction. For example, Meta recently announced two AI computing clusters with a total of 49,152 GPUs, each equipped with NVIDIA H100 GPUs, and customized optimizations in hardware, networking, storage, design, performance, and software. Therefore, constructing high-performance, large-scale AI computing clusters to meet the growing AI demand is imperative.
2. Goals and Principles for AI Computing Cluster Construction
When constructing an integrated AI computing cluster, targeted construction goals need to be proposed. First, the primary goal of the AI computing cluster is to provide high-performance computing capabilities to meet the needs of complex algorithms such as deep learning and large models. This can be achieved by adopting advanced AI chips, continuously optimizing algorithms, and improving data transmission efficiency to achieve a leap in computing power. Secondly, the continuous development of AI applications leads to a growing demand for AI computing power. Therefore, the AI computing cluster needs to have high scalability, enabling flexible addition of computing nodes and storage resources according to demand, ensuring that the system can continuously meet future needs. Additionally, while enhancing computing power, it is essential to emphasize data security and privacy protection. By employing encryption technologies and access controls, the security of data within the AI computing cluster can be ensured, preventing data leakage and misuse. Finally, the AI computing cluster should serve the innovation and application of AI, providing strong computing support for researchers and enterprises, and promoting the rapid development and widespread application of AI technology.
When building an integrated AI computing cluster, certain principles should also be followed. First, advanced technologies and equipment should be adopted to ensure that the AI computing cluster maintains a leading position in terms of performance and efficiency, meeting the ever-changing demands of AI applications. Second, measures such as optimizing system architecture, improving equipment quality, and strengthening fault prevention and recovery should be implemented to ensure the stability and reliability of the AI computing cluster. Third, while ensuring performance and quality, efforts should be made to minimize the construction costs of the AI computing cluster and maximize resource utilization efficiency to achieve both economic and social benefits. Fourth, the AI computing cluster should be open and compatible, supporting various operating systems, development tools, and algorithm frameworks to facilitate user development and application. Fifth, as AI computing clusters have high energy consumption characteristics, they bear the social responsibility of energy conservation and carbon reduction. During the construction process, attention should be paid to environmental protection and energy-saving concepts, adopting green and low-carbon technologies and equipment to achieve sustainable development.
In summary, the goals and principles for constructing AI computing clusters should focus on enhancing computing performance, improving scalability, ensuring data security and privacy, and promoting AI innovation and application, while adhering to principles of advancement, reliability, economy, openness, and sustainable development to accelerate the rapid development of AI technology and its widespread application across various industries.
For a mainstream AI computing cluster, it can be described from three main aspects: hardware architecture, software optimization, and cluster resource scheduling and management.
To design a high-performance hardware architecture, it needs to include key components such as AI chips, memory, CPU processors, storage, and cluster networking. AI chips (also known as AI accelerator cards) are the core of the AI computing cluster. Classified by technical architecture, they can be divided into GPU, FPGA, ASIC, and neuromorphic chips. GPUs have strong floating-point computing capabilities, alleviating the training challenges of deep learning algorithms and unleashing the potential of artificial intelligence. FPGAs possess both hardware pipeline parallelism and data parallel processing capabilities, suitable for processing a single data point in a hardware pipeline manner, commonly used in the inference stage of deep learning algorithms. However, FPGAs require hardware configuration to implement software algorithms, making it somewhat challenging for complex algorithms. ASICs are chips customized to meet specific requirements, which helps improve the performance-to-power ratio of ASICs, but the circuit design needs to be customized, resulting in longer development cycles and limited extensibility. However, they have advantages in terms of power consumption, reliability, and integration, especially evident in high-performance, low-power mobile applications. For example, Google’s TPU and Horizon’s BPU are both ASIC chips. Neuromorphic chips simulate the computational mechanisms of biological neural networks, structurally approximating the brain, with their memory, CPU, and communication components fully integrated, processing information locally and overcoming the speed bottlenecks between traditional computer memory and CPU. At the same time, neurons can easily and quickly communicate with each other, acting simultaneously upon receiving pulses from other neurons.
Functionally, AI chips can be divided into training cards and inference cards. The training process involves massive training data and complex deep neural network structures, requiring significant computational scale and high performance requirements for processors in terms of computing power, precision, and scalability. Currently, NVIDIA GPU clusters are commonly used to accomplish this, with Google’s TPU series and Huawei’s Ascend 910 AI chips also supporting deep network acceleration in the training phase. The inference phase typically does not involve parameter adjustment optimization and backpropagation; it primarily focuses on efficiently converting inputs into outputs. This phase has significantly lower computational demands compared to the training phase but still involves a large number of matrix computations. In the inference phase, in addition to CPUs or GPUs for computation, FPGAs and ASIC-type chips can also play significant roles. Typical inference cards include NVIDIA Tesla T4 and Atlas 300I Duo inference cards. Currently, the vast majority of AI computing clusters are built using NVIDIA GPU computing cards. Their DGX and HGX servers are the preferred devices for major computing service providers and cloud service providers. NVIDIA’s high-end GPU card models include A100 and H100, which are not available in the domestic market due to sales restrictions. Domestic computing card manufacturers, including Huawei, Cambricon, Haiguang, Moore Threads, and Birun, are also increasing their R&D investments to achieve independent control over basic computing hardware. However, it must be said that there is still a significant performance gap between domestic computing cards and NVIDIA’s computing cards.
For AI computing clusters, RDMA networks are the preferred choice for cluster networking. This is a high-bandwidth, low-latency large-scale communication network, well-suited for AI computing clusters. There are three main technologies for RDMA networks: InfiniBand, RoCE, and iWARP. InfiniBand is a network designed specifically for RDMA, ensuring reliable transmission at the hardware level, but the hardware devices are expensive and high-cost. RoCE and iWARP are both RDMA technologies based on Ethernet, enabling the deployment of high-speed, ultra-low-latency, and extremely low CPU usage RDMA technology on the most widely used Ethernet. Compared to InfiniBand, the communication effect of the network may be slightly inferior, but the hardware devices are cheaper. If it is a large-scale production-type AI computing cluster with sufficient funds, InfiniBand networking is recommended. If it is a small or laboratory-type AI computing cluster with less stringent latency requirements, RoCE or iWARP networks can be used. The RoCE protocol has two versions: RoCE v1 and RoCE v2. RoCE v1 is an RDMA protocol implemented at the Ethernet link layer, requiring switches to support PFC and other flow control technologies to ensure reliable transmission at the physical layer, while RoCE v2 implements the RDMA protocol at the UDP layer of the Ethernet TCP/IP protocol, with the main improvement of supporting IP routing, solving scalability issues through the introduction of IP.
In training AI models, a large amount of data exchange is required. Storage, as the foundational hardware for data, is not just a simple data recording device but deeply participates in the entire process of model training, such as data aggregation, data flow, and efficient utilization. If storage performance is inadequate, it may take a significant amount of time to complete a training session, severely constraining the model’s iteration speed. Typically, high-speed NVMe SSDs are used as the main data storage devices to improve I/O performance. Additionally, a reasonable storage optimization plan should be designed to meet the storage needs of different scenarios by combining various types of storage media like flash and hard drives. For example, distributed file systems (such as Hadoop HDFS and Ceph) can be used to improve data storage capacity and reliability. At the same time, for data that requires quick access, memory-mapped files (such as Redis and Memcached) can be used to enhance access speed. Moreover, high-performance multi-core CPU processors, such as Intel Xeon Platinum processors or AMD EPYC processors, should be selected to meet the demands of large-scale parallel computing. High-speed, large-capacity memory, such as DDR4 or DDR5, should also be configured to support rapid data access and processing.
In addition to hardware architecture, software optimization is also necessary to improve computational efficiency, including operating systems, virtualization technologies, parallel computing libraries, load balancing monitoring, and management. The operating system plays a crucial role in AI computing clusters, managing hardware such as CPU processors, AI accelerator cards, memory, and disks while providing computational support for application software. Therefore, high-performance operating systems, such as CentOS and Ubuntu, should be selected and appropriately tuned to enhance system performance. Furthermore, high-performance AI clusters also employ high-performance virtualization technologies to virtualize GPUs and establish computing pools, achieving efficient resource utilization and dynamic scheduling. High-performance parallel computing libraries, such as OpenMPI and CUDA, are used to realize efficient parallel computing. Load balancing technologies, such as software load balancing or hardware load balancing, are also implemented to ensure that resources within the cluster are reasonably allocated and fully utilized. Real-time system monitoring and management should be implemented to ensure the stable operation of the cluster and timely detection of potential issues.
To fully leverage the performance of AI computing clusters, effective resource scheduling and management strategies need to be implemented, primarily including task scheduling, resource allocation, automatic scaling, fault tolerance, and recovery. First, different scheduling strategies, such as priority, round-robin, and multi-level feedback queues, should be adopted based on the characteristics and requirements of tasks. Additionally, dynamic adjustment of task priorities and resource requirements can be implemented to improve overall performance. Second, resources such as AI accelerator cards, CPU processors, memory, and storage should be allocated reasonably according to the resource demands of tasks and the actual conditions of the cluster to ensure efficient execution of tasks. Then, the automatic scaling function of the cluster should be realized, allowing for automatic expansion by adding more computing nodes and storage devices as business volume increases to meet business needs. At the same time, resource allocation and task scheduling strategies can be dynamically adjusted based on actual load conditions. Finally, a comprehensive fault tolerance and recovery mechanism should be established to prevent hardware failures, software errors, and other issues from causing cluster downtime, requiring data backup and rapid recovery to ensure data security.
3. Development Characteristics of Integrated AI Computing Clusters
In the AI era, the development of integrated computing systems presents several significant characteristics:
1. Competition for Large-Scale AI Chips. Driven by the widespread demand for large models in AI, chip design focuses more on large computing power, with new computing architectures continuously emerging. For example, the industry currently uses two mainstream routes to meet the computing power needs of the AI industry: one represented by NVIDIA’s GPU route, which is a general-purpose chip route capable of completing diverse computing tasks and suitable for large-scale parallel computing; the other represented by Google’s and Huawei’s ASIC route, which involves specialized chip designs used to execute customized tasks, showing better energy efficiency in specific scenarios, particularly in high-performance, low-power mobile applications.
2. Surge in Demand for AI Servers. The emergence of large models has led to a surge in demand for AI servers. Training large AI models typically requires thousands of high-end AI accelerator cards, resulting in a shortage of high-end AI servers. Although the inference phase has relatively lower requirements for AI computing resources, it still necessitates high-performance AI server support.
3. Emergence of AI Model Training and Inference Integrated Machines. The concept of AI model integrated machines is gradually rising, providing enterprises with an “out-of-the-box” solution. It simplifies the deployment and usage processes of AI technology, making it easier for businesses to enjoy the benefits brought by AI technology. For instance, the Zhipu-Shengteng ChatGLM training and inference integrated machine, based on Shengteng AI hardware, adapts to the ChatGLM series foundational models and boasts comprehensive functionality, leading technology, and out-of-the-box advantages, providing enterprises with localized deployment solutions for large models, including intelligent computing, large model training and inference capabilities, and application development and deployment.
4. Rise of Intelligent Computing Centers. Intelligent computing centers play a supportive role in the development and operation of AI, with computing power leasing models becoming a new solution. Intelligent computing centers provide strong computing support, meeting the demands of AI in training, inference, and other stages. Through AI computing power scheduling platforms, AI computing power can be service-oriented, scenario-based, and automated, while the computing power leasing model enables many small and medium-sized enterprises to flexibly obtain and utilize computing resources, accelerating the commercialization process of AI computing resources.
5. Transformation of Cloud Service Paradigms. Model-as-a-Service (MaaS) is reshaping cloud service paradigms, accelerating the closure of AI business models. This transformation allows AI models to be more easily integrated into various applications, thereby enhancing the popularity and application efficiency of AI technology.
In summary, the development characteristics of integrated computing systems in the AI era primarily reflect the pursuit of high-performance AI servers, the provision of comprehensive solutions through AI model integrated machines, the rise of intelligent computing centers lowering research and usage thresholds, and the transformation of cloud service paradigms accelerating application implementation. These characteristics collectively drive the rapid development and widespread application of AI technology.
4. Opportunities and Challenges in Building Integrated AI Computing Clusters
With the deepening development of the digital and intelligent era, it is expected that the demand for computing power across various industries will experience explosive growth, providing a vast market space for integrated computing systems. First, from the perspective of industry applications, integrated computing systems will be widely applied in healthcare, education, finance, manufacturing, transportation, and other fields. For example, in the financial sector, integrated computing systems can support large-scale data processing, risk analysis, and investment decision-making, enhancing the intelligence level of financial services. In the manufacturing sector, computing systems can assist enterprises in achieving intelligent production, supply chain management, and product innovation, enhancing the competitiveness of the manufacturing industry. Second, from the perspective of technological development, integrated computing systems will continuously drive innovation and breakthroughs in technology. With the rapid advancement of cloud computing, big data, artificial intelligence, and other technologies, the computing systems need to deeply integrate with them to jointly promote digital transformation and intelligent upgrading. At the same time, integrated computing systems also need continuous innovation in hardware, software, algorithms, and other aspects to enhance computing performance, efficiency, and reliability. Additionally, from the perspective of policy environment, governments worldwide are actively promoting the development of the digital economy, and computing power, as the infrastructure of the digital economy, will receive more policy support and funding investment. This will provide a favorable external environment and development opportunities for the growth of integrated computing systems. However, the development of integrated computing systems also faces challenges such as data security, privacy protection, and energy consumption. Therefore, while promoting the development of computing systems, it is also necessary to strengthen technological research and innovation, improve data security and privacy protection capabilities, and promote AI computing clusters towards low-carbon and energy-saving pathways to ensure sustainable development.
In conclusion, the development prospects of integrated computing systems are very bright, and they will be driven and supported in multiple aspects such as industry applications, technological innovation, and policy environment. With the continuous advancement of AI technology and the expansion of application scenarios, integrated AI computing clusters will become an important force driving economic and social development.
■ Written by Li Guodong, Mashang Consumer Finance Co., Ltd.
Previous Highlights
During the college entrance examination, decoding the “Safe College Entrance Examination” essay.
New business opportunities and cases, the cultural tourism market is hot, and security welcomes “new” transformation.
Interpretation of the “Interim Regulations on the Flight Management of Unmanned Aerial Vehicles”.
Product and solution evaluation recommendations empower the development of new-quality productivity in security.
“China Security”
– The authoritative magazine in the domestic industry –
Published by: Editorial Department of “China Security”
Supervised by: China Security and Prevention Products Industry Association
Phone: 010-88825523