Artificial intelligence requires four networks:
-
The computing structure connects AI accelerators, GPUs, CPUs, and other components within servers. This structure is designed for high-speed operation over short distances, typically relying on copper cables using PCIe or proprietary interfaces like NVLink.
-
The backend network connects the aforementioned servers to AI clusters using interconnected network switches and optical modules. The network protocols are InfiniBand or Ethernet, but both protocols operate over optical modules. “Whenever you see InfiniBand or Ethernet, you should consider the backend network running over fiber,” Nguyen said.
-
The frontend network connects AI clusters to cloud data centers for storage, switching, etc. The CPUs located inside AI servers move data in and out, with each CPU having its own network card connected to its optical module. The frontend network always uses Ethernet protocol instead of fiber.
-
Finally, the Data Center Interconnect (DCI) network connects one data center to other data centers in the region using links of 100 kilometers or longer.
AI servers use two-layer switched optical links in the backend network, shown in blue.
The frontend CPU (shown in green) connects the AI servers to the rest of the data center.
The DCI interconnect (shown in yellow) connects the data center to the outside world.
In the computing structure, the Switch chip is the core component for achieving high-speed data exchange, responsible for data communication between components within the server (GPU, CPU, NIC, SSD), ensuring high-speed, lossless data transmission between thousands of cards, directly impacting the overall efficiency of the computing cluster.
The core components of Ethernet switches and InfiniBand switches in the frontend and backend networks are also switch chips.
PCI Express (PCIe), as the core internal interconnect bus technology, has critical design and bandwidth requirements for its Switch chips.By 2025, PCIe 5.0 has become the standard configuration for mainstream AI servers, significantly alleviating the data interaction pressure between GPUs and networks, storage, with its doubled bandwidth compared to PCIe 4.0 (theoretical bidirectional bandwidth of up to 128 GB/s under x16 lanes).
For high-density multi-GPU node internal (Intra-node) “Scale-up” communication, the standard PCIe bus topology still has bottlenecks. Therefore, NVIDIA has designed its own 5-3nm NVSwitch chip and dedicated interconnect technology, providing bandwidth and efficiency far exceeding PCIe, creating a coexistence scenario with the open standard PCIe solutions.
Meanwhile, PCIe 6.0 technology has entered the early commercial stage, bringing another doubling of bandwidth and more complex PAM4 signaling technology, indicating the evolution direction of the next generation AI server interconnect architecture.
Large model parameters are enormous, and computing power demands multiply, requiring vast memory for loading. The intermediate parameters of computation need to be cached, relying on thousands or even tens of thousands of accelerator card computing clusters. Thus, the data flow between GPU-CPU, GPU-GPU, and CPU-CPU becomes the key to cluster efficiency:
-
GPU-to-GPU communication (Peer-to-Peer, P2P): In model parallel or pipeline parallel training, GPUs need to frequently exchange large amounts of weight gradients and activation values. If the interconnect bandwidth between GPUs is insufficient, it will cause computing units (GPU cores) to be in a waiting state for extended periods, severely reducing training efficiency.
-
Data interaction between CPU and GPU: The CPU is responsible for data preprocessing, task scheduling, etc., and needs to efficiently load training data into GPU memory. The bandwidth of this path determines the speed of data supply.
-
Collaboration between network and GPU: In large-scale distributed training (Scale-out), communication between nodes relies on high-performance network cards (NICs). Data needs to be quickly transferred from the NIC to the GPU, implementing technologies like GPUDirect RDMA to bypass the CPU, reducing end-to-end latency.
-
Connection between storage and GPU: As the scale of models and datasets expands, high-speed NVMe SSDs become a crucial part of data caching and rapid loading. GPUs need to be able to read data directly and quickly from storage devices (GPUDirect Storage), which places extremely high demands on the bandwidth of the PCIe channels connecting the storage.
The traditional PCIe bus adopts a tree topology, where all devices communicate through the Root Complex (usually within the CPU, meaning all data exchanges require the CPU. It is easy to imagine that everyone is already dissatisfied with Intel’s bus ecosystem, but downstream customers have not yet had replacement needs). This architecture is prone to congestion and scalability limitations in scenarios with dense multi-GPU P2P communication; therefore, high-performance PCIe Switch chips have emerged, allowing multiple PCIe devices to establish direct, low-latency communication paths by constructing a switching matrix (Fabric), thus breaking through the limitations of the number and topology of host PCIe interfaces.
Each generation of PCIe standards upgrades the core by doubling the data transfer rate of a single lane, thereby achieving a doubling of total bandwidth.
PCIe 7.0 (128 GT/s): Currently still in the standard formulation stage, the draft is expected to be released in 2025, aiming to double the bandwidth again to 512 GB/s (x16) to meet the endless demand for data interconnect in the post-Moore’s law era of AI computing power.
The current ecosystem of Switch chips
The main suppliers in the global PCIe Switch chip market are Broadcom and Microchip, which together account for the vast majority of the market share.
In actual AI server deployments, one or more PEX89000 series chips are typically used to construct complex GPU interconnect topologies. For example, in an 8-GPU server, a high-channel-count switch can connect all 8 GPUs and provide an uplink to the CPU, or multiple switches can be used to build layered or more complex network topologies to optimize bandwidth and latency under specific communication patterns.
Microchip, as another major supplier, has its Switchtec series PCIe switches, which also hold an important position in the market. Its PCIe 5.0 products offer configurations of up to 100 channels and emphasize advantages in signal integrity, low power consumption, and reliability, widely used in AI servers and storage systems.
Even with the advent of PCIe 5.0, PCIe still presents performance bottlenecks in certain extreme scenarios.
The main bottleneck points lie in aggregated bandwidth and topology:
-
Taking a typical 8-GPU server as an example, suppose all 8 GPUs are connected to a large PCIe Switch chip via a PCIe 5.0 x16 link. Although each GPU has a bandwidth of 128 GB/s to the switch, if all 8 GPUs need to communicate with the CPU simultaneously or perform “All-to-All” pairwise communication, the switch’s internal switching matrix and the uplink to the CPU (usually also x16 or x32) need to handle aggregated traffic of up to 1 TB/s (8 * 128 GB/s). This traffic pattern is prone to congestion within the switch or the uplink, thus limiting overall performance.
To thoroughly resolve the bottleneck of GPU-to-GPU communication within nodes, NVIDIA has developed its proprietary NVLink and NVSwitch technologies.
In the latest H100/H200 systems, the combination of the fourth-generation NVSwitch and NVLink 5.0 has pushed bandwidth to new heights, with each NVSwitch ASIC providing a total bandwidth of 57.6 Tb/s (7.2 TB/s). This is several orders of magnitude higher than PCIe 5.0 x16’s 128 GB/s.
By combining multiple NVSwitch chips, a non-blocking fully connected network can be constructed among 8 GPUs, allowing any two GPUs to achieve full-speed direct communication without going through the CPU or other intermediaries, greatly accelerating the performance of model parallel and collective communication operations.
In the high-end AI server market of 2025, a clear division has formed: for top training servers (such as NVIDIA DGX/HGX platforms) pursuing extreme intra-node performance, “NVSwitch + NVLink” is used for GPU-to-GPU communication, while PCIe is responsible for connections between GPUs and CPUs, networks, and storage; for more general AI servers or inference servers, high-performance PCIe Switch chips are relied upon to build flexible interconnect architectures. Although companies like “Leading Company” have designed their own “Leading Link” to compete with NVLink, it is limited to clusters that fully utilize that ecosystem; if customers choose other clusters, PCIe is still needed.
As a critical link in servers, PCIe chips have not yet entered the overseas supply chain of Broadcom, but under the context of domestic computing power being self-controlled, the localization of server chips is an important direction. Major manufacturers will fill the gaps, and the promotion of localization needs to accelerate. The market should pay attention to its future elastic opportunities. From market space calculations, a conventional AI server with eight cards corresponds to six PCIe switch chips, four for GPU interconnect services and two for CPU-GPU services. A PCIe 5.0 switch chip costs about $450, while a PCIe 6.0 chip costs about $1000. It is expected that by 2026, the shipment of non-NVIDIA-level computing power chips will be about 10 million, corresponding to 7.5 million switch chips, with a market scale of about $3.8 billion. By 2027, as switch chips upgrade to PCIe 7.0, the market scale may double or more.
A new technological direction: In the future, with the further maturity and application of the CXL (Compute Express Link) protocol at the PCIe physical layer, future AI servers will not only achieve data interconnect but also realize memory pooling and sharing, constructing a more flexible and efficient heterogeneous computing resource pool.
Shengke Communication Company
Founded in 2005 by Sun Jianrong (former Cisco engineer) and Zheng Xiaoyang (former LSI Logic engineer), Shengke Communication has focused on the research, design, and sales of Ethernet switch chips and supporting products since its establishment. In the domestic commercial Ethernet switch chip market, it holds a 15% market share, ranking first among local manufacturers. From a global revenue scale perspective, Shengke Communication ranked 8th in 2023, successfully breaking the long-standing monopoly of international giants like Broadcom and Marvell.
-
2005: The company was officially established, starting the research and development journey of domestic Ethernet switch chips.
-
2007: Successfully launched China’s first Ethernet switch chip supporting IPv4/IPv6 dual stack—the Bay series, achieving an early technological breakthrough.
-
2013: Launched the first gigabit Ethernet switch chip in China supporting SDN (Software Defined Networking)—the GreatBelt series, proactively laying out network virtualization technology.
-
2015: Released the SDN intelligent high-density 10G Ethernet switch chip—the GoldenGate series, enhancing product capacity to 1.2 Tbps, meeting the early high-speed interconnect needs of data centers.
-
2019: Launched the first 10G aggregation Ethernet switch chip integrated with a high-performance CPU—the TsingMa series, improving product intelligence and processing capabilities.
-
2021: Released the TsingMa.MX series for 5G and data center applications, integrating FlexE interfaces for the first time, supporting up to 400G port rates, and increasing switching capacity to 2.4 Tbps.
-
2023: The company was listed on the U.S. Department of Commerce’s “Entity List” in March, facing external challenges; in September of the same year, it successfully went public on the STAR Market, gaining support from the capital market.
-
2024 to present: The high-end flagship Arctic series chips have entered customer sample testing and small batch delivery stages, supporting switching capacities of up to 25.6 Tbps and port rates of 800G, which are key products for the company to penetrate the ultra-large-scale data center and AI market.
The Arctic series chips, with their 25.6 Tbps switching capacity and 800G high-speed ports, can construct high-radix, low-convergence Fat-Tree network topologies, providing powerful non-blocking interconnect capabilities for AI server clusters at the level of tens of thousands of cards. This makes it an attractive Ethernet solution to replace or supplement the expensive and relatively closed NVIDIA InfiniBand network.
Domestic leading switch manufacturers: Shengke has entered the supply chains of ZTE, H3C, Ruijie Networks, and Maipu Technology, with a domestic market share of 5.5% in 2023.
The lifecycle of switch chips lasts 8-10 years, and the cost of switching suppliers is high for customers. Shengke, with its first-mover advantage and technological iteration, has strong stickiness. Affected by price increases from Marvell/Broadcom and priority supply to North America, domestic customers are accelerating the adaptation of Shengke products, with incremental demand driven by AI data center construction in 2025.
High-end chips with 12.8 Tbps/25.6 Tbps switching capacity have been sent to customers for sampling, with port rates reaching 800G, and performance comparable to international competitors (such as Broadcom, Marvell).
Shengke Communication has reached cooperation with leading AI server manufacturers to provide chip solutions, even replacing traditional PCIe solutions to solve computing cluster communication bottlenecks. This cooperation has promoted the upgrade of network communication within AI servers and between clusters, greatly enhancing the performance of AI servers in data processing and transmission.
The average price of 800G Ethernet switch chips in the domestic market is about $6,000, but the specific price may vary due to cooperation relationships, purchase volumes, and agreement terms. In practical applications, generally, a module contains one 800G chip for constructing a single-module architecture. However, for core network nodes to achieve redundancy design, a dual-module architecture is sometimes used, with each module configured with two 800G chips to ensure the stability of critical node operations. Additionally, under ideal conditions, a high-end switch with a capacity of 51.2T can support 64 channels, each with a transmission rate of 800Gbps, achieving a total uplink and downlink bandwidth of 51.2 Tbps. However, due to the uneven load of lines in actual operation, some channels may be set as redundant channels or dynamically allocated tasks through multiple chips to optimize overall performance.
The 51.2T switch does not correspond to an independent switch chip module for each channel but is achieved through a single powerful switch chip. Taking Tomahawk5 as an example, this chip can handle up to 64 channels of 800G, with a total bandwidth of 51.2T. In practical applications, to ensure system stability, it is common to reserve 10%-20% of the channels as redundant. Additionally, there are designs using two chips to ensure that all 64 channels can operate at full load.
The core value of switches mainly lies in optical modules and switch chips, with optical modules being the major cost component. For example, an 800G switch based on Broadcom chips typically has a market price ranging from $40,000 to $50,000. Specifically, the cost of 64 800G optical modules alone exceeds $30,000. Furthermore, PCB, power management, and mechanical structures also account for a certain proportion, but compared to optical modules and chips, these parts have a lower cost ratio.
Broadcom has long been in a leading position in the switch chip field, especially holding a significant monopoly advantage in the data center switch market. Currently, its overall market share exceeds 50%, approaching 60%. Specifically, in the market for switch chips at different rates, Broadcom holds nearly 80% of the 800G switch chip market and about 70% of the 400G switch chip market.
In the domestic 400G switch chip market, the market share distribution among major manufacturers is as follows: Broadcom about 40%-50%, Marvell about 20%-30%, Huawei about 20% (difficult to count Huawei’s self-developed products), Realtek about 4%-5%. Additionally, Shengke has also achieved certain results in the domestic 400G field due to its cost-performance advantage, with its market share reaching as high as 15%. Overall, domestic and foreign brands each account for half of the market share.
Regarding Ethernet switches for communication, campus data centers, etc., at 200G, 400G, and 800G rates
In 2023, the market size of China’s Ethernet switch market reached 32.5 billion yuan, and it is expected to exceed 50 billion yuan by 2026. Among them, data center switches dominate the market with a 58% share (2023 scale of 18.8 billion yuan), while campus switches (32%) and industrial switches (10%) form a differentiated competitive landscape. The demand for AI computing power drives the rapid expansion of the high-rate switch market, with the shipment proportion of 400G ports increasing from 5% in 2019 to 38% in 2024. Although the 800G market is still in its early stages, its growth potential is significant.
200G/400G Market
Domestic manufacturers dominate: Huawei, H3C, and Ruijie Networks occupy the top three positions in the domestic market, with a combined share exceeding 95%. Among them, Ruijie Networks ranks first in the Chinese 200G/400G data center switch market in 2024 (32%), with a 58% share in the internet industry, serving leading customers like Tencent and ByteDance. H3C maintains a leading position in the enterprise network and data center market through deep cooperation with Shengke Communication.
International manufacturers still have penetration: Cisco and Arista still hold a certain share in the high-end market (such as finance and operator core networks), but their market share is gradually shrinking due to domestic substitution policies.
800G Market
International manufacturers like Broadcom and Marvell dominate the global 800G switch chip market with high-end chips like Tomahawk 4 and Teralynx 8, while domestic switch manufacturers initially relied on imported chips.
Huawei, H3C, and Ruijie Networks have launched 800G switch products, and Shengke Communication’s Arctic series 25.6 Tbps switch chip (supporting 800G ports) is expected to achieve small batch delivery in 2024, with performance comparable to Broadcom’s Tomahawk 4, filling the domestic gap. Industrial Fulian, as a global high-speed switch foundry, has a market share of over 30% for 400G/800G products, with customers including Cisco and Huawei.
Silicon Photonics Technology Applications: International manufacturers like NVIDIA and Broadcom have launched CPO (Co-Packaged Optics) switches, integrating optical modules with ASICs through silicon photonics technology, reducing power consumption by over 40%.
Lossless Network Technology: RoCEv2 protocol has become mainstream, with switches from Huawei and Ruijie Networks supporting end-to-end lossless transmission, with latency as low as microseconds, adapting to AI training cluster needs.Broadcom and Marvell occupy over 90% of the global commercial Ethernet switch chip market, with their single-port rate products above 50G accounting for 70% and 29% of shipments, respectively, in 2020. NVIDIA and Cisco meet their own needs through self-developed chips and gradually open up to the market.
Shengke Communication is the only company in China capable of mass-producing high-end chips above 12.8T, with its market share expected to rise to about 5% in 2024. Its Arctic series 25.6 Tbps chips perform at the international top level. Huawei’s self-developed chips mainly supply Huawei switches and have not been sold on a large scale.
Ethernet Chips
Shengke HW and ZTE chips can be utilized because they also produce switches, rapidly scaling up under the demand for domestic substitution.
In summary, we are optimistic about the demand for 800G switches brought by AI data centers, with Shengke being a key player in domestic substitution.
The trend of GaN drivers in humanoid robot joint motors
Computing power, transportation capacity, and electrical power