Introduction

When we talk about 100G networks, many people’s first reaction is:

“Isn’t this bandwidth a bit excessive? Can ordinary server CPUs handle it?”

Indeed, the gigabit ports on home computers often do not reach full capacity, let alone 100G. However, the reality is that in data centers, AI training clusters, distributed storage, and telecom backbones, 100G has long been the “starting bandwidth.” In fact, in some scenarios, links of 200G, 400G, or even 800G are already in operation.

So the question arises: Where is the massive bandwidth of 100G actually used? This article will take you through how 100G networks play a role in modern computing architectures.

Data Centers

In data centers, the most common application of 100G is rack aggregation. Imagine a rack with dozens of servers, each equipped with 25G/50G network ports; if all traffic needs to go to the core switch, a “big pipe” is needed to handle it. This “big pipe” is the uplink of the Top-of-Rack (TOR) switch, which is typically 100G.

Therefore, 100G in data centers is not about a single machine running at full capacity, but rather about aggregating multiple small flows. This way, whether users are accessing websites, applications are running big data, or internal traffic exchanges in cloud services, they will not be bottlenecked by a “narrow pipe”.

AI Training and HPC

In AI training clusters, the memory bandwidth of GPUs can reach TB/s levels, providing immense computing power, but they need to frequently exchange gradients and parameter data. If the network is inadequate, GPUs will be left waiting for data, leading to idle computing power.

This is why in large model training scenarios, 100G/200G/400G networks have become standard. Combined with RDMA (RoCE / InfiniBand), data can be transferred between GPUs as if it were “memory direct connection,” with latencies as low as microseconds.

Here, 100G networks are the “lifeline” for GPUs: no matter how powerful the computing is, if the network does not keep up, it is like “a luxury car stuck in a narrow alley”.

Distributed Storage

Modern storage systems generally adopt a distributed architecture, such as Ceph, Lustre, NVMe-oF. A storage server often has dozens of NVMe SSDs, and the total bandwidth far exceeds the limits of a 10G network.

If a 25G network is used, the performance of the SSDs will be “locked down”; but with 100G, data can be smoothly transferred between computing nodes and storage nodes. This is particularly important for high IO scenarios such as databases, video on demand, and logging systems.

It can be said that 100G networks are the performance guarantee for distributed storage; without it, the speed of SSDs cannot be fully utilized.

Telecom and Operators

In operator networks, 100G is not even considered “high-end”. From the 5G core network to metropolitan networks and then to national backbone networks, 100G links are standard configurations, with 200G/400G already being deployed on a large scale.

The demands here differ from those of enterprises:

• • Catering to internet traffic for tens of millions of users
• • Supporting various services such as voice, video, gaming, and IoT
• • Requiring extremely high reliability and low latency

For operators, 100G is the basic bandwidth, like an overpass in a city; without it, the entire network would collapse.

Network Security and Virtualization

In security devices, firewalls, DPI (Deep Packet Inspection), and 5G UPF, massive packets often need to be processed at line speed. For example:

• • DPI needs to analyze the content of each data packet
• • Firewalls need to perform rule matching
• • NFV/cloud computing needs to implement virtual network cards, encryption/decryption, and isolation

In these scenarios, 100G is essential to ensure uninterrupted services. Processing such large traffic often relies on FPGA/ASIC/SmartNIC/DPU to offload most data plane operations from the CPU.

Thus, 100G networks here act like a moat, needing to ensure fast traffic while being secure and controllable.

Short Video Live Streaming Industry

VPU (Vision Processing Unit)

Currently, VPUs are increasingly used in the video streaming/short video/live streaming industry, especially in areas related to beautification, effects, and real-time video processing.

Real-time Beautification & Effects

• • Beautification, filters, green screen replacement, virtual backgrounds, and portrait segmentation are essentially image processing + AI inference.
• • Relying solely on the CPU would result in high latency and low frame rates; while GPUs can handle it, they consume a lot of power and can easily overheat and generate noise during long live streams.
• • VPUs are optimized for these scenarios, maintaining high frame rates + low latency even at low power consumption.

Video Encoding and Decoding

• • Streamers need to compress the raw video captured by the camera into H.264/H.265/AV1 streams to send to the live streaming platform.
• • If this step is handled by the CPU, it will consume a lot of computing power; VPUs/video acceleration engines can perform hardware encoding, which is both fast and energy-efficient.

Multi-channel Video Processing

• • Some streamers conduct “multi-camera live streaming,” processing multiple video streams simultaneously (main camera, secondary camera, screen capture).
• • VPUs can handle multiple inputs simultaneously, synthesizing them before streaming, avoiding excessive burden on the CPU/GPU.

Why do VPUs need high-bandwidth networks (100G)?

Video Data Volume is Huge

• • High-definition video streams (4K/8K), multiple camera inputs, and real-time transcoding require bandwidth in the tens of Gbps.
• • If deployed in the cloud (such as video conferencing, security monitoring), data must be transmitted over the network, and 100G can demonstrate its value.

Edge Computing Scenarios

• • In applications like smart traffic, security monitoring, and smart cities, edge nodes will connect to numerous cameras for real-time video stream analysis.
• • These nodes typically use video acceleration cards with VPUs, which then transmit data or features back via 100G networks.

Cloud Video Processing Services

• Platforms like ByteDance, Kuaishou, Bilibili, and YouTube have significant video transcoding/rendering demands in the backend.
• They utilize GPU + VPU hybrid cards, with 100G networks ensuring high-speed data transfer between storage and computing nodes.

Conclusion

With the explosion of AI and large models, 100G is even beginning to be replaced by 200G/400G/800G. For FPGA/IC practitioners, this implies that: high-speed interface design, protocol offload, and low-latency architecture are all worthy directions for in-depth exploration.

Previous Reviews

Ethernet Column

Implementing Verilog Simulation with Python: Basics

Other Technical Columns

Appendix A

Welcome to join the FPGA/IC exclusive AI knowledge base, where you can access more knowledge without downloading an app,just use the WeChat mini-program for one-click access.

Click the link to join with one click

Appendix B

Welcome to join the FPGA and digital IC learning group! Here, you can ask questions, share your learning experiences, or participate in interesting discussions.

Link: FPGA Chip Research Society

#FPGA#Digital IC#QoS#Quality of Service#Hardware Design#Pipeline#Master Device#Latency Sensitive#Greedy#Real-time#Bandwidth Allocation#Latency Control#Interconnect Systems#NoC#Memory Controller#Resource Arbitration#Priority Arbitration#Traffic Shaping#Bandwidth Utilization#Critical Path#System Performance#IC Design#Hardware Pipeline#Cache Loading#Context Recovery#Data Supply#Resource Coordination#Timing Assurance#Performance Optimization