This time, the Chinese AI industry is truly different.
Recently, there has been a piece of news in the tech circle that, although seemingly ordinary, is incredibly exciting upon reflection—DeepSeek has released version 3.1, introducing the UE8M0 FP8 precision format, specifically optimized for domestic AI chips. Following the announcement, A-share chip stocks surged across the board, with the market fully charged.
This is not just a technical iteration; it could become a critical juncture for the Chinese AI industry to break through the “bottleneck” dilemma.
The ‘Shackles’ of 18 Years
To understand the significance of this step, we need to talk about NVIDIA’s CUDA.
What is CUDA? In simple terms, it is the “operating system” in the field of AI computing. From 2006 to now, NVIDIA has spent nearly 18 years building it into an almost unshakeable ecosystem.
Currently, the mainstream AI frameworks globally—such as PyTorch and TensorFlow—rely on CUDA; almost all AI training cannot do without NVIDIA’s GPUs; the code written by developers is basically all based on CUDA.
The most critical point is that CUDA is not completely open-source. NVIDIA keeps the core technology tightly in hand, like a “black box”; others can only use it, not modify it.
What is the result?The Chinese AI industry has long been developing while “looking at others’ faces”.
When they say to raise prices, we raise prices; when they say to impose restrictions, we impose restrictions; we don’t even have room for negotiation. This feeling is indeed very helpless.
UE8M0 FP8: The ‘Secret Weapon’ to Bypass CUDA
At this point, many may ask: Why can UE8M0 FP8 bypass CUDA? What is its principle?
Don’t worry, let me explain.
The ‘Routine’ of Traditional Floating Point Numbers
First, we need to understand the traditional floating-point formats. For example, FP16 (half-precision floating point) consists of 1 sign bit, 5 exponent bits, and 10 mantissa bits. This format has been used in AI computing for a long time, but it has a problem—it requires specialized hardware support to run efficiently.
NVIDIA’s CUDA is precisely optimized for these traditional floating-point formats. Its GPU architecture, computing units, and software libraries are all designed around these formats. This creates a “closed loop”: if you want to compute efficiently using these formats, you must use CUDA; if you want to use CUDA, you must use NVIDIA’s GPUs.
The ‘Breakthrough’ of UE8M0 FP8
UE8M0 FP8 is entirely different. Its core feature ispure exponent encoding, without sign bits and mantissa bits.
What does this mean? In simple terms, it discards the part used to represent “exact values” (the mantissa) in traditional floating-point numbers, retaining only the part used to represent “magnitude” (the exponent).
What are the benefits of this approach?
Simpler hardware implementation. Because there are no complex mantissa operations, the computing units of UE8M0 FP8 can be designed to be lighter and do not require specialized hardware acceleration units like traditional floating-point numbers.
No dependence on specific hardware architecture. The computation of UE8M0 FP8 is more like “integer operations” rather than “floating-point operations”. Almost all processors can execute integer operations efficiently without needing specialized hardware support like CUDA.
Larger dynamic range. Although precision is sacrificed, UE8M0 FP8 achieves a larger dynamic range through pure exponent encoding. This means it can represent a wider range of values, making it more suitable for scaling factor calculations in AI models.
Why Can This Bypass CUDA?
Now you understand, right? The core reason why UE8M0 FP8 can bypass CUDA is thatit fundamentally changes the computation method.
Traditional AI computing relies on the “floating-point operations + hardware acceleration” model, which is precisely CUDA’s strength. In contrast, UE8M0 FP8 adopts the “integer operations + software optimization” model, no longer relying on specialized hardware acceleration units.
It’s like before, we had to use a special pot (CUDA) to cook; now UE8M0 FP8 tells us: actually, we can make good food with an ordinary pot, the key is mastering a new cooking method.
The ‘Magical’ Effects of UE8M0 FP8
After discussing so many principles, what are the actual effects of UE8M0 FP8?
Significant Reduction in Memory Usage
One of the biggest advantages of UE8M0 FP8 is its ability to significantly reduce memory usage. Specifically:
- • Training a model with 70 billion parameters requires 140GB of GPU memory using FP16 format
- • However, using UE8M0 FP8 only requires 70GB of GPU memory
What does this mean? With the same hardware, larger models can be trained; with the same model, less hardware is needed. For resource-limited domestic chips, this is like a timely help!
Significant Speed Improvement
In core AI operations like matrix multiplication, the computation speed of UE8M0 FP8 is 2-3 times faster than FP16. It’s like a training task that originally took a day can now be completed in 4-8 hours.
Support for Larger Scale
Due to the efficiency improvement, UE8M0 FP8 can support larger Batch Sizes or longer contexts, such as 128K tokens. This is crucial for handling long texts and complex reasoning tasks.
Mixed Precision Training
UE8M0 FP8 also supports mixed precision training, reducing hardware resource requirements while maintaining model accuracy, making it particularly suitable for training large models with hundreds of billions of parameters.
DeepSeek’s ‘Forethought’
DeepSeek-V3.1 is the first large model to apply UE8M0 FP8, and the application of this technology has significantly improved the efficiency and response speed of the model in agent tasks.
But more critically, the DeepSeek team has long recognized the compatibility of UE8M0 FP8 with domestic chips. They have clearly stated that UE8M0 FP8 is designed for the “next generation of domestic chips” that are about to be released.
Although not explicitly stated, everyone understands that this likely refers to Huawei’s Ascend series.
This proactive adaptation of models to hardware is completely opposite to the previous approach of “hardware adapting to models”.
It’s like before, we always tried to find ways to make domestic chips adapt to NVIDIA’s CUDA ecosystem; now DeepSeek tells us: why not let the models adapt to the characteristics of domestic chips?
This shift in thinking could be the key to breakthroughs in domestic AI!
The ‘New Hope’ for Domestic AI
The emergence of UE8M0 FP8 is of milestone significance for the domestic AI chip industry. It is not only a major technological breakthrough but also an important boost to the domestic AI computing system!
In a global AI chip market still dominated by NVIDIA, UE8M0 FP8 provides a differentiated development path for domestic chips. Through collaborative design of software and hardware, domestic chips are expected to achieve more efficient computing in specific scenarios, gradually narrowing the gap with international leading levels. Moreover, as hardware computing power is fully unleashed, it is believed that the gap between domestic large models and models like Claude will further shrink, or even surpass.
UE8M0 FP8 is just the first step. For domestic AI chips to truly achieve a comprehensive breakthrough, a process and a lot of facts are still needed.
But at least, we have found a path of our own, no longer completely “looking at others’ faces”!
Li Zi KK,
A founder of AI products swimming in the AI waveLike, watch, and follow Let’s talk about technology, products, and the future together 🚀