Editor’s Note: On November 13, 2025, Arm China launched the “Zhouyi” X3 NPU IP product. As the first major product born under the “All in AI” product strategy, it boasts impressive performance metrics: 8-80 FP8 TFLOPS, single-core bandwidth of 256GB/s, prefill computing utilization rate of 72%, and decode effective bandwidth utilization rate exceeding 100%, enabling a tenfold enhancement in AIGC large model capabilities.
With the rapid development of AI large model technology, edge devices such as smartphones, AI PCs, and smart cars are experiencing explosive demand for AI capabilities. From multimodal voice assistants to real-time image generation, from smart cockpit interactions to assisted driving decisions, edge devices need to handle increasingly complex AI computing tasks.
However, deploying AI on edge devices faces multiple challenges, including limited computing power, high energy efficiency requirements, bandwidth bottlenecks, and high development thresholds. Achieving efficient operation of large models within limited power and computing resources has become a critical challenge that the industry urgently needs to overcome.
As is well known, in the semiconductor field, “forward-looking layout for product directions over the next five years” has become an industry consensus, covering IP development, chip, and terminal product R&D and mass production.
In response to demand, Arm China’s “Zhouyi” X3 adopts a universal architecture design that balances CNN and Transformer, supporting all types of computing power and enhancing floating-point operations (FLOPS), facilitating the critical transition from fixed-point to floating-point computing, and meeting the computing needs of large models, laying a technological foundation for the deployment of Gen AI, Agentic AI, and Physical AI on edge devices over the next five years.
Hardware Architecture: Latest DSP+DSA Architecture Designed for Large Models
The “Zhouyi” X3 is based on the latest DSP+DSA architecture designed specifically for large models, achieving upgrades in four dimensions: computing efficiency, bandwidth, precision adaptation, and task processing, precisely addressing the challenges of running large AI models on edge devices and setting a new benchmark for AI computing efficiency.
Transitioning from fixed-point to floating-point computing, a single cluster possesses 8-80 FP8 TFLOPS of computing power and can be flexibly configured, with single-core bandwidth reaching up to 256GB/s, supporting the essential W4A8/W4A16 computing acceleration modes for running large models on edge devices. It integrates Arm China’s self-developed decompression hardware (WDC), allowing for an additional approximately 15% equivalent bandwidth gain through hardware decompression after lossless compression of large model weights, significantly enhancing computing efficiency and density.
At the same time, the “Zhouyi” X3 integrates an AI-specific hardware engine (AIFF) along with a dedicated hardened scheduler, achieving an ultra-low CPU load of 0.5% and low scheduling latency, providing a highly efficient and smooth experience when the NPU processes multiple AI tasks in parallel.
Software Ecosystem: Optimizing End-to-End Performance of Large Models, Supporting Custom AI Development
The “Zhouyi” X3 is equipped with a comprehensive and user-friendly Compass AI software platform that fully optimizes the end-to-end performance of large models. The platform is widely compatible with mainstream AI frameworks such as TensorFlow, ONNX, and PyTorch, featuring industry-leading support for dynamic shapes in large models, and supporting mainstream quantization schemes like GPTQ, as well as models from the Hugging Face model library, including LLM, VLM, VLA, and MoE.
Additionally, Arm China is actively building an open ecosystem. The Compass AI software platform has successively open-sourced core components such as the Parser, Optimizer, Linux Driver, TVM, and internal IR formats, and offers a rich set of debugging tools to meet developers’ white-box deployment needs. It also supports a more user-friendly DSL operator programming language, allowing customers to develop their custom operators using the Compiler, Debugger, and DSL language within the platform in deep development mode, or create their own model compilers using tools like Parser and Optimizer, greatly enhancing development efficiency.
The “Zhouyi” Compass AI software platform now supports over 160 operators and more than 270 models, significantly improving development and deployment efficiency, helping customers rapidly achieve AI application innovation and implementation.
Performance Enhancement: Ensuring Edge Large Models Run Smoothly and Respond Quickly
The “Zhouyi” X3 adheres to the product principle of “hardware-software collaboration, full-cycle service, and customer success,” providing full-link support from hardware and software to after-sales service. With forward-looking design, professional team delivery, and deep service investment, it comprehensively assists customers in the successful commercialization of large model products.
The “Zhouyi” X3 has achieved multiple innovations at the architectural level, integrating several key features that have garnered industry attention.
1. Integrated self-developed decompression hardware (WDC): This allows for an additional 15%-20% equivalent bandwidth increase through hardware decompression after lossless compression of large model weights.
2. New W4A8/W4A16 computing acceleration modes essential for running large models on edge devices: These modes perform low-bit quantization of model weights, significantly reducing bandwidth consumption and supporting the efficient migration of cloud large models to edge devices.
3. Integrated AI-specific hardware engine (AIFF) and dedicated hardened scheduler: This achieves an ultra-low CPU load of 0.5% and low scheduling latency, flexibly supporting multi-task scenarios on edge devices and any priority scheduling scenarios, ensuring immediate response for high-priority tasks.
4. Supports multi-precision fused computing (int4/int8/int16/int32/fp4/fp8/fp16/bf16/fp32) and strong floating-point computing: This can flexibly adapt to the data type requirements from traditional CNNs to cutting-edge large models for edge deployment in smartphones, AI PCs, and smart cars, balancing performance and energy efficiency.
With the support of these key technologies, the “Zhouyi” X3 achieves excellent computing performance: a single cluster supports up to 4 cores, with 8-80 FP8 TFLOPS of computing power that can be flexibly configured, and single-core bandwidth reaching up to 256GB/s. Compared to the “Zhouyi” X2 product, the CNN model performance of the “Zhouyi” X3 has improved by 30%-50%, with multi-core computing linearity reaching 70%-80%. Under the same computing power specifications, the AIGC large model capabilities have achieved a tenfold increase compared to the previous generation product, thanks to a 16-fold increase in FP16 TFLOPS, a 4-fold increase in computing core bandwidth, and over a tenfold improvement in Softmax and LayerNorm performance.
With the super capabilities of the “Zhouyi” X3, edge large model inference will be more efficient.
Test results show that in the Llama2 7B large model tests, the “Zhouyi” X3 NPU IP achieved a computing utilization rate of 72% during the prefill phase, and with the support of Arm China’s self-developed decompression hardware (WDC), it achieved an effective bandwidth utilization rate exceeding 100% during the decode phase, far exceeding the industry average, meeting the high throughput demands of large model decoding.
Conclusion
The launch of the “Zhouyi” X3 NPU IP marks the official start of Arm China’s “All in AI” product strategy. From the basic perception of “Zhouyi” Z1 to the complex cognition of “Zhouyi” X3, this technological evolution path clearly indicates that edge AI is transitioning from single-function implementations to a new stage of “universal accessibility” that integrates multiple models and adapts to dynamic scenarios.
Source | Arm ChinaRecommended Reading——
SoftBank once considered acquiring Marvell to merge it with ARM to create a chip giant
Arm-compatible CPUs have crossed 5GHz for the first time!
Intel partners with NVIDIA, while TSMC, AMD, and Arm have mixed fortunes
Arm executives indirectly respond to Xiaomi’s self-developed chip controversy!
Arm CEO: We will manufacture our own chips!
Arm abandons the Cortex brand! Launches five new brands to meet new challenges!
Arm has sold its Artan foundational IP business to Cadence
