The rapid development of Artificial Intelligence (AI) technology is reshaping the performance boundaries of personal computing devices. In recent years, neural processing units (NPUs) integrated into processors have gradually become core components of AI PCs, especially in the mobile device sector. However, the desktop PC market still lacks dedicated AI hardware. AMD recently revealed that it is exploring the possibility of developing a standalone NPU for consumer PCs, a solution similar to the design of standalone GPUs, which can provide ordinary users with more powerful AI computing capabilities, further promoting the widespread application of edge AI.
An NPU is a hardware accelerator optimized for AI tasks, capable of efficiently processing matrix operations, deep learning inference, and other compute-intensive workloads. Compared to traditional CPUs and GPUs, NPUs have a higher energy efficiency ratio when executing AI tasks. For example, in image recognition, speech processing, or large language model (LLM) inference, NPUs can achieve fast computations with lower power consumption, making them an indispensable part of AI PCs. AMD launched its first x86 processor with an integrated standalone NPU—the Ryzen 7040 series—in 2023. This processor is based on the XDNA architecture, with an NPU computing power of 10 TOPS (trillions of operations per second). Subsequently, the Ryzen 8040 series increased the NPU computing power to 16 TOPS, while the Ryzen AI 300 series, set to be released in 2024, will push this number to 50 TOPS, surpassing Microsoft’s defined minimum standard of 40 TOPS for AI PCs.
The NPU in the Ryzen AI 300 series is based on the upgraded XDNA 2 architecture and supports the Block FP16 data format. This format combines the high performance of 8-bit integers (INT8) with the high precision of 16-bit floating-point (FP16), achieving a balance between storage requirements and computational efficiency. For instance, tests show that Block FP16 achieves an accuracy close to 99.9% of FP16 when processing the Llama2-7B model, while the throughput is nearly on par with INT8. This allows the Ryzen AI 300 series to excel in running local AI applications, such as Stable Diffusion XL Turbo image generation or RAG retrieval-augmented generation. Additionally, AMD supports frameworks like PyTorch, TensorFlow, and ONNX through a unified AI software stack, enabling developers to quickly port pre-trained models to the Ryzen AI platform, further lowering the development threshold for AI applications.
Despite significant advancements in AI performance for mobile processors, desktop PC users lack similar hardware options. Currently, consumer-grade desktop processors mostly rely on CPUs or GPUs to handle AI tasks, but neither of these hardware types is tailored for AI workloads. While CPUs have strong general computing capabilities, they are less energy-efficient; GPUs excel in parallel computing but are often unsuitable for all users due to high power consumption and cost. AMD’s standalone NPU solution is aimed at addressing this market gap. With a modular design similar to standalone GPUs, a standalone NPU is expected to provide desktop PC users with a flexible path for AI performance upgrades. Users can choose to add an NPU card based on their needs without having to replace the entire processor or motherboard.
The concept of a standalone NPU is not entirely new. Qualcomm’s Cloud AI 100 Ultra inference card has been launched for the enterprise market, focusing on cloud AI inference acceleration. Intel has also showcased similar AI acceleration card products. However, these solutions are primarily targeted at data centers or professional workstations, with high price and power consumption thresholds that are difficult for ordinary consumers to reach. If AMD’s standalone NPU can target the consumer market, it may lower the entry barrier for AI hardware. For example, content creators could use a standalone NPU to accelerate AI denoising in video editing, gamers might optimize real-time image enhancement through the NPU, and ordinary users could run chatbots or smart assistants locally, reducing reliance on cloud services.
From a technical perspective, AMD has the capability to develop a standalone NPU. The successful application of its XDNA architecture in mobile processors provides a technical foundation for the standalone NPU. The XDNA 2 architecture supports various data types (such as INT4, INT8, FP16, and Block FP16) and is optimized for common AI models (like Llama, Mistral, and Stable Diffusion). Furthermore, AMD’s experience in the GPU field also aids in the design of the standalone NPU. GPUs and NPUs share similarities in parallel computing and memory management, and AMD’s RDNA architecture design capabilities can provide references for the memory bandwidth and computational unit layout of the NPU. The unified memory architecture of the Ryzen AI Max series processors has achieved support for up to 128GB of system memory, ensuring smooth operation of large AI models.
In terms of market prospects, the potential of standalone NPUs depends on their positioning and ecosystem support. For ordinary consumers, the level of AI application adoption is key. Currently, the Copilot+ feature in Windows 11 (such as real-time subtitles, image generation, and Recall) has created a clear demand for NPU computing power, but these features are primarily aimed at mobile devices. Desktop PC users who want to experience similar functionalities must rely on high-performance GPUs or cloud services, while a standalone NPU could provide a more cost-effective local alternative. Additionally, the demand for AI acceleration cards is also growing in professional markets (such as video editing, 3D rendering, and scientific computing). If AMD can launch an affordable standalone NPU, it may attract a wide user base ranging from individual developers to small and medium-sized enterprises.
The development and promotion of standalone NPUs face several challenges. First is the fragmentation of the software ecosystem. Different manufacturers’ NPU hardware (such as AMD, Intel, and Qualcomm) have differences in instruction sets and APIs, making it difficult for AI applications to run uniformly across platforms. Although Microsoft’s Windows ML platform attempts to achieve “write once, run anywhere” through ONNX Runtime, developers still need to optimize for specific hardware. Secondly, the “killer app” scenarios for AI applications have not fully emerged. While there is strong demand for generative AI and edge computing, most consumers still view AI as a nice-to-have feature rather than a necessity. This may limit the market size for standalone NPUs, especially in the price-sensitive consumer market.
AMD’s exploration of standalone NPUs is still in its early stages, and specific product specifications and release dates have not yet been clarified. However, based on existing information, it is likely to be based on the XDNA 2 or a newer architecture, with computing power potentially exceeding 50 TOPS and power consumption kept within an acceptable range for desktop PCs (such as 50-100W). In terms of interfaces, PCIe 5.0 or OCuLink is expected to be the preferred choice to ensure high-bandwidth data transmission. If the price can be controlled in the range of $200-400, it may compete with mid-range GPUs, attracting users with limited budgets but needing AI performance.
AMD’s standalone NPU plan reflects the hardware development trend in the AI PC era. As AI workloads shift from the cloud to the edge, the demand for local computing power is becoming increasingly urgent. Whether for integrated NPUs in mobile devices or standalone NPUs for desktop PCs, AMD is attempting to secure a place in the AI computing market through collaborative innovation in hardware and software. In the coming years, the level of AI hardware adoption and the maturity of the application ecosystem will determine the success or failure in this field. For consumers, the emergence of standalone NPUs undoubtedly provides new possibilities for performance upgrades in desktop PCs.
Scan the QR code to join the hardware group discussion
July 2025 Graphics Card Ladder Ranking | July 2025 CPU Ladder Ranking
Contact for cooperation:[email protected]