From Mystery to Shock: Understanding NPU Chips
In today’s rapidly advancing technology, artificial intelligence (AI) has become the core force driving the transformation of the era. Behind AI technology, there lies a “super brain” quietly supporting it, which is the NPU chip. NPU, or Neural Processing Unit, is a hardware accelerator specifically designed for deep learning algorithms. Its emergence has completely changed the predicament of traditional chips in processing AI tasks, injecting strong momentum into the development of AI.
The design inspiration for NPU chips comes from the neural network structure of the human brain. The human brain processes information through a complex network of neurons and synapses, while NPU chips simulate this structure to enhance data processing efficiency. Compared to traditional CPUs (Central Processing Units) and GPUs (Graphics Processing Units), NPU chips have a higher energy efficiency ratio and faster processing speed when handling deep learning tasks. In simple terms, the CPU is like a versatile “jack of all trades,” adept at handling various complex logical operations and general tasks, but when faced with the massive matrix operations and parallel processing required in deep learning, it can be somewhat overwhelmed; the GPU, on the other hand, is like a powerful “graphics expert,” excelling in graphics processing and large-scale parallel computing, but still cannot match the NPU chip in terms of energy consumption and specificity. The NPU chip, however, is a “special forces” unit born for deep learning, specifically designed and optimized for neural network computing tasks, enabling it to efficiently complete these complex calculations with lower energy consumption and achieve higher computational performance.
The Working Principle of NPU Chips: Simulating the Wisdom of the Brain
The working principle of NPU chips is essentially a brilliant simulation of the human brain’s neural network, aimed at enabling computers to process information as efficiently as the human brain does, achieving intelligent computation.
In the human brain, neurons are the basic units of information processing, interconnected via synapses, forming an extremely complex neural network. When we perceive external information, neurons are activated and transmit electrical signals through synapses, which continuously pass, process, and integrate signals within the neural network, ultimately allowing us to make various decisions and reactions. For example, when we see an image, visual neurons convert the image information into electrical signals, which are transmitted through the neural network to the visual cortex of the brain for analysis and recognition, allowing us to understand what we see.
NPU chips borrow from this way of working, simulating the functions of neurons and synapses through hardware circuits. They contain numerous computational units that function like neurons in the brain, capable of rapidly processing input data. Additionally, NPU chips employ efficient interconnect structures, similar to synapses in the brain, used for data transmission between different computational units. When processing image recognition tasks, NPU chips input image data into the computational units, which simulate the working of neurons, extracting and analyzing features from the image. Through parallel computing and efficient interconnect structures, NPU chips can complete a large number of computational tasks in a very short time, quickly and accurately identifying objects within images.
To further improve computational efficiency, NPU chips also employ specialized technologies. Tensor computation acceleration technology is designed specifically for processing multi-dimensional arrays (tensors), optimizing matrix multiplication and convolution operations, which are the most computationally resource-intensive parts of deep learning models. Through this optimization, NPU chips can significantly enhance computation speed and efficiency when handling deep learning tasks. Furthermore, NPU chips utilize a data flow architecture that allows data to be transmitted directly between processing units, reducing data transmission latency and power consumption. They are also equipped with high-speed caches and on-chip memory to lessen reliance on external memory, further improving processing efficiency.
The Rise of NPU Chips: A History of Development
The development history of NPU chips is a tale of innovation and breakthroughs in technology. Each evolution injects new vitality into the development of artificial intelligence, propelling humanity into the intelligent era.
The origin of NPU chips can be traced back to the early 21st century. With the rise of deep learning algorithms, traditional CPUs and GPUs gradually revealed issues of inefficiency and high energy consumption when processing neural network computing tasks. To meet the urgent demand for computational power in AI development, researchers began exploring hardware accelerators designed specifically for neural network computations, leading to the concept of NPU chips.
In 2013, Qualcomm launched the first mass-produced processor with an innovative architecture design, Zeroth, which more closely simulates biological neural cells using circuits. Qualcomm referred to this new processing architecture as NPU, enabling them to introduce capabilities such as machine learning into mobile platforms, thus pioneering the application of NPU in mobile terminals. The year 2016 marked a significant milestone in the development of NPU chips. That year, Cambricon released the world’s first commercially available AI processor for terminals, surpassing traditional CPUs and GPUs in performance and energy consumption across key intelligent processing fields like computer vision, speech recognition, and natural language processing, thereby heralding the advent of terminal AI. In the same year, the National Key Laboratory of Digital Multimedia Chip Technology successfully developed China’s first embedded neural network processing (NPU) chip, “Star Intelligence No. 1,” becoming the world’s first embedded video capture and compression encoding system-level chip with deep learning AI capabilities.
Since then, NPU chip technology has entered a rapid development phase, with major tech companies increasing R&D investment and launching a series of representative NPU chip products. In 2017, Huawei collaborated with Cambricon to integrate an AI module into the Kirin 970 for the first time, marking the beginning of the AI era in smartphones; Apple also launched the iPhone X with an AI bionic engine. In 2018, MediaTek introduced the AI platform NeuroPilot, integrating the AI processor APU; Samsung first integrated NPU in Exynos 9820, ushering smartphones into the AI era with features like intelligent photo classification and app preloading, while AI technologies such as facial recognition and deep learning began to be widely applied in smartphones.
In the cloud domain, 2019 saw Alibaba release the Hanguang 800 chip, an AI chip NPU using ASIC technology for cloud inference. In performance tests, Hanguang 800 excelled, achieving an inference performance of 78,563 IPS and an energy efficiency ratio of 500 IPS/W on the ResNet-50 image recognition task, leading similar products at the time. This achievement showcased the powerful capabilities of NPU chips in large-scale data processing and inference tasks in the cloud, providing strong support for the development of cloud computing and AI services. That year, Huawei also released the Kirin 810 smartphone, utilizing Huawei’s Da Vinci architecture NPU, further enhancing the performance and application range of NPU in mobile chips.
In recent years, driven by the wave of generative AI, NPU chips have encountered new development opportunities and challenges. Qualcomm has continued to innovate in the NPU field; in 2022, the Hexagon NPU in Snapdragon 8 Gen 2 achieved numerous important technological enhancements, including dedicated power transmission tracks, micro-slice inference, INT4 technology, and transformer network acceleration. In 2023, Qualcomm further upgraded the NPU microarchitecture and related capabilities in Snapdragon 8 Gen 3, achieving a 98% performance increase and a 40% energy efficiency improvement for sustained AI inference, thus better supporting generative AI in fields like image generation and text creation. These technological upgrades enable NPU chips to better meet the rigorous demands for computational power and efficiency of generative AI.
Distinct Advantages: The Unique Features of NPU Chips
NPU chips exhibit numerous unique advantages in the field of artificial intelligence computing, making them stand out compared to CPUs and GPUs and becoming a key force in driving AI development.
Ultra-High Computational Efficiency
In deep learning tasks, the computational efficiency of NPU chips is astonishing. For instance, in image recognition, a traditional CPU may take seconds or even tens of seconds to complete complex feature extraction and classification computations for a high-definition image. In contrast, NPU chips, with their hardware architecture optimized specifically for neural network algorithms, can accomplish the same task in milliseconds. This is because NPU chips utilize a large number of parallel computing units, akin to having numerous efficient “assistants” that can simultaneously process different parts of the image, greatly accelerating computation speed. In security surveillance, cameras equipped with NPU chips can quickly and in real-time identify faces, vehicles, and other target objects, providing strong support for security prevention.
Outstanding Energy Efficiency Ratio
NPU chips demonstrate exceptional performance in terms of energy efficiency ratio. Compared to GPUs, NPU chips can reduce energy consumption by several times, even dozens of times, when executing the same AI computing tasks. This is particularly significant for devices with strict energy consumption requirements, such as smartphones, smartwatches, and drones. In smartphones, NPU chips can run various AI applications like voice assistants and smart photography in low-power states, extending battery life while reducing device heat and improving user experience. In data centers, extensive use of NPU chips can lower energy consumption, reduce operational costs, and align with environmental protection principles.
Powerful Parallel Processing Capability
The parallel processing capability of NPU chips allows them to handle large-scale data with ease. They can process multiple tasks simultaneously, achieving true parallel computing. In natural language processing, when extensive text needs to undergo sentiment analysis or semantic understanding, NPU chips can split the text data into multiple parts and process them simultaneously, greatly enhancing processing efficiency. In contrast, CPUs, due to their serial processing characteristics, must handle tasks one at a time, resulting in relatively slower speeds. Although GPUs possess some parallel processing capabilities, they still fall short of NPU chips in optimization for neural network computations.
Application Showcase: The Diverse Fields of NPU Chips
The powerful capabilities of NPU chips have led to their widespread application across numerous fields, providing strong support for the intelligent development of various industries. From the smartphones we use daily to self-driving cars that lead the future of transportation, to smart home systems that construct intelligent living, NPU chips play an indispensable role.
Smartphones: Ushering in a New Era of Intelligent Experience
In the smartphone realm, the application of NPU chips has enabled a qualitative leap in the intelligence level of phones. Take the OPPO Find X5 Pro, for example; this phone is equipped with OPPO’s self-developed NPU chip—MariSilicon X. In photography, this chip endows the phone with unprecedented computational power, achieving 4K ultra-clear night scene video recording, fast imaging speed, ideal noise control, and high picture purity, with the ability to capture the contours of streetlights and trees even in night scenes, making it easy for users to take high-quality photos and videos. Additionally, Huawei’s Kirin series chips also integrate powerful NPUs, enabling features like intelligent voice assistants and intelligent scene recognition photography. The intelligent voice assistant can quickly and accurately understand user voice commands, facilitating operations like making calls, querying information, and setting reminders, providing users with a convenient interactive experience; intelligent scene recognition photography can automatically identify shooting scenes, such as landscapes, people, and food, optimizing based on different scenarios for more vivid and beautiful photos.
Autonomous Driving: Driving Towards the Future of Intelligent Mobility
In the field of autonomous driving, NPU chips are the core element in achieving vehicle intelligence. They are responsible for processing complex neural network computing tasks, meeting the performance demands of high throughput and low latency in autonomous driving, while optimizing power consumption to meet the stringent energy efficiency requirements of onboard systems. The Turing chip, independently developed by Xpeng Motors, features a 40-core processor and 2xNPU, capable of running large models with up to 30 billion parameters, specifically designed for Level 4 autonomous driving. A single Turing chip can provide a Level 3+ advanced intelligent driving experience, while a configuration with two Turing chips can achieve Level 4 autonomous driving. NPU chips are primarily applied in three critical layers of autonomous driving: perception, decision-making, and execution. In the perception layer, NPU chips can rapidly process the vast amounts of data collected by cameras and radars, achieving precise recognition of roads, vehicles, pedestrians, and other targets; in the decision-making layer, based on the information provided by the perception layer, NPU chips quickly make driving decisions such as acceleration, deceleration, and turning by running complex algorithms; in the execution layer, NPU chips convert decision outcomes into control signals to manage the vehicle’s power and steering systems, achieving autonomous driving.
Smart Homes: Building a Convenient Intelligent Life
NPU chips also bring new opportunities for the development of smart homes. Amlogic has launched over 15 chips integrated with Neural Processing Unit (NPU) capabilities, covering multiple smart home application scenarios. For instance, the S905X5 chip is the industry’s first commercially available chip integrated with 4K and AI features, boasting strong edge AI capabilities that can execute inference tasks locally, achieving functions such as local subtitle translation, real-time translation, meeting transcription, and game motion recognition. In smart home systems, NPU chips enable smart speakers, smart cameras, and smart appliances to become more intelligent. After equipping smart speakers with NPU chips, they can more accurately recognize user voice commands, understand semantics, and respond quickly to achieve functions like music playback, information querying, and home control; smart cameras, leveraging the powerful computing capabilities of NPU chips, can achieve real-time facial recognition, abnormal behavior detection, and automatic alarm functions, ensuring home security; smart appliances with NPU chips can automatically adjust their operational states based on user habits and environmental changes, achieving intelligent operation; for example, smart air conditioners can automatically adjust temperature and wind speed based on indoor temperature, humidity, and human activity, providing users with a comfortable living environment.
Future Blueprint: The Development Trends of NPU Chips
Looking ahead, NPU chips will continue to evolve across multiple key dimensions, opening up broader horizons for the development of artificial intelligence.
In terms of integration, with the continuous advancement of semiconductor process technology, NPU chips will achieve higher integration levels. In the future, we can expect to see smaller and more powerful NPU chips. These chips will be able to integrate more computational units and functional modules within a smaller space, improving the overall performance of devices and providing more possibilities for compact and lightweight designs. In wearable devices like smartwatches and smart glasses, highly integrated NPU chips will enable devices to maintain a compact and lightweight form while possessing robust AI processing capabilities, enhancing user wearing experiences.
Improving energy efficiency ratios is also an important direction for the future development of NPU chips. As people become increasingly concerned about energy consumption and environmental protection issues, NPU chips will place greater emphasis on optimizing energy efficiency ratios. By adopting advanced process technologies, optimizing circuit designs, and innovating architectural designs, future NPU chips will significantly reduce energy consumption while maintaining high performance. This will allow NPU chips to find broader applications in energy-sensitive fields such as mobile devices and IoT devices. In smart cameras, low-power NPU chips can run for extended periods, achieving real-time video monitoring and intelligent analysis while reducing battery replacement frequency and lowering usage costs.
The application fields of NPU chips will also continue to expand. Beyond existing areas like smartphones, autonomous driving, and smart homes, NPU chips will play important roles in more industries such as healthcare, education, and finance. In healthcare, NPU chips can assist in medical image diagnosis, quickly and accurately identifying lesions in X-rays and CT scans, providing doctors with more reliable diagnostic support; in education, NPU chips can enable personalized learning, offering customized learning plans based on students’ learning situations and characteristics, thereby improving learning efficiency; in finance, NPU chips can be used for risk assessment and fraud detection, ensuring the safety and stability of financial transactions.
As artificial intelligence technology continues to evolve, NPU chips will usher in an even more brilliant future. They will become the core force driving intelligent transformation across various industries, bringing more convenience and surprises to our lives. Let us wait and see and witness the infinite possibilities that NPU chips will create in the future.
Summary and Outlook
As the core hardware in the field of artificial intelligence, NPU chips are profoundly changing our lives and society with their unique design philosophy, powerful computational capabilities, and broad application prospects. They have not only brought revolutionary changes to fields like smartphones, autonomous driving, and smart homes but also laid a solid foundation for the future development of artificial intelligence.
With continuous technological advancements, NPU chips will achieve greater breakthroughs in integration, energy efficiency ratios, and application fields. They will become smaller and more efficient, capable of functioning in more devices, bringing us a smarter and more convenient living experience. Meanwhile, the development of NPU chips will also promote the application and innovation of artificial intelligence technology across more industries, facilitating the intelligent upgrade and development of various sectors.
In this era full of opportunities and challenges, the development of NPU chips deserves our attention. Let us look forward to NPU chips creating more miracles in the future and making greater contributions to the progress and development of human society.
