Remember to star this public account ⭐️ to receive notifications promptly.
Source: ContentTranslated fromxda-developers.
Today, artificial intelligence has empowered various functions from search tools to video calls, yet some laptops still struggle to run these features smoothly. This is because certain CPUs are not designed for efficiently handling real-time AI tasks. When purchasing a computer for the first time, you might focus on the specifications of the CPU and RAM, without considering whether it is equipped with an NPU. This situation is beginning to change.
The NPU, or Neural Processing Unit, is designed to handle AI functions such as background blurring, real-time subtitles, and voice isolation without overloading the CPU. This not only extends battery life and enhances smooth operation but also allows applications to run quickly even when operating in the background for longer periods. These features sound great, but can they really assist in daily use? You will immediately feel the difference it brings, especially when running AI functions.
Using AI is indeed exciting, but its biggest drawback is the additional power it requires. As the tasks your computer processes increase, you will find yourself needing to charge more frequently.
It’s like adding some trivial tasks to your daily life. Each task seems small, but cumulatively, they can make your system more power-hungry. A study by Enovix even found that the energy consumed by an AI model to create an image is equivalent to charging your phone. If your laptop lacks an NPU, the GPU and CPU must work harder.
The NPU takes over AI tasks that are typically handled by the CPU. The NPU performs the same tasks with lower power consumption, thereby extending battery life and allowing your laptop to run without needing a charge.
Before you heard about NPUs, AI functions like background blurring or real-time subtitles were usually driven by the CPU or GPU. This meant that the speed of other applications would slow down, especially when you had multiple applications open for multitasking. Now, the NPU can handle these AI tasks, allowing your system to run faster. The NPU can perform matrix multiplication, which is a repetitive mathematical operation frequently used in AI, but the GPU and CPU were not designed for this purpose.
You can edit Google Docs, open multiple Chrome tabs to browse the web, or switch between applications. Even with AI effects running in the background, the system does not feel sluggish.
Once you use a laptop equipped with an NPU, you will notice a significant speed improvement. Applications open faster, run more smoothly, and do not lag when using AI-related programs.
On the Copilot+ PC, the NPU provides functionalities that older laptops cannot handle. Recall is one notable and somewhat controversial feature. It builds a searchable timeline that records everything you have seen or done on your PC. You can search for “the person in the red shirt from the last Team call” or “my photo on the Hollywood Walk of Fame,” and Recall can find it without knowing your saved location.
The Cocreator feature in Paint allows you to sketch rough shapes, describe your needs, and convert them into images. It responds to your actions, providing assistance even if you are not good at drawing.
The NPU also supports Windows Studio effects, including eye contact correction, portrait lighting effects, and filters that make videos more dynamic. These features can be applied instantly without slowing down other parts of the system. Even without an internet connection, real-time subtitles can now translate because all functions run on the device.
When your PC is equipped with a built-in NPU, AI features like Recall, Live Captions, and Cocreator no longer rely on the cloud. All functions run directly on your laptop, utilizing its hardware to process text, voice, and images. This means you can use these tools without leaving your device. Even searching for content from last week’s screen or adjusting the background of a video call can be done directly on the device.
For example, with Recall, Microsoft claims it saves snapshots of your activities in a private encrypted folder linked to your Windows account. You can limit what it records, pause recording, or prevent certain applications from being added. These snapshots are not uploaded or shared. You can still use the same features, but everything remains within your account.
According to Microsoft, no information is sent to the cloud or to the Redmond company. The snapshots are said to be encrypted and tied to your account, and you can only see them after Windows Hello confirms it is you.
An on-chip NPU may not be very eye-catching, but it quietly enhances the laptop experience. Applications respond quickly, AI functions run smoothly, and the battery maintains better endurance under pressure. If you use your computer for work, video calls, or creative endeavors, you will appreciate having an NPU and realize its presence.
The Battle Among Chip Giants
While GPUs have long been the default accelerators for various demanding workloads, the introduction of NPUs has shaken up the processing hierarchy, enabling PCs to run AI and machine learning workloads quickly without consuming as much energy as GPUs.
So far, the most popular implementation of NPUs in PCs has been integrating processors with CPUs and GPUs on a system-on-chip, such as Intel’s Core Ultra 200 series, AMD’s Ryzen AI 300 series, and Snapdragon X series chips.
However, there are also cases where PCs use independent NPU components that are separate from the CPU or system-on-chip. For example, before Intel introduced NPUs in its Core Ultra 100 series, Microsoft released a Surface Laptop in 2023 that used a standalone Intel Movidius vision processing unit (the predecessor of Intel’s NPU).
Then there is the recently released Dell Pro Max Plus laptop, which uses Qualcomm’s AI 100 PC inference card. Dell claims this device is “the world’s first workstation equipped with an enterprise-level independent NPU.” According to Dell, this inference card contains two Cloud AI 100 data center processors and 64 GB of LPDDR4x memory, allowing it to operate at a performance of 4.5 TOPS (trillions of operations per second) with a thermal range of up to 75 watts.
Additionally, some lesser-known companies, such as Encharge AI, are working to introduce independent NPUs. This California-based startup announced in May the launch of a 200 TOPS NPU with a power consumption as low as 8.25 watts, suitable for M.2-sized laptops; furthermore, the company also released a four-NPU PCIe card that provides approximately 1,000 TOPS of computing power, achieving what it calls “GPU-level computing capability at extremely low cost and power consumption.”
Recently, a senior PC executive at AMD revealed that the company is exploring the potential of independent neural processing units that could serve as alternatives to independent GPUs in PCs.
Rahul Tikoo, head of AMD’s client CPU business, confirmed during a briefing ahead of last month’s AMD Advancing AI event that the California-based company is “discussing” the use cases and potential opportunities for dedicated accelerator chips that are not GPUs but could be neural processing units (NPUs).
As Tikoo made these remarks, OEM manufacturers such as Lenovo, Dell Technologies, and HP are beginning to explore the use of independent NPUs and other types of dedicated accelerator chips as alternatives to GPUs for AI workloads in PCs. For instance, Dell announced last month that it would use a Qualcomm AI 100 PC inference card based on NPU in its new Dell Pro Max Plus laptop.
“This is a very new set of use cases, so we are closely monitoring this area, but if you want to enter this field, we do have solutions—we can do it,” Tikoo stated, having returned to AMD last year as senior vice president and general manager of the client business after a 12-year leadership role at Dell.
As for when AMD will launch such products, Tikoo (shown above) stated he could not “discuss future roadmaps” and added that it is “under (non-disclosure agreements).”
“But if you look at the breadth of our technology and solutions, it is not hard to imagine that we will achieve this goal soon,” he said.
The CTO of AMD system integration partner Sterling Computers told CRN last week that he believes AMD’s efforts to leverage AI engine technology acquired from Xilinx as a foundation for NPU components in Ryzen processors “opens up a broad path for the company to launch discrete products with faster NPU performance in the future.”
The company’s CTO, Christopher Cyr, stated, “If this specific NPU module can execute 50 TOPS (trillions of operations per second), then with two, it can reach 100 TOPS.” His company is located in North Sioux City, South Dakota, and ranks 54th on CRN’s 2025 Solution Provider 500 list.
However, he added that independent NPU solutions “must consume less energy than independent GPUs.”
Cyr stated that AMD’s efforts to support AI software on its processors (including NPUs) have impressed him. He cited AMD’s open-source project Gaia, which aims to run local large language models well on Ryzen-based Windows PCs.
He said, “They have made significant progress in leveraging the entire ecosystem.”
*Disclaimer: This article is original by the author. The content reflects the author’s personal views, and Semiconductor Industry Observation reprints it only to convey a different perspective, not representing Semiconductor Industry Observation’s endorsement or support of this view. If there are any objections, please contact Semiconductor Industry Observation.
END
This is the 4121st issue shared by Semiconductor Industry Observation. Welcome to follow.
Recommended Reading
★A Chip That Changed the World
★U.S. Secretary of Commerce: Huawei’s Chips Are Not That Advanced
★“ASML’s New Lithography Machine, Too Expensive!”
★The Quiet Rise of Nvidia’s New Competitors
★Chip Crash, All Blame Trump
★New Solutions Announced to Replace EUV Lithography
★Semiconductor Equipment Giants, Salaries Soar by 40%
★Foreign Media: The U.S. Will Propose Banning Software and Hardware Made in China for Cars
Star this account ⭐️ to receive notifications promptly, small account to prevent loss
Seeking likes
Seeking shares
Seeking recommendations
