Why Do Desktop Computers Need NPU?

Arrow Lake-S, which is the second generation desktop CPU (Core Ultra 200S series), has been released earlier this month, but has only recently hit the market with supporting 800 series chipset motherboards and OEM brand machines appearing one after another. For example, the machine below is said to cost six figures and is packed with a custom Gundam storage unit…

This PC is reportedly priced at six figures; along with the Lunar Lake laptop displayed on-site, there are about 30 models of the Core Ultra second-generation processor showcased this time…

Readers familiar with the Core Ultra processor products should know that the next PC processors Intel will push are the Core Ultra second generation CPUs codenamed Lunar Lake (Core Ultra 200V series) and Arrow Lake. The former targets high-end thin and light laptops, while the latter is aimed at high-performance laptops, desktops, workstations, and other applications.

From the perspective of AI PCs, at the recent Core Ultra media conference, Feng Dawei (Vice President of Intel Client Computing Group and General Manager of Marketing) reiterated the roles of different “Lakes” in the concept of AI PCs during his keynote speech: last year’s Meteor Lake kicked off the AI PC era, Lunar Lake leads the AI PC experience, while Arrow Lake “accelerates the popularization of AI PCs”.

Compared to Lunar Lake, Arrow Lake will become the absolute main force in Intel’s client CPU shipments, including desktops and mainstream priced laptops: not only is there Arrow Lake-S for desktops, but also the Arrow Lake-H series aimed at high-performance laptops coming early next year. Therefore, Arrow Lake, as a product that “accelerates the popularization of AI PCs,” especially with the introduction of NPU to desktop processors, is truly deserving of its name.

Recently, we have already analyzed the Arrow Lake desktop CPU products and their architecture in detail—readers who haven’t seen it are advised to go read it. To summarize the improvements of Arrow Lake-S (Core Ultra 200S) compared to Raptor Lake-Refresh (Core 14th generation), we believe it can be condensed into three points:

(1) The CPU adopts a new architecture and uses 3nm+ advanced packaging based on chiplet;

(2) Significant energy efficiency improvement, achieving similar performance with half the power consumption of the previous generation;

(3) Introduction of NPU to achieve AI PC enhancement in an XPU manner.

Based on these points, especially with the AI PC ecosystem further improving, with the official release of Core Ultra 200S, this article will discuss the role and value of Arrow Lake—although we have been busy with activities lately, we haven’t had time to fully experience the Core Ultra 9 285K processor that has already arrived…

There are indeed many visually appealing host devices displayed on-site; these three models are from Ming Long Tang, and Xing Chao Ping’s PC products;

Discussing Arrow Lake’s Performance, Efficiency, and Functional Improvements

This part will not reiterate the new core architecture of Skymont and Lion Cove, the storage subsystem, as well as the system-level micro-architecture and chiplet-based 2.5D advanced packaging technology of Arrow Lake, since many previous articles have already analyzed these.

From the results, the improvement of the two heterogeneous cores and corresponding cache, especially the enhancement of E-core, allows Arrow Lake’s CPU to exceed the multi-core performance of the previous generation’s 32 threads with 24 threads.

Feng Dawei stated at the Core Ultra 200S launch that in terms of single-thread performance, compared to the Ryzen 9950X, the Core Ultra 9 285K leads in all tests, with performance up to 18% ahead; multi-thread performance also has a 5-10% advantage.

The new architecture and process have significantly improved efficiency and reduced power consumption. Previously, Intel provided data showing that, based on Cinebench 2024 tests, Arrow Lake only requires half the power of Raptor Lake-Refresh to achieve the same performance; the power consumption for various daily application loads has also decreased by about 50%.

Comparing the CPU power consumption during the entire Procyon productivity test, Arrow Lake reportedly averaged 45% lower than competitors with nearly identical performance scores. We will also attempt to replicate these in subsequent tests. Even though most desktop users are not sensitive to power consumption, Intel also mentioned at the media conference that improving the energy efficiency ratio of processors will create space for further performance improvements in future products.

OEM manufacturers displayed desktop computers on-site, many of which reportedly use liquid cooling solutions, managing to keep CPU package temperatures around 80+°C under full load; idle states can even keep core temperatures around 30+°C… showing a significant difference in temperature control and energy consumption compared to previous generations of Core processors.

In addition to performance and efficiency, Intel also emphasized two major capabilities of Arrow Lake at this launch: connectivity and overclocking.

Regarding IO capabilities, in addition to Arrow Lake supporting Wi-Fi 6E, Bluetooth 5.3, 1Gbps Ethernet, and with platform expansion can connect Wi-Fi 7, Bluetooth 5.4, and 2.5Gbps Ethernet, the key point emphasized this time is that the processor itself supports Thunderbolt 4 (x2 ports) for the first time, meaning the chip has a built-in Thunderbolt controller and corresponding PHY (platform expansion can connect to four Thunderbolt 5 interfaces).

“This allows OEM/ODM to provide Thunderbolt interfaces on motherboards at a lower cost.” MSI demonstrated on-site two PCs using Arrow Lake, directly connected via Thunderbolt interface, using Thunderbolt Share for fast point-to-point file transfer of large sizes. Thunderbolt Share is also a software solution Intel has been promoting for the past two years, facilitating screen sharing, data transfer, and storage sharing between PCs.

HP Shadow Spirit

In terms of overclocking, although Intel mentioned at previous media conferences that the CPU core frequency overclocking space is not as large as before, Uncle Tony still demonstrated core overclocking capabilities of P-core at 6.8GHz and E-core at 6.4GHz on-site; memory can even be overclocked to a transmission rate of 12066 MT/s; even NPU frequency can be overclocked.

Intel’s own summary is “more finely controlled overclocking new features.” This includes core clock frequency stepping at 16.6MHz; SoC tile and Compute tile running independent BCLK—thus it has “dual base clock frequency”; DVLR bypassing, going around internal voltage management limits, “using external power” for extreme overclocking; P-core V/f curve control per core, E-core V/f control per cluster; adjustable die-to-die connection frequency to improve communication levels between dies, etc…

There is also a new batch of CKD clock-driven CUDIMM memory, which can further enhance the stability and reliability of high-speed data transmission, allowing the Arrow Lake platform to have greater potential for memory overclocking. OEM manufacturers on-site demonstrated >8000 MT/s memory, which seems to be the norm…

Is This AI Computing Power Really Useful?

Desktops need to be equipped with microphones, seemingly largely to give AI voice commands… For example, this colorful solution from Colorful allows the AI assistant to switch between different usage modes and even perform overclocking operations…

The actual situation seems to differ from our expectations. For instance, at this event, there were ISV developers demonstrating PC gaming AI assistants; there were even AI features based on contextual environments to switch the internal motherboard and fan lighting effects of the desktop host…

Feng Dawei mentioned, “Although compared to discrete graphics, 36 TOPS computing power seems a bit thin, the XPU architecture, including the newly introduced NPU, provides flexibility for application software.” Gao Yu (General Manager of Intel China Technical Department) specifically discussed the AI PC characteristics of desktop processors, particularly mentioning “AI Gaming”—that is, AI game assistants, providing “emotional value” and “game strategies,” and even offering competitive training.

He mentioned an interesting point of view: “Game AI assistant functions can be implemented in two intuitive ways: cloud AI or discrete graphics AI. We believe neither of these methods is ideal. Because cloud AI consumes network bandwidth and cloud computing power, it cannot guarantee low latency and consistent AI performance; running AI on discrete graphics will inevitably affect normal game rendering tasks, impacting the experience.”

In the “AI gaming” scenario, “game scene recognition tasks are deployed on NPU for high accuracy and speed; processing user voice input models run on CPU for quick responses; large models are deployed on iGPU for high throughput generation rate.” This allows discrete GPU to focus solely on game rendering, ensuring high frame rates and high visual quality.

“After crossing the bridge, you are getting closer to Guangzhi…”, adding the upper left map, the official physical外挂 is…

For example, during the demonstration of playing “Black Myth: Wukong”, Intel’s engineers specifically developed an AI game assistant for this game—able to provide various prompts based on the player’s current progress in the game, such as map, BOSS location, treasure chest location, etc.; even when fighting the BOSS in the Guanyin Zen Temple, it can recognize BOSS actions in real-time and prompt players to dodge and various strategy points; or players can actively ask the AI “how to fight Guangzhi” through voice, etc… These experiences are achieved through XPU collaboration.

Gao Yu believes that AI game assistants can exist not only as real-time gaming consultants but also provide emotional value, and offer targeted training for competitive games. For example, some AI game assistants developed in cooperation with Intel will encourage or tease players based on actual game progress, thus providing emotional value. In terms of competitive training, “Valorant” is said to support full map, all heroes, and various skill training; next month Intel will also “launch another AI assistant in collaboration with ‘League of Legends'”.

These indeed allow desktops to function as AI PCs, with NPU truly playing a role. Meanwhile, multiple OEM manufacturers demonstrated AI technology-based coordination with chassis lighting control, as well as collaboration with peripherals and desktop setups. For example, Mingxuan demonstrated that during competitive gaming, when a character dies, the chassis can flash red. In terms of experience, AI is indeed likely to become an enhancement factor in future gaming.

Yes, this is also a PC (FUXK 2024 supercar series)

Arrow Lake Will Scale Up, AI PC Ecosystem Will Be Built

It is worth mentioning that Zong Ye (General Manager of Intel China Client Business Department) mentioned in the opening speech that Intel expects the sales of desktop K series (i.e., unlocked CPU versions) to double in 2024, with the K series proportion continuing to rise.

According to recent online sales results, “thanks to the windfall of Double 11, U9K sold out upon launch”; offline, 28 internet cafes have “lit up Core Ultra,” and Dongyu eSports has updated 169 gaming hosts to Intel’s flagship product Ultra 9 285K, with related eSports activities ongoing.

As a generation of processors aimed at “accelerating the popularization of AI PCs,” in addition to the release of desktop CPUs, the upcoming Arrow Lake-H early next year will inject new vitality into the market. In our view, Arrow Lake and its NPU will bring AI PCs into the daily entertainment and work of more people, which is of considerable value for the continuous cultivation of the AI PC ecosystem.

During this Intel media event, we could clearly feel that there are more consumer-oriented, valuable AI applications emerging, no longer just OEMs randomly assembling chicken rib-like AI assistant Q&A. For instance, more AI applications are starting to emphasize the RAG method to achieve targeted customization of knowledge base reserves for Q&A there are local real-time transcription, translation, and customization for offline meetings, natural language interaction-based Office operations (typical such as complex Excel operations); or even directly conversing with iQIYI, discussing episodes with AI…

In fact, another main character of this event is Lunar Lake—it is the star product of AI PCs, and the more valuable AI PC applications mentioned above are all emerging on the Lunar Lake platform— the image gallery at the end will introduce this, as its total AI computing power far exceeds that of Arrow Lake. However, in terms of the popularization of AI PCs, the latter will still become the main force. Only when Arrow Lake scales up can it drive the enthusiasm of developers.

Gao Yu said that Intel is not only the most active in the speed, breadth, and depth of AI model adaptation; collaborating with over 150 ISVs globally, over 40 ISVs domestically to develop various AI PC applications; pushing the landing of five major scenarios with ISVs including knowledge assistants, office assistants, entertainment assistants, creative assistants, and vertical application assistants; and launching ISV acceleration plans to promote various AI PC software competitions. In terms of edge AI hardware infrastructure, Arrow Lake is tasked with the mission of scaling up.

Finally, it is worth mentioning that the surrounding support for Arrow Lake should be gradually in place, not only motherboards; for instance, we recently saw in a document from Intel aimed at the media that although the cooling solution for LGA1700 is still applicable to the Core Ultra 200S processor, there are also ODMs that have designed new cooling solutions, “specifically to adapt to the chip floorplan layout planning where the compute tile is located in the southwest corner, and the P-core and E-core are interspersed, distributing thermal density across the entire tile design.”

We expect to release the user experience of Arrow Lake processors next week, stay tuned.

Lunar Lake Appendix Image Gallery:

▲ Intel collaborated with Zhiyuan Huazhang to develop a development kit CodeGeeX for novice AI developers, which provides hardware acceleration for local operation and application development of domestic large models through AI frameworks such as OpenVINO, IPEX, and integrates the Zhiyuan Huazhang CodeGeeX large model programming assistant;

▲ Real-time transcription, summarization, translation of offline meetings—Intel collaborated with Beike Ruisheng on the AIPC Meeting Master, which runs completely locally; from the on-site experience, the accuracy seems quite good…

▲ Locally AI-generated PPT—without the need for discrete graphics, the effect is indeed somewhat astonishing—quickly generating a well-designed PPT based on the drafted outline, speed is quite fast, unsure of its practicality; from AiPPT.com;

▲ The first to be eliminated by AI PCs are probably those who are “skilled in using Excel, Word” and other tools, because Office documents can remove hyperlinks and formatting, and operations like summing and averaging tables can be solved by communicating with AI using natural language with KuTools AI, without needing to know how to operate… This may also become a common developmental direction for many complex operations apps, just as EDA has also begun to add generative AI assistants;

▲ A new batch of developed intelligent PC assistants generally incorporate the RAG method; RAG has always been, in our view, the key to making local knowledge and office assistant-type AI truly effective…

▲ The AI applications highlighted by Intel this time come from Evernote, for example, Evernote can intelligently generate tags based on user notes and organize classifications, which is quite practical; clearly addressing the information overload problem for humans, will evidently be one direction for the future development of knowledge base AI; at the same time, this approach of adding AI features to existing applications is something we genuinely welcome…

▲ iQIYI intelligent assistant, on-site staff demonstrated not only conversing with AI about plots but also asked questions like “Is ‘The Hidden Corner’ suitable for children to watch?” and the AI was able to provide very clear guidance…

▲ The least practical AI PC feature we found on-site was QQ Music’s ability to use AI to infer ordinary sound quality content to master-level quality… this is essentially a method of exchanging computing power for storage space, do you find it practical?

▲ The Wall Smart in the promotion stated that the edge model Wall Smart MiniCPM series includes the base model MiniCPM and multi-modal model MiniCPM-V, “opening edge-side ChatGPT moments,” “multi-modal direction achieving comprehensive benchmarking against GPT-4V level”;

▲ Xiangxin Digital Human: Users can upload a 3-5 minute live-action video to achieve a 1:1 replica, realizing an image, expression, and voice “indistinguishable from a real person”; subsequent content generation can run smoothly on the PC locally;

▲ More vertical applications we have previously introduced, including medical, legal consulting, etc… The above solution from Covestro is based solely on image processing + AI to achieve non-contact physiological and emotional index detection;

▲ HP StarBook Ultra14, which has an AI assistant built in, can answer various questions related to PC operations and usage, which is relatively practical. We have always believed that as operating systems become more complex, good user interaction will gradually shift towards AI assistants—even though many of these AI assistants are still in their infancy…

▲ The Dell XPS 13, which still looks stunning, has also switched to Lunar Lake

▲ Mini hosts equipped with Lunar Lake are also available, this one is from the Mechanic’s Creator