Introduction: At the recent SIGGRAPH conference, Arm announced the NSS super sampling technology aimed at mobile gaming—similar to DLSS, this is an AI-based super sampling solution; features like AI frame generation and ray reconstruction are also on the way. Next year, Arm’s GPU IP may undergo significant changes…
Readers of Electronic Engineering Magazine, who are interested in graphics technology and gaming reports, should be familiar with super sampling and frame generation technologies like DLSS and XeSS that are applied in desktop environments. NVIDIA’s DLSS technology has evolved to encompass various technical solutions including super resolution, ray reconstruction, and multi-frame generation.
When targeting gaming and professional visual applications, the goals of these technologies are clear: the GPU renders a portion of the pixels, and AI generates the remaining pixels based on the rendered pixels; combined with super sampling and multi-frame generation, these technologies have become so advanced that for every pixel rendered, 15 pixels can be generated—over the past two years, we have experienced many such technologies; ultimately achieving higher game resolutions and dynamic frame rates with lower load and power consumption enhances the experience.
However, similar technologies have primarily thrived in the desktop and gaming console markets: as a technology that can effectively reduce power consumption and improve efficiency, it has not been widely adopted in mobile devices like smartphones, which is quite regrettable. Even in the past two years, mobile chip companies, including MediaTek, have attempted to lay out super sampling technologies, and earlier this year, Arm officially opened up its Arm Accuracy Super Resolution technology (Arm ASR)—though most of these are based on traditional time-domain upscale solutions without utilizing AI.
Fortunately, at the recent SIGGRAPH summit, Arm officially released its neural technology for mobile gaming, as well as the NSS (Neural Super Sampling) technology that is set to be applied soon. Arm NSS can essentially be seen as the future standard answer for AI super sampling on mobile devices, which will undoubtedly provide immense value in enhancing gaming experiences on smartphones and tablets—after all, we know that super sampling has almost become a standard in the PC and console gaming markets.
AI-Generated Pixels: Possible on Mobile
At this year’s GDC, Arm fully opened its Arm ASR super sampling technology—this is a super sampling solution based on AMD FSR 2, optimized for mobile platforms. Readers who pay attention to gaming and graphics technology should know that since it is based on FSR 2, its super sampling principles are relatively traditional, unlike DLSS which is AI-based.
At a recent media event, Steve Steele, Product Management Director of Arm’s Client Division, explained that with historical frames, object motion information in the scene, and depth information, Arm ASR can perform image super sampling, reduce overhead and power consumption, and improve game frame rates. Currently, studios behind games like Fortnite and Infinite Lullaby have begun utilizing Arm ASR technology.
Initially, we thought that with Arm ASR and the existing architecture design of Arm Immortalis/Mali GPU IP, Arm might not consider AI-based super sampling or frame generation technologies for the time being. However, it was unexpected that Arm soon released its neural technology, and the NSS super sampling solution based on this neural technology is now ready—using AI to upscale low-resolution images to high resolution (e.g., 540p to 1080p). Further details will be introduced later.
Additionally, Steve mentioned that for gaming, Arm is not only preparing the NSS super sampling solution, but will also introduce NFRU (Neural Frame Rate Upscaling) and NSSD (Neural Super Sampling and Denoising). From Steve’s introduction at the media event, NFRU is expected to be an AI-based frame generation solution, while NSSD is similar to the ray reconstruction introduced in DLSS 3.5, which helps reduce ray tracing load.
The neural network used in NFRU will be “tightly coupled with new hardware,” as “we have added hardware to accelerate the generation of motion vectors in the GPU, tracking pixel movement in the scene”—this is a common solution for contemporary real-time interactive frame generation, similar to earlier DLSS which also relied on “engine motion vectors”; ultimately, “it can upscale 30fps content to 60fps with low overhead.”
As for NSSD, Steve stated that this is an “NSS extension for ray tracing content.” “For fully path-traced content, thousands of rays originating from each pixel reflect multiple times in the scene before reaching the light source, allowing the graphics engine to accurately calculate the brightness of each pixel”—this point was discussed in our analysis of ray tracing technology on the Electronic Engineering Magazine WeChat video account. “However, path tracing technology incurs significant overhead, even for desktop systems.”
By combining path tracing applications with neural networks like NSSD, the number of rays the GPU needs to process can be effectively reduced, “and neural technology can be used to reconstruct missing details and infer data—not only from adjacent pixels but also based on historical frame information.” During the Q&A session, Steve mentioned that based on various low-overhead ray tracing solutions, such as random ray casting, and fewer rays with smarter sampling, combined with AI to reconstruct missing details—especially the crucial denoising process—fewer ray samples and calculations can introduce various noise to the image, hence the “D” in NSSD stands for the final presentation of a clean, natural image based on AI.
NSS Neural Super Sampling: Coming Next Year
Among the three features based on Arm’s neural technology mentioned above, the latter two (NFRU and NSSD) will likely take a bit longer to be implemented. However, NSS neural super sampling is already on the verge of release. It is particularly noteworthy that the implementation of these technologies typically requires a combination of software and hardware, as mentioned earlier, frame generation requires motion vector analysis.
From historical experience, AI super sampling clearly requires dedicated AI acceleration units, and considering the low-latency requirements of gaming interactions, these acceleration units must also be tightly coupled with the GPU’s rendering pipeline. Arm provides data indicating that the latency introduced by NSS in the entire pipeline is about 4ms, which is essential for real-time rendering and interaction in games.
Steve stated: “The technology we released at SIGGRAPH, this neural accelerator is deeply embedded in the Arm Mali GPU.” He described the accelerator as a “mini-NPU within each shader core,” used to support algorithms like neural super sampling. During the presentation, he indeed mentioned that the Arm GPU IP set to be launched in 2026 will include a neural accelerator. We can look forward to the subsequent release of new Arm GPU IP.
Returning to the NSS technology itself, Steve stated that based on NSS technology, games can render at a resolution of 540p, achieving 2x super sampling through AI, ultimately outputting 1080p images. “Compared to full-frame rendering using traditional solutions, the GPU load is reduced by up to 50%.” Therefore, “developers can choose how to utilize these saved resources, such as reducing the overall power consumption of the game (extending battery life), or improving frame rates and visual quality.”
The image above shows a comparison of the rendering effects in the scene named “Enchanted Castle” from 540p to 1080p NSS images compared to native 1080p rendering, Arm ASR’s traditional time-domain super sampling solution, and similar super sampling solutions on desktop platforms.
In fact, the true test of AI super sampling technology’s ability to achieve image quality levels lies in dynamic images—interested readers can watch the video comparison demo released by Arm. “NSS achieves image quality levels comparable to desktop upscalers, while saving 50% of GPU workload for every frame.”
Regarding the specific implementation details of NSS, Arm did not elaborate at the media event—currently, it is unclear whether NSS will offer different quality and performance mode options like DLSS; and what optimizations NSS has for mobile scenarios compared to desktop-oriented super sampling solutions like DLSS. These details may need to wait until the new GPU IP is released next year for more technical specifics to be revealed.
It is worth mentioning that for the ecological construction of neural technology, in addition to the hardware aspect where Arm GPUs will soon include neural accelerators, Arm also released an ML extension for the Vulkan API, as well as a “Neural Graphics Development Kit” for developers—Steve stated this is “the industry’s first open SDK for the neural graphics field.”
Ecological Construction: Ready for Developers
Aside from NVIDIA, other major participants in the graphics accelerator market are pushing open-source solutions to build their own AI graphics ecosystems more quickly—after all, graphics rendering is now generally moving towards AI. “Arm firmly believes that an open ecosystem creates the greatest opportunities for innovation and differentiation.”
Given Arm GPU’s absolute market share advantage, building such an ecosystem in the mobile field is generally more advantageous and compelling compared to competitors and downstream chip design companies building ecosystems independently. In the absence of new hardware, pushing neural technology and NSS use cases ahead of time clearly indicates that they are already working on developing the ecosystem before the technology reaches consumer users.
In addition to the ecosystem and partnerships that Arm has accumulated in the gaming industry, let’s take a look at the aforementioned Neural Graphics Development Kit. The image above shows the composition of the “world’s first open neural graphics development kit”—these are the foundations for developers to practice NSS technology.
Steve stated that key components include a plugin for the Unreal Engine, “with just a few clicks, NSS features can be integrated into games.” Based on the open principle, the AI super sampling model of NSS is included in the development kit and is also publicly available in the Hugging Face community.
Additionally, since there is currently no hardware, Arm has prepared a complete PC simulation solution (including the Vulkan-based Arm ML extension), “allowing developers to run the entire stack on desktop platforms to validate model quality in their games.” Furthermore, there are accompanying sample codes, documentation, performance optimization tools, etc.
For NFRU and NSSD, Arm is also providing early access plans—regarding these two features, Steve mentioned that they currently lack the technical ecosystem support like Arm ASR, so Arm has chosen to collaborate with studios to refine the technology and prepare for future public releases (shape the technologies). Companies like NetEase Games, Tencent Games, Enduring Games, Sumo Digital, and Traverse Research have already partnered with Arm to jointly develop the Neural Graphics Development Kit.
Regarding the Arm ML extension for the Vulkan API, Steve briefly mentioned the “new concepts” introduced, “which allow developers to easily integrate AI into the graphics pipeline”—these should also be key software efforts by Arm in the implementation of neural technology. “First, we introduced VkTensor, which reinterprets existing images as tensors, the inputs and outputs of neural networks.” This approach can reduce data copying overhead when running neural networks.
“Secondly, the Graph pipeline in the middle of this image—similar to how the traditional graphics pipeline is responsible for the complete graphics computation process, the Graph pipeline conducts the entire neural network inference, which is the entry and exit of tensors.” Meanwhile, because a representation of the neural network is also needed, a “third concept” has been introduced, “we defined an open data format VGF—this format is based on SPIR-V extensions, expressing the operations of neural networks.”
“We are also actively discussing with many industry partners to promote these technologies in mobile ecosystems and more fields.” Clearly, they will be the unified standards for Arm in advancing AI technologies, including NSS. All components maintain an open attitude—thanks to this openness, the tool-level work can also be quickly ready, including options for developers to retrain or fine-tune the NSS model, adapting to their different workloads. Therefore, for developers with deep development capabilities, “we also retain complete control at the underlying level.”
Finally, we want to discuss the trend of graphics rendering and gaming technology generally moving towards super sampling, frame generation, and even future Neural Shading in the AI era. In the past two years, there has been considerable skepticism regarding AI-generated pixels: many believed this was the chip industry cutting corners to save on the area and cost of graphics rendering units.
However, Electronic Engineering Magazine has expressed multiple times that the slowdown of Moore’s Law has significantly reduced the returns of fixed-function units in GPUs for applications like gaming and professional visuals—this is the current state of industry development and the fundamental reason why IP suppliers and chip companies are introducing AI into GPU and graphics rendering processes.
It can be said that AI is the future of graphics technology development—this is evident from the technological achievements released at SIGGRAPH in recent years. Arm’s neural technology and the pioneering NSS are key for mobile gaming enthusiasts to enjoy higher quality visuals and smoother gaming experiences with relatively lower power consumption.
