A recent research conclusion from NVIDIA has garnered significant attention in the industry— Small Language Models (SLM) are the future of intelligent agents. Following this, NVIDIA introduced its new small language model:Nemotron-Nano-9B-V2, which achieved the highest performance among its peers in certain benchmark tests.
In fact, the trend of small language models (SLM) has also reached the fields ofMCUs and MPUs.
Small models are “compressed” large models.
Small language models (SLM) may already be familiar to us.SLM parameters range from millions to billions, while LLM can have hundreds of billions or even trillions of parameters.
SLM is derived from compressing LLM, and the compression model needs to retain as much accuracy as possible while reducing the model size. Common methods include:
-
Knowledge Distillation: Training a smaller “student” model using knowledge transferred from a larger “teacher” model;
-
Pruning: Removing redundant or less important parameters in the neural network architecture;
-
Quantization: Reducing the numerical precision used in computations (e.g., converting floating-point numbers to integers).
Small language models are more compact and efficient than large models. Therefore,SLM requires less memory and computational power, making it ideal for resource-constrained edge or embedded devices.
Many small yet powerful language models have emerged, proving that size is not everything. CommonSLMs range from1 billion to40 billion parameters, includingLlama3.2-1B (a 1 billion parameter variant developed by Meta),Qwen2.5-1.5B (Alibaba’s 1.5 billion parameter model),DeepSeek-R1-1.5B (DeepSeek’s 1.5 billion parameter model),SmolLM2-1.7B (HuggingFaceTB’s 1.7 billion parameter model),Phi-3.5-Mini-3.8B (Microsoft’s 3.8 billion parameter model),Gemma3-4B (Google DeepMind’s 4 billion parameter model).
RunningSLM is not solely dependent on computational power.
ForMPUs, runningSLM does not seem to be a challenge. However, for developers, how can they determine whether anMCU supports running generativeAI?
This question does not have a single, straightforward answer— however, there is a hard requirement that theMCU’s neural processing unit (NPU) must be able to accelerateTransformer operations.
In addition, running generativeAI requires theMCU’s bandwidth system bus and a large-capacity, high-speed, tightly coupled memory configuration.
In fact, many people now only useGOPS (billion operations per second) orTOPS (trillion operations per second) to compare the raw throughput of microcontrollers. Currently, the best-performingMCU can provide up to250GOPS of computing power, while MCUs used for generativeAI will need to provide at least double that performance. However, raw throughput is not an ideal metric for measuring actual system performance.
Because successful generativeAI applications require support forTransformer computations, which will transfer large amounts of data between the system’s internal memory, neural processing unit, central processing unit, and peripheral functions such as image signal processors. Therefore, a system with high raw throughput may theoretically process large amounts of data quickly, but if the system cannot transfer data quickly to the neural processing unit, the actual performance will be very slow and disappointing.
Of course, forMPUs, high bandwidth and tight coupling between memory and buses are also crucial.
Aizip has partnered withRenesas on anSLM project.
As early as last August,Aizip collaborated withRenesas to showcase ultra-efficientSLM and compactAI Agents for edge system applications onMPUs. These small and efficient models have been integrated into the Renesas RZ/G2L and RZ/G3S boards based onArm Cortex-A55.
Aizip has created a series of ultra-efficient small language models (SLM) and artificial intelligence agents (AI Agents), namedGizmo, ranging from300 million to2 billion parameters. These models support various platforms, including MPUs and application processors for a wide range of applications.
SLM enables AI agents on edge applications to provide the same functionality as large language models (LLM), but in a compact form factor. The models on devices offer enhanced privacy protection, resilience, and cost savings. While some companies have successfully reduced the size of mobile language models, ensuring accurate tool invocation for automation applications on low-cost edge devices remains a significant challenge for theseSLMs.
It is reported that theseSLMs can achieve response times of less than3 seconds on a singleA55 core of the RZ/G2L running at a frequency of1.2 GHz.
MCUs are also increasing their investment inSLM.
Alif Semiconductor recently released its latest series ofMCUs and fusion processors—Ensemble E4,E6, andE8, primarily targeting the operation of generativeAI models, includingSLM.
At the same time,Alif is the first chip supplier to use theArm Ethos-U85 NPU (neural processing unit), which supportsTransformer-based machine learning networks.
Benchmark results show that this series can perform high-energy efficiency object detection in less than2 milliseconds, image classification in under8 milliseconds, and executingSLM on E4 devices consumes only36mW of power when generating text to build stories based on user prompts.
Ensemble E4 (MCU) features dualArm Cortex-M55 cores, while Ensemble E6 andE8 fusion processors are based onArm Cortex-A32 cores and dualCortex-M55 cores, notablyE4/E6/E8 are equipped with dualEthos-U55+Ethos-U85, providing powerful computing capabilities.
Alif believes that compared to other manufacturers, they have an earlier layout, as the first generation ofEnsemble MCU series was released as early as2021, and since then they have been shippingE1,E3,E5 and E7 devices in volume. While otherMCU manufacturers are still at the first generation ofAI MCUs,Alif has released second-generation products, and is also the industry’s first MCU to supportTransformer-based networks, forming the foundation forLLM and other generativeAI models.
SLM will be the future of embedded systems.
SLM significantly compresses model size while retaining as much accuracy as possible. This efficient and compact characteristic makes it a perfect fit for resource-constrained edge and embedded devices, bringing unprecedented intelligent capabilities to these devices.
In fact, the future landscape of edgeAI is gradually unfolding, andSLM will also be one of the key areas for MCU and MPU manufacturers to focus on.
For example, STMicroelectronics’STM32N6, Infineon’sPSoC Edge latest generationMCUs, TI’sAM62A andTMS320F28P55x, NXP’si.MX RT700 andi.MX 95, and ADI’sMAX7800X are all beginning to emphasizeNPU.
EmbeddedAI was initially a feature of relatively expensive microprocessor-based products running onLinux systems. However, the market quickly realized that there is also space forAI in edge and endpoint devices—many of which are based onMCUs. Therefore, by the second half of2025, advancedMCU manufacturers will incorporate products withAI capabilities into their product lines. These manufacturers’ NPU are divided intoArm Ethos IP and self-developed types. Currently, the latestEthos-U85 has begun to supportTransformer, and demonstrated the effects ofSLM six months ago, with other manufacturers continuously following suit. In the future,SLM is expected to completely change the landscape ofMCUs andMPUs.
References
[1]IBM：https://www.ibm.com/cn-zh/think/topics/small-language-models
[2]Hugging-Face：https://hugging-face.cn/blog/jjokah/small-language-model
[3]Alif：https://alifsemi.com/comparing-mcus-for-generative-ai-its-not-just-about-the-gops/
[4]Alif：https://alifsemi.com/who-wins-in-the-race-to-make-ai-mcus/
[5]Arm：https://newsroom.arm.com/blog/small-language-model-generative-ai-edge
· END ·
Follow the subscription account of EEWorld: "Robot Development Circle"
Reply "DS" to receive the complete version of "DeepSeek: From Beginner to Expert"
Scan to add the assistant and reply “Robot”
to join the group and exchange experiences face-to-face with electronic engineers.
