Machine Learning (ML) is a very good tool for solving problems involving pattern recognition. ML algorithms can transform chaotic raw data into usable signals. The basic process is to generate a model based on data, and then use the model to predict outputs, achieving learning, reasoning, and decision-making without human interaction. However, the demand for high-performance computing resources has restricted many ML applications to the cloud. In other words, only performance at the level of cloud data centers can meet the computational requirements of ML. Excitingly, with the continuous advancement of algorithm design and microprocessor architecture, running complex ML workloads on the smallest microcontrollers (MCUs) is becoming possible.
Running ML models on embedded devices is often referred to as Embedded Machine Learning. There are many benefits to machine learning on embedded devices:
1. It eliminates the need to transfer and store data on cloud servers, thus reducing the risk of data and privacy leakage involved in data transmission.
2. It strengthens the protection of intellectual property, personal data, and trade secrets.
3. Executing ML models can effectively avoid the need to transfer data to cloud servers, saving precious bandwidth and network resources.
4. Using ML-based embedded devices is sustainable, as their carbon footprint is much lower. This is because the microcontrollers used in these devices are low-power.
5. Embedded systems are more efficient than cloud-based systems, allowing ML models on edge devices to achieve real-time responses.
TinyML: New Business Opportunities for MCU Manufacturers
The initial success of deep learning models was primarily due to large servers with abundant memory and GPU clusters. While cloud deep learning has been very successful, it is not suitable for all scenarios, as many applications require inference on the device itself. Most AI applications today are based on machine learning technology, and if machine learning models can run smoothly on resource-constrained devices, it will open a technological door for many emerging applications. This is also an important reason why edge computing and embedded machine learning have become increasingly popular in recent years.
Embedded machine learning is a field of machine learning where models known as Tiny Machine Learning (TinyML) are very suitable for edge devices with limited memory and processing capabilities, where internet connectivity is non-existent or constrained. Today, TinyML has become a rapidly developing field in machine learning, enabling analysis of sensor data at mW and below power consumption, realizing the embedding of AI on small hardware.
Although TinyML is a new concept, applying machine learning to smart devices is not new. For instance, most smartphones have some form of neural network, with music recognition and many camera modes (such as night vision and portrait mode) relying on embedded deep learning examples. These are all applications of TinyML and have advanced Edge AI.
Edge AI chipsets bring AI into countless endpoints, including mobile devices, cars, smart speakers, and wireless cameras. However, these devices often struggle to fully utilize the generated data due to their inability to support high computational performance and high data throughput. The emergence of TinyML makes it possible to run machine learning models on MCUs. These MCUs are generally low-cost, compact, with hundreds of KB of low-power memory (SRAM) and several megabytes of storage space, low power consumption, and wide applicability. The main goal of TinyML chipsets is to solve cost and energy efficiency issues, achieving data analysis performance on low-power, low-processing capability, and small memory hardware through software designed for small inference workloads. This technology has the potential to fundamentally change the future of the Internet of Things (IoT).
Today, there are over 25 billion active IoT devices globally, with an expected annual growth of 20%. These devices collect vast amounts of data daily, and processing this data in the cloud presents significant challenges. Now, TinyML is expected to bridge the gap between edge hardware and device intelligence. McKinsey researchers predict that by 2025, the IoT industry will generate a potential economic impact of $4-11 trillion, with manufacturing being the largest vertical industry, reaching $1.2-3.7 trillion.
Market consultancy ABI Research predicts in its new white paper, TinyML: The Next Major Opportunity in Technology, that the number of IoT connections will nearly triple from 2021 to 2026, reaching 23.6 billion. Each new connection represents an opportunity to leverage AI and machine learning, and TinyML technology will be key to seizing this opportunity for businesses. Therefore, ABI expects the shipment of TinyML devices to increase from 15.2 million in 2020 to 2.5 billion by 2030.
Major Manufacturers Gather in the TinyML Arena
Since the birth of TinyML, the innovation market has been bustling, with many products attracting attention. For example, the industrial AI smart camera based on NVIDIA Jetson Xavier NX, launched by Adlink, is the industry’s first industrial smart camera. This camera, based on NVIDIA’s Jetson Xavier NX, is high-performance, compact, and approximately ten times more efficient than its predecessor, making it a compact, reliable, and powerful Edge AI application product, opening the door for AI innovation in manufacturing, logistics, healthcare, agriculture, and many other business fields.
TinyML focuses on optimizing machine learning workloads so they can run on low-power microcontrollers. The surge of TinyML will lead to the expansion of Edge AI beyond traditional critical markets, allowing more end-users to benefit from smart connected sensors and IoT devices based on sound, temperature, pressure, vibration, and other data sources. Today, TinyML is at the intersection of machine learning and embedded IoT, potentially bringing disruptive change to many industries. The potential applications of TinyML are nearly limitless, such as predicting when industrial robots need servicing, sensors that monitor crops for harmful insects, in-store shelves that request reordering when stock runs low, and medical monitors that track vital signs while maintaining privacy.
Audio analysis, pattern recognition, and voice human-machine interfaces are currently the most prevalent fields of TinyML applications. NXP’s 3D face recognition solution based on EdgeReady MCU utilizes the i.MX RT117F crossover MCU, enabling developers to quickly integrate 3D face recognition and advanced liveness detection into their products, even under outdoor lighting conditions. The 3D liveness detection feature can identify and prevent fraud using photos or 3D models, requiring only a high-performance 3D structured light camera module (SLM) and an optional low-cost CMOS sensor-based RGB camera, without the need for expensive, power-hungry, Linux-based MPUs.
The i.MX RT1170 used in the solution is a crossover MCU, featuring a Cortex-M7 core with a frequency of up to 1GHz and an Arm Cortex-M4 core with a frequency of up to 400MHz, offering outstanding computing power, various media functions, and real-time capabilities. Face recognition and liveness detection can be performed entirely offline on the i.MX RT117F MCU, eliminating reliance on the cloud, thus not only eliminating latency issues but also effectively protecting consumer privacy.
Figure 1: Hardware structure diagram of i.MX RT117F 3D face recognition
(Image source: NXP)
Vision, motion, and gesture recognition are also important application areas for TinyML. ST’s AI solutions are primarily based on the STM32 product line, allowing embedded developers to port, optimize, and validate using pre-trained neural networks on any STM32 based on Cortex M4, M33, and M7. STM32CubeMX is a graphical tool that makes it easy to configure STM32 microcontrollers and microprocessors through a step-by-step process and generate corresponding initialization C code for Arm Cortex-M cores or specific Linux device trees for Arm Cortex-A cores.
STM32Cube.AI is an AI extension package for STM32CubeMX, allowing designers to develop their AI products more efficiently. FP-AI-VISION1 is a function package (FP) of STM32Cube, containing examples of computer vision applications based on convolutional neural networks (CNN).
Currently, FP-AI-VISION1 includes three examples of image classification applications based on CNN:
-
Food recognition application running on color (RGB 24-bit) frame images;
-
Presence detection application running on color (RGB 24-bit) frame images;
-
Presence detection application running on grayscale (8-bit) frame images.
Now, the TinyML computer vision solutions provided by ST can recognize 18 common foods, implement presence detection, or count the number of people in a scene based on object detection models, etc.
Figure 2: Execution process of food recognition model
(Image source: ST)
With the expansion of the IoT market, the amount of data at the edge is growing rapidly, giving rise to AIoT powered by TinyML. According to analysis data from Markets and Markets, the AIoT market size was approximately $5.1 billion in 2019 and is expected to grow to $16.2 billion by 2024, with a compound annual growth rate (CAGR) of 26%. The main role of AIoT is to empower connected devices with machine learning capabilities, enabling them to perform complex intelligent computations.
In June 2021, Infineon launched ModusToolbox ML, aimed at enabling the company’s PSoC MCUs with deep learning capabilities. ModusToolbox ML is a new feature based on ModusToolbox software, providing developers with the middleware, software libraries, and dedicated tools needed for deep learning-based ML models. ML can seamlessly integrate with existing software frameworks in ModusToolbox, making it easy to integrate into secure AIoT systems. ModusToolbox ML allows developers to use their preferred deep learning frameworks (such as TensorFlow) to directly deploy to PSoC MCUs. Additionally, ML helps engineers optimize models for embedded platforms, reduce platform complexity, and provide performance verification based on test data.
Figure 3: Internal architecture of Infineon PSoC6 MCU
(Image source: Infineon)
To help developers quickly add local intelligence to their IoT designs, Infineon chose to collaborate with SensiML. SensiML, a subsidiary of QuickLogic, provides cutting-edge software to enable ultra-low-power IoT endpoints to achieve AI. The company’s flagship solution, SensiML Analytics Toolkit, offers an end-to-end development platform covering data acquisition, labeling, algorithm, and firmware auto-generation and testing. SensiML’s “Analytics Toolkit” Edge AI development software can now be used in conjunction with Infineon ModusToolbox, providing developers with a fast and easy way to record data from Infineon’s XENSIV sensors, create complex AI/ML-based models, and run customized applications on PSoC6 MCUs.
The Growing TinyML Ecosystem
The TinyML community was established in 2019, consisting of researchers and industry engineers dedicated to bringing ML capabilities to microcontroller devices. TinyML comprises machine learning architectures, technologies, tools, and methods capable of performing analysis on various sensing modalities (vision, audio, motion, chemical, and others) on low-power target devices primarily powered by batteries. One of the founders of TinyML, Evgeni Gousev, believes, “We are in a digital transformation revolution, and TinyML executes machine intelligence and analysis on devices at low cost, providing significant energy savings along with inherent privacy features.”
TinyML will proliferate across many industries, impacting almost every sector, including retail, healthcare, transportation, health, agriculture, fitness, and manufacturing. Meanwhile, industry participants quickly recognized the value of TinyML and took swift action to create a supportive ecosystem.
Arm is a strong supporter of TinyML and a leader in TinyML technology. With over 180 billion Arm-based chips shipped, its IP, tools, and over 1,100 software partners have built billions of micro-intelligent IoT devices.
Today, the Arm®Cortex®-M series MCUs have become the most widely used platform for TinyML applications, capable of executing real-time computations quickly and efficiently, at low cost, with high reliability, fast response times, and low power consumption. The Cortex-M55 processor is Arm’s most AI-capable Cortex-M processor, providing enhanced, energy-efficient DSP and ML performance. The Ethos-U55 NPU, a new ML processor called microNPU, is specifically designed to accelerate ML inference in space-constrained embedded and IoT devices. Ethos-U55 combined with the AI-supporting Cortex-M55 processor improves ML performance by 480 times compared to existing Cortex-M-based systems.
In fact, at the beginning of 2021, Raspberry Pi released the first microcontroller board, one of the most affordable development boards on the market, priced at only $4. This board, called Raspberry Pi Pico, is based on the RP2040 MCU, featuring a powerful dual-core Cortex-M0+ processor capable of running TensorFlow Lite Micro, and we will soon see various TinyML use cases for this board.
For decision-makers overwhelmed by vast amounts of data, TinyML is like a savior, fully utilizing edge data and enabling faster access to the right information. Furthermore, TinyML has improved the widespread privacy concerns by processing data on-device and only transmitting key information.
Next, we will see a new world filled with trillions of intelligent devices, supported by TinyML technology, capable of sensing, analyzing, and acting autonomously, creating a healthier and more sustainable environment for us.
This published article is an exclusive original article. Please indicate the source when reprinting. We reserve the right to pursue legal responsibility for unauthorized copying and non-compliant reprints.
Mouser Electronics is a global authorized distributor of semiconductors and electronic components, serving a vast community of electronic design engineers worldwide. Mouser Electronics distributes nearly 1,200 well-known brands and offers millions of products online, providing customers with a one-stop procurement platform. We welcome you to follow us for the latest design and industry information!
