(Written by Silicon Labs) Artificial Intelligence (AI) and Machine Learning (ML) technologies are not only rapidly evolving but are also being innovatively applied to low-power microcontrollers (MCUs) to achieve edge AI/ML solutions. These MCUs are an essential part of many embedded systems, capable of supporting AI/ML applications due to their cost-effectiveness, high energy efficiency, and reliable performance. The benefits derived from AI/ML capabilities are particularly significant in applications such as wearable electronics, smart home devices, and industrial automation. The rise of MCUs with AI optimization features and TinyML (focused on running ML models on small, low-power devices) reflects advancements in this field. TinyML is crucial for enabling intelligent decision-making directly on devices, facilitating real-time processing, and reducing latency, especially in environments with limited or no connectivity.
TinyML refers to the application of machine learning models on small, low-power devices, particularly on microcontroller (MCU) platforms that are optimized to operate within the limited resource constraints of the device. This enables edge devices to make intelligent decisions, support real-time processing, and reduce latency. Techniques such as quantization and pruning are used to reduce model size and improve inference speed. Quantization significantly reduces memory usage by lowering the precision of model weights with minimal impact on accuracy; pruning further reduces model size and enhances latency performance by removing less important neurons. These methods are essential for deploying ML models on resource-constrained devices.
PyTorch and TensorFlow Lite are the mainstream frameworks for implementing machine learning models. PyTorch is an open-source machine learning library widely used for developing AI applications, including those that can be deployed on microcontrollers. PyTorch provides tools and libraries for machine learning, including computer vision and natural language processing, suitable for low-power and small-sized devices.
TensorFlow Lite for Microcontrollers (TFLM) can run TF Lite models with Flatbuffer conversion capabilities on very constrained MCU-class devices. This reduces the model size and optimizes its inference on MCUs.
Another important tool is the CMSIS-NN library from ARM, which provides optimized neural network kernels for Cortex-M processors to run TFLM models. The CMSIS-NN library enhances performance and reduces memory footprint, making it easier to run ML models on ARM-based MCUs.
Additionally, some MCUs are equipped with dedicated AI/ML hardware accelerators, such as Silicon Labs’ EFM32 wireless SoC and MCU, which can significantly enhance the performance of ML models, allowing more complex applications to run faster and more efficiently on these devices. AI accelerators excel at parallelizing tasks such as matrix multiplication, convolution, and graphics processing. By leveraging diverse parallelism, they can perform a large number of computations simultaneously. This greatly speeds up AI workloads while maintaining low power consumption. These accelerators also enhance memory access patterns, reducing data transfer overhead, allowing the main CPU—CortexM—to enter low-power sleep modes to save more energy or manage additional tasks. By bringing data closer to the compute units, they reduce wait times. The result is improved performance, reduced power consumption, and latency.
Real-World Applications
The real-world applications of TinyML are diverse and impactful. A notable example is audio and visual wake words, where devices trigger actions when a specific keyword is spoken or when someone is detected in an image. This technology is used in smart speakers and security cameras, enabling them to activate upon recognizing wake words or detecting motion. Another application is predictive maintenance in industrial environments. Sensors on factory equipment continuously monitor parameters such as vibration and temperature, using TinyML models to detect anomalies and predict maintenance needs before failures occur, helping to reduce downtime and maintenance costs.
Gesture and activity recognition is another exciting application of TinyML. Wearable devices equipped with accelerometers and gyroscopes can monitor physical activities such as walking, running, or specific gestures. These devices use TinyML models to analyze sensor data in real-time, providing valuable insights for fitness tracking or medical diagnostics. In agriculture, TinyML is used for environmental monitoring. Smart agricultural systems analyze soil moisture and weather conditions to optimize irrigation, improving crop yield and resource efficiency.
TinyML also enhances health monitoring capabilities. Devices such as continuous glucose monitors (CGMs), which require long battery life and real-time data processing, can greatly benefit from this technology. Additionally, smart bed sensors can assess patients’ breathing patterns without direct contact, providing continuous health data for remote monitoring. This innovation is particularly valuable in managing elderly care and chronic diseases, as continuous monitoring helps in early detection of potential health issues.
Getting Started with Development
To start building your own TinyML applications, you need to understand the basics of TinyML and choose the right hardware. Depending on your application, you may need sensors to collect data, such as accelerometers, microphones, or cameras. Setting up the development environment includes installing the Simplicity Studio integrated development environment (IDE), SDK, and libraries required for TinyML.
The next step is to collect and prepare data relevant to your application. For example, if you are building a gesture recognition system, you need to collect accelerometer data for different gestures. After collecting the data, you need to preprocess it to make it suitable for training your model. Training the model requires using advanced frameworks like TensorFlow or PyTorch on a powerful machine. Once trained, the model needs to be optimized using techniques such as quantization and pruning.
After optimization, the model can be converted to a format suitable for MCUs, such as TensorFlow Lite format. The final step is to deploy the optimized model to the MCU, integrate it with the application code, and conduct thorough testing to ensure it meets performance and accuracy requirements. Continuous iteration and improvement based on actual performance are crucial for refining TinyML applications.
Implementing AI and Machine Learning on Microcontrollers with Silicon Labs Solutions
Silicon Labs offers a range of solutions to facilitate AI/ML implementation on MCUs. The EFR32/EFM32 (xG24, xG26, xG28) and SiWx917 series microcontrollers are well-suited for TinyML applications due to their low power consumption and robust performance. Below is a detailed technical guide for implementing AI/ML on Silicon Labs MCUs:
Data Collection and Preprocessing
Data Collection: Use sensors connected to the MCU to collect raw data, such as accelerometers, gyroscopes, and temperature sensors, which can be used for various applications.
Preprocessing: Clean and preprocess the data to make it suitable for training. This may include filtering noise, normalizing values, and segmenting data into windows. For this, Silicon Labs provides data collection and preprocessing tools.
The data collection tool is provided by partner SensiML: https://github.com/sensiml/sensiml_xG24_dual_audio_imu_capture
Model Training
Model Selection: Choose an appropriate ML model based on the application. Common models include decision trees and support vector machines.
Training: Use TensorFlow to train the model on high-performance cloud servers or local PCs with GPUs. This involves inputting the preprocessed data into the model and adjusting parameters to minimize error.
Model Conversion: Use the TensorFlow Lite converter to convert the trained model into a format compatible with TF Lite Micro. The FlatBuffer conversion in TensorFlow Lite for Microcontrollers (TFLM) involves converting TensorFlow Lite models into FlatBuffer format, a compact binary format that can be efficiently stored and accessed quickly. This process is crucial for running machine learning models on microcontrollers with limited memory and processing capabilities. FlatBuffers support direct access to models without unpacking. Once in FlatBuffer format, the model can be executed by the microcontroller, enabling it to perform inference tasks. This conversion reduces model size, making it suitable for devices with very limited memory, and allows for quick access and execution of the model without extensive parsing. Additionally, it ensures that the model can be seamlessly integrated and executed on MCUs running at speeds below 1GHz, with limited code space (typically less than 3MB) and SRAM (around 256KB).
Model Deployment
Integration with Simplicity SDK: Use Silicon Labs’ Simplicity SDK to integrate TF Lite Micro with the MCU.
Flashing the Model: Transfer the converted model to the MCU’s Flash memory. This can be done using Simplicity Studio, which provides a user-friendly interface for programming Silicon Labs MCUs.
Inference and Optimization: Apply optimization techniques such as quantization and pruning to reduce model size and improve performance.
Running Inference: Once the model is deployed, it can run inference on the MCU. This includes inputting new data into the model and obtaining prediction results.
The software toolchain: The new software toolkit is designed to support developers in quickly building and deploying AI and machine learning algorithms using some of the most popular tool suites (such as TinyML and TensorFlow). AI/ML software helps designers create new applications. In addition to native support for TensorFlow to provide optimized kernels for efficient inference, Silicon Labs has partnered with leading AI/ML tool providers (such as SensiML and Edge Impulse) to ensure developers have an end-to-end toolchain to simplify the development of machine learning models optimized for embedded deployment in wireless applications. By combining this new AI/ML toolchain with Silicon Labs’ Simplicity Studio development platform and the xG24, xG28, and xG26 series SoCs, developers can create applications that can gather information from various interconnected devices that can communicate with each other to make intelligent, machine learning-driven decisions.
Silicon Labs provides various tools and resources to support ML applications. Here are some examples:
Machine Learning Applications: Silicon Labs offers integrated hardware, software, and development tools to help customers quickly create secure smart devices suitable for industrial and commercial applications. The development platform supports embedded machine learning (TinyML) model inference, supported by the TensorFlow Lite for Microcontrollers (TFLM) framework. This repository contains a set of embedded applications that leverage ML: https://github.com/SiliconLabs/machine_learning_applications
Machine Learning Toolkit (MLTK): This is a Python package with command-line utilities and scripts that support the development of machine learning models based on Silicon Labs’ embedded platforms. It includes functionalities to perform ML operations from the command line interface or Python scripts and can determine the execution efficiency of ML models on embedded platforms, as well as train ML models using Google TensorFlow.
Reference Datasets: MLTK comes with reference datasets used for the models. These datasets can be found on GitHub: https://github.com/SiliconLabs/mltk/blob/master/docs/python_api/datasets/index.md
Audio Feature Generator: Silicon Labs provides an audio feature generator used with TensorFlow Lite models. It generates features based on the configuration in sl_ml_audio_feature_generation_config.h and initializes and starts the microphone in streaming mode. https://docs.silabs.com/machine-learning/latest/machine-learning-tensorflow-lite-api/ml-audio-feature-generation
MLPerfTiny Benchmark: The MLPerf Tiny Benchmark is a performance evaluation suite designed by an open engineering alliance, MLCommons. It aims to measure the performance and energy efficiency of ML systems in inference, applying trained ML models to new data. This benchmark is specifically tailored for low-power minimal devices, typically used in deep embedded applications such as the Internet of Things (IoT) or smart sensing.
Silicon Labs participated in the MLPerf Tiny benchmarking, submitting solutions that showcase the capabilities of the Machine Learning Toolkit (MLTK). This toolkit includes several models used for TinyML benchmarking, available on GitHub, covering applications such as anomaly detection, image classification, keyword recognition, and visual wake words.
Compared to previous versions, results using MLPerf Tiny v1.0 show improved inference speed, as well as reduced code size and memory usage. For example, Plumerai’s inference engine has shown significant enhancements, including support for time-series neural networks, such as LSTM-based recurrent neural networks (RNNs), which are common in motion sensors, health sensors, voice, and audio applications.
AI/ML Partnerships
Silicon Labs collaborates with leading suppliers, including Edge Impulse, SensiML, NeutonAI, and Eta Compute, to establish partnerships with AutoML toolchains and SaaS cloud partners. Additionally, solution providers such as Sensory and MicroAI, as well as design partners including Capgemini and Jabil, are part of this network. These alliances provide platforms that simplify the development of comprehensive solutions, making it easier for beginners to access edge AI/ML.
The advantages of TinyML on MCUs include:
• Low cost – MCUs are affordable
• Eco-friendly – low power consumption
• Easy integration – MCUs can be easily integrated into existing environments
• Privacy and security – data processed locally without the need for network transmission
• Rapid prototyping – quick development of proof-of-concept solutions
• Autonomous reliability – micro devices can operate stably in any environment
• Real-time processing – minimizing latency
Embedded Development Application Process
Developing applications with machine learning capabilities requires two distinct workflows:
• Using Simplicity Studio to create the embedded application development workflow for wireless applications.
• Creating a machine learning workflow that will be added to the embedded application.
Target Applications
Motion Detection: In commercial office buildings, many lights are controlled by motion detectors that monitor occupancy to determine whether lights should be turned on or off. However, when employees are typing at their desks, the motion sensors may fail to recognize their presence due to limited movement of hands and fingers, potentially leading to lights turning off automatically without providing illumination for employees. By connecting audio sensors with motion detectors, additional audio data (such as the sound of typing) can be processed through machine learning algorithms, allowing the lighting system to make smarter decisions about whether to turn the lights on or off.
Predictive Maintenance: Silicon Labs’ EFR32 MCU can be used to develop a predictive maintenance system. This requires using connected sensors to collect vibration and temperature data from machines while training a model to predict potential failures based on this data, then deploying that model on the MCU for real-time monitoring and maintenance planning of the machines.
Health Monitoring: Build a wearable health monitoring device using the EFM32 MCU. Use sensors to collect vital sign data such as heart rate and temperature. Train a model to detect anomalies in the data. Deploy that model on the MCU to help users gain real-time insights into their health status.
Smart Agriculture: Develop a smart irrigation system using Silicon Labs’ MCUs. Use connected sensors to collect soil moisture and weather data. Train a model to optimize water usage based on this data. Deploy that model on the MCU to control the irrigation system and improve crop yield.
Conclusion
MCUs are no longer limited to simple tasks; they are becoming powerful platforms for implementing AI. By exploring AI-optimized MCUs, we can open up new potential applications for battery-powered smart devices. Whether for smart home devices or industrial sensors, AI-driven MCUs are reshaping the future of embedded systems.