MCUs (Microcontrollers) and DSPs (Digital Signal Processors) are commonly used hardware components in edge AI devices. This article provides a brief analysis of the characteristics of these two types of devices, in conjunction with the book “AI at the Edge“.
A microcontroller is a small, inexpensive computer that is involved in the manufacturing of almost all devices, from automotive engines to smart appliances, making it a foundation of the modern world. The manufacturing volume of microcontrollers is astonishing, with annual shipments reaching tens of billions.
Microcontrollers are typically used for single-purpose applications, such as controlling a specific machine. This means they can be much simpler than computers that need to run multiple programs, and they often do not use an operating system within the MCU.
The software for microcontrollers is referred to as firmware, which runs directly on the hardware and contains the instructions needed to drive all peripheral devices. This can make the work of MCU software engineers quite challenging, but it also allows developers to have better control over the exact conditions under which the program runs.
One notable feature of microcontrollers is that most of their components are implemented on a single silicon chip, which is key to their relatively low cost. In addition to the processor, microcontrollers typically come equipped with FLASH memory for storing programs and other useful data, RAM for storing state during program execution, and various peripherals that use digital or analog signals to communicate with other devices, such as IIC, SPI peripherals that can communicate with sensors to obtain data, and ADC and DAC peripherals that can perform analog-to-digital and digital-to-analog conversions.
The field of microcontrollers is very diverse, and their value partly comes from the variety of variants available to suit a wide range of imaginable application scenarios. We categorize them into three main types: low-end, high-end, and digital signal processors.
Low-End MCU
Many MCUs are designed for low cost, small size, and high energy efficiency, and correspondingly, their computational resources and capabilities are limited. Here are some typical specifications:
-
4-bit to 16-bit architecture
-
<100 MHz clock speed
-
2 KB to 64 KB FLASH
-
64 bytes to 2 KB RAM
-
Digital input and output
-
Current consumption: running voltage of 1.5–5 V, current in the single to tens of milliamps range, and microamp level in sleep mode waiting for input
-
Cost: one to two dollars per unit when purchased in bulk
Power Supply Notes
The power consumption of microcontrollers depends on many factors, most of which are controlled by the developer. Power consumption can be reduced by lowering the processor’s operating speed, turning off functions when not in use, and putting the entire microcontroller into idle mode when no data is being processed.
This flexibility, combined with the general diversity of the microcontroller market, makes it difficult to provide exact power consumption figures. If designing for strict power limits, it is necessary to evaluate the hardware and measure energy consumption independently.
Many of the low-end MCUs currently in use are based on designs that have been in use since the 1980s. While technology continues to advance, there remains a need for simple, low-cost, and low-power hardware, ensuring that these chips will continue to exist. They are extremely common across many industries.
Low-end MCUs have some obvious disadvantages in edge AI applications. Due to their lack of memory and computational power, they are less suited for handling large amounts of data or complex signal processing. They typically lack any hardware implementation for floating-point operations, meaning calculations involving rational numbers can be very slow. These attributes limit the types of edge AI algorithms they can run.
Existing typical application scenarios leverage the characteristics of low-end MCUs, such as high reliability and low cost. They are widely used in automotive, medical devices, appliances, and infrastructure. A popular low-end MCU is the Atmel 8-bit AVR platform. While they are an important part of the MCU domain, their computational limitations mean that low-end MCUs may not be the preferred target for edge AI applications.
Edge AI programs are not always computationally challenging; low-end MCUs are fully capable of running complex conditional logic operations. They can form the network part of edge AI connected devices, for example, low-end MCUs can capture sensor data and forward it to more complex devices for decision-making.
High-End MCU
In the realm of MCUs, there are also high-end MCUs, with today’s most powerful microcontrollers having enough computational power to meet personal computer processing needs. In many cases, they still maintain high energy efficiency. Here are some typical specifications:
-
32-bit architecture
-
<1000 MHz clock speed
-
16 KB to 2 MB FLASH
-
2 KB to 1 MB RAM
-
Optional hardware support to accelerate mathematical operations
-
Floating Point Unit (FPU)
-
Single Instruction, Multiple Data (SIMD) instructions
-
Optional multiple processor cores
-
Digital and analog input and output
-
Current consumption: voltage of 1.5–5 volts can be as low as single digits to tens of milliamps; microamp level in sleep mode
-
Cost: from single digits to tens of dollars per unit
High-end MCUs have faster clock speeds and a 32-bit architecture, resulting in significant performance improvements. Additionally, many models of MCUs include clever hardware support to enhance computational speed. One such method is SIMD, which allows processors to run multiple calculations in parallel, which is very useful when running signal processing and machine learning applications that involve a lot of computation.
High-end MCUs are increasingly designed with edge AI applications in mind. Manufacturers often provide software and libraries to help optimize edge AI code for efficient operation on devices. Another major trend is the provision of large amounts of FLASH and RAM, which is very helpful for operating data and storing large machine learning models.
High-end MCUs are suitable for a variety of scenarios, from perception and IoT to digital devices, smart appliances, and wearables. They represent the best balance of cost, energy usage, and computational capability for embedded machine learning. They have enough computational power to run effective deep learning models, including those that can process visual information, yet they are simple enough to be embedded very cheaply into various applications.
Microcontrollers based on the Arm Cortex-M core are very common, such as the Nordic nRF52840 and STMicroelectronics’ STM32H743VI. There are also commonly used options based on the RISC-V architecture, such as the Espressif ESP32.
As edge AI becomes increasingly important, it is becoming more common for general-purpose high-end microcontrollers to be paired with dedicated coprocessors designed to accelerate deep learning workloads.
Performance Characteristics: Ordinary high-end microcontrollers can process audio with deep learning in near real-time and handle low-resolution video at about one frame per second.
Digital Signal Processors (DSP)
DSP is a special subclass of microcontroller designed to efficiently convert digital signals. Its architecture is not for general-purpose computing but is designed to run specific algorithms and mathematical operations as quickly as possible, including multiply-accumulate and Fourier transforms.
Fortunately, many of these mathematical operations are very helpful in edge AI, whether for processing data or running machine learning models, making DSPs valuable tools. DSP has the drawback of not being designed for general-purpose computing, meaning they may not be suitable for running applications outside of edge AI.
Today’s high-end MCUs often incorporate some features of DSPs, such as SIMD instructions that help improve throughput for signal processing tasks. In fact, to highlight these features, some MCUs are described as “digital signal controllers.” However, dedicated DSPs are still useful. For example, many smartphones with voice assistants, such as Google Assistant, have built-in DSP chips to run online keyword recognition models that can run continuously without affecting battery life.
Heterogeneous Computing
Hardware designers are not limited to designing a single microcontroller for a given application scenario. In fact, it is common to combine multiple microprocessors within a single product. For example, an edge AI device may include a small low-power MCU to run its basic operations, along with a larger, more powerful MCU for occasional signal processing and machine learning workloads.
This type of setup is known as heterogeneous computing, which is becoming increasingly important in edge AI as it supports true concurrency, with the ability to execute multiple tasks at once. A major challenge of heterogeneous computing is how to allocate computational workloads between the two processors to maximize efficiency. If this issue can be handled well, the device will see significant improvements.
Edge AI applications utilize architectures such as cascading models, which are particularly suited for heterogeneous computing. The rise of deep learning accelerators makes this an increasingly important concept.
In summary, this article introduces the application of microcontrollers in edge AI, analyzing the characteristics and advantages and disadvantages of low-end MCUs, high-end MCUs, and DSPs, and briefly introducing heterogeneous computing.