Author: Zach Shelby (Founder of Edge Impulse, Co-founder of Yunhe Capital)IoT Think Tank Original
This is my 364th column article.
Recently, the field of Tiny Machine Learning (TinyML) has made milestone progress, crossing an important watershed. The maturity and development potential of this technology will reach a new level.
The most representative event was Qualcomm’s announcement on March 11 to acquire Edge Impulse.
Edge Impulse is a representative company in the TinyML field, and this acquisition is expected to be completed within weeks. Prior to this, the company released a report titled “The Ultimate Guide to Edge AI: A Handbook for Transforming Edge Intelligence Business,” which is of significant reference value for understanding the development trends and commercial value of TinyML.
In several of my previous articles, such as “From LLM Large Models to SLM Small Models to TinyML, this field is expected to grow 31 times and give rise to new business models,” I have provided in-depth introductions and analyses of TinyML.
TinyML is a technology that optimizes machine learning models to run efficiently on resource-constrained devices (such as microcontrollers). Although these models are small and computationally efficient, they are capable of performing tasks such as speech recognition and sensor data analysis. Compared to chips running large language models (LLMs) and other AI technologies, TinyML devices are more cost-effective, consume less power, yet still deliver excellent performance.
The rise of TinyML will greatly promote the development of edge AI and edge intelligence.
Currently, people’s understanding of generative AI is undergoing subtle changes. In the past, it was generally believed that the more model parameters, the better the performance; now, more and more people are beginning to realize that small models can also exhibit tremendous potential.
As AI toolkits become increasingly powerful and programming methods become simpler (for example, using natural language), the use of AI technology is becoming more democratized. Additionally, the industry is striving to bring the inference process as close to the data source as possible, which is why edge AI is receiving more and more attention.
In light of this, this article will delve into the significance of Qualcomm’s acquisition of Edge Impulse, analyze noteworthy edge AIoT platforms, and distill the essence of the “Ultimate Guide to Edge AI” to provide readers with a comprehensive analysis of the technical trends and commercial prospects of TinyML.
Edge Impulse Joins Qualcomm, Simplifying Edge AI Development Processes
Qualcomm announced on March 11 the acquisition of Edge Impulse, an edge AI technology company.
This acquisition will integrate Edge Impulse’s edge AI development platform, enhancing Qualcomm’s capabilities in the AIoT field and promoting intelligent penetration in industrial automation, smart healthcare, and other areas.
Edge Impulse is a company founded only six years ago by Finnish entrepreneur Zach Shelby and Dutch engineer Jan Jongboom in 2019.
The two founders met while working on IoT systems at ARM, where they discovered a blue ocean market—TinyML micro machine learning. They realized that with the development of microcontroller computing power, it was now possible to run specific domain AI models directly on the board. If AI could be shifted from the cloud to the edge, countless application scenarios would benefit.
Since its inception, Edge Impulse’s mission has been to simplify the machine learning development process for edge devices, helping developers quickly build and deploy embedded AI applications, thereby promoting the proliferation of smart devices.
The AIoT platform built by Edge Impulse can significantly shorten the time required to create machine learning models for small devices such as sensors, microcontrollers, and cameras.
Edge Impulse provides developers with a complete set of tools that can automatically collect data, simplify model training, offer advanced optimization features, and support one-click deployment to various types of hardware, from MCU microcontrollers to CPU central processors, GPU graphics processors, and NPU neural network processors.
Currently, Edge Impulse’s platform supports over 170,000 developers, covering 250,000 machine learning projects, with clients including top global chip suppliers (such as STMicroelectronics and NXP) and end enterprises.
To date, Edge Impulse has raised $54.3 million in funding. In 2021, the company secured $34 million in Series B funding, reaching a valuation of $234 million. Two years later, the company reported revenues of $14.7 million.
Qualcomm’s acquisition of Edge Impulse can fill its technological gaps in edge and edge AI and accelerate its transition to AIoT.
Edge Impulse’s platform can complement Qualcomm’s chipsets and software architecture, helping developers quickly deploy AI applications on low-power devices.
It is reported that this acquisition is expected to be completed within weeks. After the acquisition is finalized, Qualcomm will be able to provide developers with more efficient AI development tools, accelerating the commercialization of IoT applications. In fields such as industrial, medical, and retail, the proliferation of edge AI will promote intelligent transformation, enhancing operational efficiency and decision-making capabilities.
The image above illustrates the advantages of the combination, where the Edge Impulse platform, when used in conjunction with Qualcomm’s edge AI workflow and tools like Qualcomm AI Hub and Foundries.io, can provide broader access to new models, testing environments, and deployment options.
In a blog post, Edge Impulse founder Shelby discussed the significance of joining Qualcomm.
-
For Edge Impulse users, this means that edge AI will become simpler and easier to use. The collaboration with Qualcomm will accelerate support for powerful Dragonwing processors in computer vision, audio, speech recognition, and generative AI. In addition to supporting Qualcomm’s hardware, Edge Impulse will continue to support edge hardware from a wide range of partners, including MCU, CPU, GPU, and NPU. Users can continue to deploy models on their preferred hardware using Edge Impulse as usual.
-
For Qualcomm, Edge Impulse will add an important component to its IoT strategic guidelines. Regardless of experience level, users can benefit from a complete and intuitive edge AI development workflow. From data preparation and AI experimentation to optimizing and deploying AI models on various edge hardware (including Dragonwing processors), users can start experimenting immediately. Additionally, enterprise and industrial applications can leverage the new integration of Edge Impulse with Qualcomm AI Hub to analyze and enhance model performance for Qualcomm hardware, accelerating innovation in high-performance IoT devices.
TinyML Devices: Small but Comprehensive Development Challenges
TinyML models run on microcontroller chips, which are commonly found in various devices such as washing machines, refrigerators, and automotive airbags.
The originally envisioned TinyML consumes very little power (less than 1 milliwatt), allowing devices to run on button batteries for a year or longer, and can almost always be powered by solar energy. Although this threshold has not yet been widely reached, TinyML devices can already run for weeks or even months on AA batteries.
The chips themselves are inexpensive and can be purchased from multiple manufacturers. Therefore, most TinyML devices range in price from a few dollars to several tens of dollars, depending on their capabilities.
These devices typically include not only chips but also cameras and sensors for detecting images and sounds for AI model analysis. Currently, the TinyML software, hardware, and datasets used by engineers are all open-source, meaning they can be freely accessed and modified.
Despite the apparent simplicity of TinyML devices, development is fraught with challenges.
First, it requires expertise in multiple skills, including hardware, software, and machine learning, which few people can master comprehensively. Secondly, combining all these skills also takes time. Therefore, a development platform that can simplify development and effectively deploy edge and edge AI has become a necessity.
Edge Impulse provides the “Ultimate Guide to Edge AI: A Handbook for Transforming Edge AI Business,” which contains key points on how edge AI is changing business and practical insights on how to leverage this groundbreaking technology.
The global growth and forecast of edge AI indicate that by 2033, the global edge AI market size is expected to grow from $19 billion in 2023 to approximately $163 billion.
The software and hardware market supporting edge AI is also continuously growing. Edge AI software solutions are becoming an indispensable part of modern digital infrastructure. This growing market encompasses various tools and platforms aimed at achieving AI inference and training for edge hardware, from IoT sensors to industrial devices. The global market size for edge AI software is expected to grow from $1.1 billion in 2023 to $4.1 billion by 2028.
In terms of hardware, according to Markets and Markets, the global edge AI hardware market is expected to reach a value of $54.7 billion by 2029.
Edge devices and processors form the pillars of edge AI implementation, supporting local processing and decision-making across various applications. These devices include dedicated AI accelerators and neural processing units (NPUs), as well as more general-purpose system-on-chip (SoC) optimized for edge computing.
The “Ultimate Guide to Edge AI” also provides insights on how to assess readiness for edge AI, how to develop a roadmap for edge AI, and how to calculate return on investment, making it a valuable reference.
The edge AI maturity assessment checklist can help organizations evaluate whether they are prepared for a successful edge AI journey. This checklist will guide the assessment of the key elements necessary for implementing and scaling edge AI initiatives.
The edge AI roadmap aims to guide the critical steps needed to plan, implement, and optimize edge AI strategies. By exploring the key components of an effective edge AI roadmap, organizations can better build for the future.
Calculating ROI is also a key step, as many organizations will face critical decisions regarding their edge AI strategy: build or buy. In the early days of edge AI, the DIY approach was often the only option, requiring companies to assemble complex components, tools, and resources. However, DIY is also a challenging approach that requires expertise across a wide range of fields to achieve.
Interested readers can click the link to download: https://pages.edgeimpulse.com/the-ultimate-guide-to-edge-ai
Tour of Edge AIoT Platforms: Tech Giants Compete
As edge AI continues to evolve, new platforms supporting AIoT are emerging.
Any platform aimed at supporting AIoT connectivity must effectively combine the functionalities of AI and IoT platforms. This is challenging because AI is characterized by rapid software iteration and updates, while IoT is characterized by resource-constrained, decentralized deployments that need to operate for years without human intervention.
In a new report titled “What Kind of Platform Does AIoT Need?” Transforma Insights identifies the essential features required for software platforms supporting AIoT.
While many features already exist, current IoT platforms are often not optimized for supporting distributed AI, and AI platforms typically do not consider all the constraints of IoT environments. Currently, AIoT platforms that can accommodate both IoT and AI are still few.
By acquiring Edge Impulse, Qualcomm will become a strong player in the edge AIoT platform space.
Other well-known edge AIoT development platforms include:
-
NVIDIA Jetson
Jetson is a powerful edge AIoT application platform that offers a range of hardware and software solutions tailored for AI and deep learning tasks. It provides high-performance computing capabilities in a compact form factor, making it a choice for applications in robotics, smart cities, and healthcare.
-
Google Coral
Coral aims to extend AI processing to edge devices. To this end, it offers modules such as the Coral Dev Board and USB Accelerator, as well as TPUs for AI inference. Google Coral works with TensorFlow Lite, making it easier for developers to deploy models.
-
AWS IoT Greengrass
AWS IoT Greengrass is a technology that extends AWS cloud services to edge devices, enabling them to take action locally on the data they generate. It supports machine learning inference, data processing, and secure communication between devices.
-
Microsoft Azure IoT Edge
Microsoft Azure IoT Edge is a fully managed service that brings cloud intelligence directly to IoT devices by deploying and running AI, Azure services, and custom logic locally. It employs a range of application development and is also compatible with Azure Machine Learning for model deployment.
-
Intel OpenVINO
Intel OpenVINO (Open Visual Inference and Neural Network Optimization) toolkit accelerates AI workloads on Intel hardware. It focuses on deep learning models and can be distributed to various edge devices. OpenVINO supports computer vision and natural language processing applications, making it a versatile tool for AI development in industries such as healthcare, retail, and industrial automation.
-
IBM Edge Application Manager
IBM Edge Application Manager is a complete solution for installing and running AI-based edge device applications. This tool is self-managing, allowing users to deploy when new versions or updates are available. IBM’s Edge Cluster Manager platform provides various use case scenarios, such as industrial automation, energy management, and remote productivity tools.
-
Sony Spresense
Spresense is a small development board designed for edge computing applications. It is built on a multi-core microcontroller and supports multiple AI frameworks, making it ideal for audio processing, computer vision tasks, and other AI uses.
Final Thoughts
With the rapid development of edge AI, TinyML is entering a new era.
Qualcomm’s acquisition of Edge Impulse not only highlights the industry’s giants’ high regard for TinyML and AIoT but also signals the intensifying competition in the edge AIoT platform space. In the future, as technologies like 5G and IoT become more widespread, edge AI will shine in more fields.
Finally, it is worth mentioning that as TinyML has become the norm for many embedded edge use cases, the TinyML Foundation has undergone a rebranding. The foundation has been renamed the Edge AI Foundation to better collaborate with ecosystem partners in delivering scalable, low-power AI at the edge.
References:
1. Our Next Chapter: Edge Impulse is Joining Forces with Qualcomm Technologies, Author: Zach Shelby, Source: Edge Impulse2. The Ultimate Guide to Edge AI, Source: Edge Impulse
