Author: C. J. Abate (USA)
Translator: Jun Qian
Machine Learning (ML), as a subset of Artificial Intelligence, has been widely applied in various fields, including atmospheric science and computer vision. As Harvard PhD Matthew Stewart states, tinyML is an emerging discipline that enables low-resource, low-power machine learning algorithms to run on resource-constrained microcontrollers.
C.J. Abate: Let’s start with your background. When did you become interested in machine learning? Did you choose this field because of your background in programming or hardware design?
Matthew Stewart: My undergraduate major was mechanical engineering, which gave me some experience in programming and mechatronics. However, it wasn’t until I entered Harvard that I began studying machine learning. In the first year of my PhD research, I took an introductory data science course at Harvard, which sparked my interest in machine learning, and I realized the vast potential of machine learning, both for general applications and specifically for atmospheric research.
Abate: What brought you to Harvard?
Stewart: Obviously, Harvard is one of the top research institutions in the world, and studying here is a goal for many passionate and hardworking students. The research interests of my professors also attracted me, as they studied the tropical Amazon rainforest using drones. During my mechanical engineering degree, I became interested in environmental science because it became increasingly clear to me that most engineering problems defined in modern times would be environmental issues, such as climate change, energy security, and sustainability. Given my interests and engineering background, this work on Amazon rainforest drones seemed ideal and was the main motivating factor for coming to Harvard.
Abate: As an environmental scientist, how do you keep yourself informed about embedded systems and programming? It must be challenging to keep up with all the new developments in AI, sensor technologies, and innovations in embedded systems. How do you stay updated across these different disciplines?
Stewart: These fields are constantly evolving rapidly, which is a very real issue for many graduate students and scholars. Personally, I utilize several resources to stay updated. First, Twitter can be a great platform for discovering new research published by other scholars in the field. I am also a fan of several Slack channels where colleagues regularly share news and research articles on relevant topics. I also regularly review new papers published in relevant journals to look for anything particularly noteworthy and worth detailed reading. Fortunately, most published works are not directly related to my own research, and broader trends often become the subject of seminar talks conducted by various departments and interest groups within universities.
Abate: Although I mentioned details in an interview with Daniel Situnayake a few months ago, this is still a new topic for many engineers in the Elektor global community. How do you define tinyML? Is it the most basic way to run machine learning applications on edge microcontrollers?
Stewart: Yes, that is actually our goal. tinyML is not a specific technology or a set of principles; rather, it is more of an important discipline involving the synergy between computer architecture, performance engineering, and machine learning. Its primary goal is to implement fast, low-resource, and efficient machine learning algorithms on resource-constrained microcontrollers. This may also involve developing custom hardware for specific tasks, creating new algorithms or tools designed specifically for resource constraints, or optimizing the performance of various hardware architectures. This article provides a useful guideline for applying tinyML as a machine learning application on microcontrollers with less than 1 MB of RAM and power consumption below 1 mW, but this is by no means a strict or exhaustive definition.
Abate: It is important to clarify that we are not discussing devices like NVIDIA and Raspberry Pi, but rather focusing on resource-constrained devices (i.e., less than 1 mW and kilobytes rather than megabytes), correct?
Stewart: Yes. Devices like Raspberry Pi and NVIDIA are not the focus of tinyML, nor are applications related to autonomous driving, which typically require more computational resources. Our research focuses on ‘resource constraints.’ For tinyML, we must make informed decisions about how to effectively optimize algorithm performance to fit the specific limitations of applications and hardware.
For example, in some applications, both fast inference and high accuracy performance must be present to improve inference speed. We can use 8-bit arithmetic instead of floating-point arithmetic, but this will affect the accuracy of the algorithm and will also impact the memory and computational resources required by the algorithm. This example helps to understand why I view tinyML as a raw engineering discipline because we consider more about meeting demands, but often these requirements compete directly, necessitating a balance.
Abate: Can you provide some examples of practical applications?
Stewart: In fact, there are already examples of tinyML being used in smartphones. One significant example is keyword spotting, which involves detecting words like ‘Hey Siri’ and ‘Hey Google.’ If a smartphone continuously monitors the microphone using its CPU, the phone’s battery could only last a few hours. Instead, a lightweight digital signal processor can continuously detect these keywords. When a keyword is mentioned, it immediately wakes up the CPU and verifies if it is from a known microphone before waiting for additional voice input.
Another example exists in smartphones for monitoring when users pick up their phones. Data from onboard inertial measurement units and gyroscopes are continuously monitored, and when a user picks up their phone, a set of signals notifies the device, waking up the CPU.
Another useful example is human detection, where a microcontroller connected to a camera can detect the presence of individuals. For instance, detecting whether users are wearing masks, which is particularly useful during the current pandemic. Anomaly detection could become an important use case in industry, where signals from heavy machinery can be continuously monitored for anomaly detection.
Abate: In 2019, you published an engaging article titled ‘The Machine Learning Crisis in Scientific Research,’ exploring whether machine learning could lead to a ‘reproducibility crisis’ in science. For example, if scientists use ML algorithms they know little about, it may mean other scientists cannot reproduce these original research results, and even non-experts can find issues there. I think the debate over machine learning and statistics has intensified over the past year. How do you view this issue now?
Stewart: I believe this is still an important issue in academia. The article I published on this subject addressed the reproducibility crisis, which was first raised by former Harvard Business School professor Amy Cuddy in controversies surrounding some of her work on power posing.
Andrew Gelman wrote an influential paper condemning poor research practices in psychology, including the use of p-hacking and other techniques for false data analysis and cherry-picking data to produce statistically significant results. This led to a series of experiments aimed at reproducing some important results in psychology literature, many of which were not reproducible. This exposed a flaw in the research process, as reproducibility studies often lack funding because they are seen as unnecessary and wasteful of resources. Since then, the reproducibility research crisis has also been found to affect other fields, including literature and economics.
Naturally, this erosion of the integrity of the research process has led to concerns about the use of large datasets and machine learning. Given enough variables in a dataset, statistically significant results will inevitably emerge. This suggests that false data will be easier to find, but testing is only valid if the experiments are designed specifically to test that hypothesis rather than testing multiple hypotheses simultaneously, so big data can more easily ‘deceive’ with data. And what about machine learning? The use of machine learning makes it easier to ‘hide’ cheating. Many machine learning algorithms have reduced interpretability, and many research communities lack a background in machine learning, making it difficult to identify these issues in published research. Fortunately, the solution to this problem is quite simple—fund reproducibility research and train researchers on proper experimental design and the application of machine learning in research.
Abate: You raised an interesting point in your article—’Another issue with machine learning algorithms is that they must make predictions, and cannot say I found nothing.’ It sounds like machine learning is not always effective.
Stewart: While I agree that machine learning is not suitable for completing some tasks, I don’t believe it is for that reason. For example, one of the issues posed by converting tasks into binary classification problems is that they may not be best summarized, leading to incorrect dichotomies; in some cases, it may be more appropriate to analyze data near the decision boundary rather than letting the algorithm make a clear decision. This type of decision-making is sometimes referred to as ‘human-in-the-loop decision-making,’ and it will be most useful in situations where the decisions made have significant consequences, such as deciding whether to grant a loan or whether someone has cancer.
Abate: In which industries do you think there will be significant innovation opportunities in tinyML?
Stewart: Overall, I think many people working in this field are looking forward to tinyML sparking a new industrial revolution. For this reason, some have begun to refer to this new conceptual industrial phase as ‘Industry 4.01.’ In this phase, any industry that uses a large number of IoT devices will benefit tremendously from using tinyML, including reduced power consumption and network load associated with tinyML.
More specifically, certain industries may gain greater benefits from the new capabilities offered by tinyML. Agriculture is a great example. Using tinyML in agriculture can achieve intelligent sensing capabilities without connecting to the grid, which can help determine when certain crops need to be harvested or require additional fertilizer or water.
Another great example is heavy industry, where, as mentioned earlier, predictive maintenance using anomaly detection can save costs and improve efficiency. Detecting transportation issues in large machinery beforehand can be cheaper than catastrophic failures, resulting in less productivity loss.
Abate: What about companies interested in developing energy-efficient computing solutions?
Stewart: Apple and ARM are currently the largest companies focused on energy-efficient computing. Developing high-performance and efficient architectures is crucial in smartphones, not only to extend battery life but also to enhance functionality and speed. In recent years, we have seen significant improvements in power efficiency in mobile architectures in terms of performance, while traditional architectures from competitors like Intel have stagnated. Thus, mobile architectures can now not only compete with more traditional architectures but also offer several unique advantages, including high efficiency in power systems. Recently, Apple publicly announced its latest ARM-based M1 chip, claiming it will provide the longest battery life ever for Mac computers. This move by Apple has been seen by some as a watershed moment in the computing industry, which will have ripple effects in the community for years to come.
Abate: Please tell us about your work with drones and chemical monitoring systems. What role does tinyML play in your research?
Stewart: Currently, some work using tinyML for micro-drone applications has been published. The focus here is to create lightweight drones capable of achieving intelligent navigation through embedded reinforcement learning methods. This could be very useful for detecting gas leaks or locating sources of pollution in indoor and outdoor applications.
For broader chemical monitoring systems, tinyML can provide the ability to create remote sensing networks that are disconnected from the grid and use chemical sensor information more intelligently. For example, the system can be designed to focus only on anomalous data rather than continuously transmitting data to a cloud server. This will reduce network load on the communication system and the power consumption associated with continuous monitoring. With the exponential growth of devices, these aspects will become increasingly important in recent years.
Abate: Your articles and research are likely to inspire many to engage more deeply in studying tinyML. Professional engineers and electronics enthusiasts may want to learn more about this topic. Besides books like those by Pete Warden and Daniel Situnayake, could you recommend some other resources?
Stewart: Unfortunately, one downside of cutting-edge technology is that there are often only a few available resources. That said, we are starting to see some peers regularly publish literature on tinyML, a significant portion of which is published on the preprint server arXiv. I suspect we will soon see a few journals dedicated to this topic. Another resource is the tinyML research symposium released by the TinyML Foundation in March 2021 (https://www.tinyml.org/researchsymposium2021/), where we might see some exciting recent developments!
Related Links
[1] Machine Learning (Elektor): www.elektormagazine.com/tags/machine-learning.
[2] C. Abate, “The Future of Machine Learning: An Interview with Daniel Situnayake/”, ElektorMagazine.com, 8/26/2020: www.elektormagazine.com/mlsitunayake.
[3] M. Stewart, “The Machine Learning Crisis in Scientific Research/”, TowardsDataScience.com, 11/18/2019: http://bit.ly/ml-crisis-stewart.
[4] Cornell University, “Hardware Architecture”, arXiv.org: https://arxiv.org/list/cs.AR/recent.
This article is authorized by Elektor Media Group, a partner of this publication. If you wish to subscribe to Elektor’s English online content for free, please visit https://www.elektormagazine.com/.
1. Which domestic embedded operating system do you have the most confidence in?
2. Yang Fuyu’s column | In-depth interpretation of the EDR data from the Wenzhou Tesla out-of-control case.
3. 【Video Launch】 The Learning Path of Embedded Real-Time Operating Systems.
4. Learning STM32 microcontrollers, the unavoidable serial port.
5. The current situation and future of the RISC-V industry~
6. When the MCU crashes, first grab the hardware~
