Author: C. J. Abate (USA)
Translator: Jun Qian
Machine Learning (ML), as a subset of Artificial Intelligence, has been widely applied in various fields including atmospheric science and computer vision. As Dr. Matthew Stewart from Harvard University states, tinyML is an emerging discipline that enables low-resource consumption and low-power machine learning algorithms on resource-constrained microcontrollers.
C.J. Abate: Let’s start with your background. When did you become interested in machine learning? Did you choose this field because of your background in programming or hardware design?
Matthew Stewart: My undergraduate major was Mechanical Engineering, which gave me some experience in programming and mechatronics. However, it wasn’t until I entered Harvard that I began studying machine learning. In the first year of my Ph.D. research, I took an introductory course in data science at Harvard, which sparked my interest in machine learning. At that time, I realized the immense potential of machine learning, both for general applications and specifically for atmospheric research.
Abate: What brought you to Harvard?
Stewart: Obviously, Harvard is one of the top research institutions in the world, and studying here is a goal for many passionate and hardworking students. I was also attracted by my professors’ research interests, particularly their use of drones to study the tropical Amazon rainforest. During my pursuit of a Mechanical Engineering degree, I developed an interest in environmental science, as it became increasingly clear to me that most engineering problems defined by modern times would be environmental issues, such as climate change, energy security, and sustainability. Based on my interests and engineering background, the work on Amazon rainforest drones seemed ideal and was the primary motivation for coming to Harvard.
Abate: As an environmental scientist, how do you keep yourself updated on embedded systems and programming? It must be difficult to keep up with all the new developments in AI and innovations in sensor technology, embedded systems, etc. How do you learn about these different disciplines?
Stewart: These fields are rapidly evolving, which is a very real issue for many graduate students and scholars. Personally, I use several resources to stay updated in the discipline. Firstly, Twitter can be a great platform for discovering new research published by other scholars in the field. I am also a fan of several Slack channels where colleagues regularly share news and research articles on relevant topics. I also regularly review new papers published in relevant journals to look for anything particularly noteworthy and worth a detailed read. Fortunately, most of the published work isn’t directly related to my own research, and broader trends often become the subject of seminar talks across departments and interest groups within universities.
Abate: Although I mentioned the details in an interview with Daniel Situnayake a few months ago, this is still a new topic for many engineers in the Elektor global community. How do you define tinyML? Is it the most basic way to run machine learning applications on edge microcontrollers?
Stewart: Yes, that is indeed our goal. tinyML is not a specific technology or a set of principles; rather, it is more like an important discipline involving the interplay of computer architecture, performance engineering, and machine learning. Its primary goal is to implement fast, low-resource, and efficient machine learning algorithms on resource-constrained microcontrollers. This may also involve developing custom hardware for specific tasks, creating new algorithms specifically designed for resource constraints, or optimizing performance across various hardware architectures. This article presents a useful guideline, applying tinyML as machine learning applications on microcontrollers with less than 1 MB of RAM and power consumption below 1 mW, but this is by no means a strict or exhaustive definition.
Abate: It is important to clarify that we are not discussing devices like NVIDIA and Raspberry Pi, but rather focusing on resource-constrained devices (i.e., less than 1 mW and kilobytes, not megabytes), correct?
Stewart: Yes. Devices like Raspberry Pi and NVIDIA are not the focus of tinyML, nor are applications related to autonomous driving, which typically require more computational resources. Our research focuses on “resource constraints.” For tinyML, we must make informed decisions on how to effectively optimize algorithm performance to fit the specific limitations of applications and hardware.
For example, in some applications, it is necessary to have both fast inference and high accuracy performance to improve inference speed. We can use 8-bit arithmetic instead of floating-point arithmetic, but this may affect the accuracy of the algorithm and the memory and computational resources required. This example helps to understand why I consider tinyML as a fundamental engineering discipline, as we are more focused on meeting demands, but often these requirements are in direct competition, necessitating a balance.
Abate: Can you provide some examples of practical applications?
Stewart: In fact, there are already some examples of tinyML being used in smartphones. One important example is keyword detection, which involves detecting words like “Hey Siri” and “Hey Google.” If a smartphone uses its CPU to continuously monitor the microphone for these words, the phone’s battery would only last a few hours. Instead, a lightweight digital signal processor can continuously detect these keywords. When a keyword is mentioned, it immediately wakes the CPU and verifies if it comes from a known microphone before waiting for other voice input.
Another example exists in smartphones for monitoring when users pick up their phones. Data from the onboard inertial measurement unit and gyroscope is continuously monitored, and when a user picks up their phone, a set of signals notifies the device to wake the CPU.
Another useful example is human detection, where a microcontroller connected to a camera can detect the presence of individuals. For instance, detecting whether a user is wearing a mask, which is particularly useful during the current pandemic. Anomaly detection could become an important use case in industry, where signals from heavy machinery can be continuously monitored for anomaly detection.
Abate: In 2019, you published an intriguing article titled “The Machine Learning Crisis in Scientific Research,” discussing whether machine learning could lead to a “reproducibility crisis” in science. For instance, if scientists use ML algorithms with little knowledge in their experiments, it could mean that other scientists are unable to reproduce these original research findings, and even non-experts can find issues there. I believe the debate over machine learning and statistics has intensified over the past year. How do you view this issue now?
Stewart: I believe this remains an important issue in academia. The article I published on this topic was in response to the reproducibility crisis, which was first raised by former Harvard Business School professor Amy Cuddy in relation to controversies over some of her work on power poses.
Andrew Gelman wrote an influential paper condemning poor research practices in psychology, including the use of p-hacking and other techniques for false data analysis and data selection to produce statistically significant results. This led to a series of experiments aimed at reproducing some key results in the psychology literature, many of which were not reproducible. This exposed a flaw in the research process, as reproducibility research is often underfunded because it is considered unnecessary and a waste of resources. Since then, the reproducibility research crisis has also been found to affect other fields, including literature and economics.
Naturally, this erosion of the integrity of the research process has led to concerns about the use of large datasets and machine learning. Given enough variables in a dataset, it is ultimately inevitable that some statistically significant results will appear. This suggests that false data will be easier to find, but testing is only valid when experiments are designed specifically to test that hypothesis rather than conducting multiple hypotheses simultaneously; thus, big data can make it easier to “cheat” with data. And what about machine learning? The use of machine learning makes it easier to “hide” cheating. Many machine learning algorithms have reduced interpretability, and many research communities lack a background in machine learning, making it difficult to detect these issues in published research. Fortunately, the solution to this problem is quite simple—fund reproducibility research and train researchers on proper experimental design and the application of machine learning in research.
Abate: You raise an interesting point in your article—”Another issue with machine learning algorithms is that they must make predictions and cannot say, ‘I found nothing.'” It sounds like machine learning isn’t always effective.
Stewart: While I agree that machine learning is not suitable for some tasks, I don’t believe it’s for this reason. For example, one of the issues posed by converting a task into a binary classification problem is that it may not be best summarized, leading to erroneous dichotomies; in some cases, it may be more appropriate to analyze data near the decision boundary rather than have the algorithm make a clear decision. This type of decision is sometimes referred to as “human-in-the-loop decision-making,” which will be most useful in situations where the decisions made have significant consequences, such as deciding whether to grant a loan or whether someone has cancer.
Abate: In which industries do you think there will be significant innovation opportunities in tinyML?
Stewart: Overall, I believe that many people working in this field are looking forward to tinyML sparking a new industrial revolution. For this reason, some have begun to refer to this new conceptual industrial phase as the “Industry 4.01 phase.” In this phase, any industry that uses a large number of IoT devices will benefit immensely from tinyML, including reduced power consumption and network load associated with tinyML.
More specifically, certain industries may benefit more from the new capabilities that tinyML offers. Agriculture is a great example. Using tinyML in agriculture can enable smart sensing capabilities without being connected to the grid, which can help determine when certain crops need to be harvested or require additional fertilizer or water.
Another great example is heavy industry, as mentioned earlier, where predictive maintenance using anomaly detection can save costs and improve efficiency. Detecting issues with large machinery in advance can be cheaper than catastrophic failures and result in less productivity loss.
Abate: What about companies interested in developing energy-efficient computing solutions?
Stewart: Apple and ARM are likely the largest companies currently focused on energy-efficient computing. The development of high-performance and efficient architectures is crucial in smartphones, not only to extend battery life but also to enhance functionality and speed. In recent years, we have seen mobile architectures significantly improve power efficiency in terms of performance, while traditional architectures from competitors like Intel have stagnated. Thus, mobile architectures can now not only compete with more traditional architectures but also have several unique advantages, including high efficiency in power systems. Recently, Apple announced its latest ARM-based M1 chip, claiming it will provide the longest battery life ever for Mac computers. This move by Apple has been viewed by some as a watershed moment in the computing industry, likely to create a ripple effect in the community over the next few years.
Abate: Please tell us about your work with drones and chemical monitoring systems. What role does tinyML play in your research?
Stewart: Currently, some work using tinyML for micro-drone applications has been published. The focus is on creating lightweight drones capable of intelligent navigation using embedded reinforcement learning methods. This could be very useful for detecting gas leaks or locating sources of pollution in both indoor and outdoor applications.
For broader chemical monitoring systems, tinyML can provide the ability to create remote sensor networks that are disconnected from the grid, as well as more intelligent use of chemical sensor information. For instance, the system can be designed to focus only on anomalous data rather than continuously transmitting data to a cloud server. This would reduce the network load on communication systems and the power consumption associated with continuous monitoring. As the number of devices grows exponentially, these aspects will become increasingly important in recent years.
Abate: Your article and research are likely to inspire many to engage more deeply in studying tinyML. Professional engineers and electronics enthusiasts may want to learn more about this topic. Besides books by people like Pete Warden and Daniel Situnayake, can you recommend some other resources?
Stewart: Unfortunately, one downside of cutting-edge technology is that there are often only a few resources available. That said, we are starting to see some of our peers regularly publish literature on tinyML, a significant portion of which is published on the preprint server arXiv. I suspect that soon there will be a few journals dedicated to this topic. Another resource is the tinyML research symposium released by the TinyML Foundation in March 2021 (https://www.tinyml.org/researchsymposium2021/), where we may see some exciting recent developments!
Related Links
[1] Machine Learning (Elektor): www.elektormagazine.com/tags/machine-learning.
[2] C. Abate, “The Future of Machine Learning: An Interview with Daniel Situnayake/” ElektorMagazine.com, 8/26/2020: www.elektormagazine.com/mlsitunayake.
[3] M. Stewart, “The Machine Learning Crisis in Scientific Research/” TowardsDataScience.com, 11/18/2019: http://bit.ly/ml-crisis-stewart.
[4] Cornell University, “Hardware Architecture,” arXiv.org: https://arxiv.org/list/cs.AR/recent.
This article is authorized by the Elektor Media Group, our partner publication. If you wish to subscribe for free to Elektor’s English online content, please visit https://www.elektormagazine.com/.
—— The End ——
Welcome to share, comment, like, and support! Thank you for your support!