From heavy industry to healthcare, from agriculture to art, there is potential for deploying Edge Artificial Intelligence in every part of the modern world. Its potential application scenarios are virtually limitless. This article summarizes the potential application scenarios of Edge AI technology based on the book AI at the Edge. The roles that Edge AI plays in these applications can generally be categorized as follows:
-
Object Tracking
-
System Identification and Control
-
Human and Biometric Recognition
-
Signal Generation and Transformation
Object Tracking
From giant cargo ships to a grain of rice, human civilization relies on transportation and movement. Tracking and analyzing the state of objects is a key application area for Edge AI. Smart sensors can encode the physical world, transforming data into a form that computers can understand, enabling us to work more effectively.
The following image summarizes some Edge AI application scenarios for tracking objects.
Let’s analyze the example of “using smart packaging to monitor goods” in depth:
Finished products often need to be transported thousands of miles before reaching the client, and they are usually not single items. Damage during transportation can lead to losses for businesses, but when goods are found damaged at their destination after a long journey, it is not easy to determine what happened beforehand.
With Edge AI, logistics companies can connect devices to high-value goods that can identify when expensive items are at risk of damage. For example, if equipped with an accelerometer, the device can use machine learning models to distinguish between normal bumps and specific types of rough handling that could lead to damage. It can also log all incidents of rough handling, along with timestamps and GPS locations.
As long as the device can obtain a wireless connection, it can periodically upload logs. Upon arrival, if there is any damage, the company can analyze the logs to determine when and where the damage occurred, allowing them to identify and resolve the issue.
In the article discussing the advantages of Edge AI, we created the BLERP model to analyze the above application.
Bandwidth: To detect sudden bumps, accelerometer data must have a relatively high sampling frequency. This makes it difficult for data to be transmitted via low-bandwidth, low-power radios. By processing data on the device, bandwidth requirements can be significantly reduced.
Latency: This is not a primary consideration for this application scenario.
Economics: The cost of transmitting data wirelessly is high, especially since the device may be located anywhere in the world. Using Edge AI helps save on data transmission and reduce costs.
Reliability: Goods in transit are unlikely to have reliable connections, so it is crucial that the device can continue logging even when out of range. If we do not have to store raw data, we can keep a small number of less significant events in memory.
Privacy: This is not a primary consideration for this use case.
The main advantages of analyzing objects tracked by Edge AI are:
This type of application scenario is often related to the connectivity features and cost advantages of Edge AI. There are many things in the world that are not always in convenient communication locations. Low-cost Edge AI sensors, which do not require real-time internet connectivity, can achieve high-resolution monitoring of supply chain goods, while traditional methods would be prohibitively expensive.
Of course, the considerations and benefits of deploying Edge AI vary by project. For example, in the use case of “using cameras to monitor inventory on store shelves,” deploying Edge AI may be more inclined to protect privacy. If cameras connected to the internet are used to monitor store shelves, employees may feel they are under constant surveillance from headquarters. However, an offline inventory tracking system meets the needs of the store team perfectly.
System Identification and Control
The modern world is built on millions of complex, interconnected systems. From production lines to transportation networks, from climate control to smart appliances, the benefits of economic development are closely tied to these systems. Disruptions in production can lead to significant time and financial losses, while improvements in efficiency can save costs, labor, and reduce pollution emissions.
Monitoring, controlling, and maintaining complex systems presents a huge opportunity for Edge AI. Making quick, reliable decisions at the edge can enhance system responsiveness and resilience, while fine-grained insights into system status can help devices better plan subsequent decisions.
Some Edge AI use cases for system identification and control are illustrated in the following image.
This is a truly vast application category that includes many “future vision” related things: autonomous vehicles, industrial robots, and smart factories. Their commonality lies in using Edge AI to monitor the status of complex systems and provide feedback and control when needed.
As a broad category, automated monitoring and control of systems leverage most of the advantages of Edge AI. Economics and reliability are particularly important for many business use cases, and low-bandwidth, low-latency solutions are more desirable; otherwise, vendors may resort to server-side systems.
Let’s analyze the “predictive maintenance of oil drilling platforms” in depth:
If an industrial device suddenly fails, the resulting downtime and production interruption can lead to enormous losses. In some cases, it may also pose a threat to human health and the environment. Predictive maintenance can identify when a system begins to fail, allowing for action to be taken before a failure occurs.
Oil wells are extremely complex machines operating under extreme conditions. Since drilling platforms are located in the middle of the ocean, failures not only result in costly downtime but can also endanger the lives of the drilling platform crew, and oil spills can pollute the marine environment.
Using Edge AI, devices equipped with sensors can be deployed to monitor critical components of oil drilling platforms, measuring factors such as vibration, temperature, and noise. They can learn the “normal” state of each part of the system, building a model of normal operation. If conditions begin to deviate, they can alert the maintenance team for further inspection. Particularly complex predictive maintenance systems can even exert some control over the equipment, automatically shutting it down when dangerous conditions are detected.
Using the BLERP model to analyze the above application.
Bandwidth: Most oil drilling platforms rely on satellite connections, making it challenging to transmit large amounts of sensor data from thousands of drilling platforms to the cloud. Additionally, the network connectivity in exposed locations during drilling operations is very limited, as drill bits may be miles below the seabed. Predictive maintenance on the device can convert large streams of mixed data into lightweight event sequences that are easy to transmit.
Latency: Paying experts to conduct regular inspections limits the speed of problem identification. Continuous monitoring by Edge AI systems can immediately identify and resolve issues as they arise.
Economics: Predictive maintenance can save money lost due to downtime. Having professionals perform inspections on heavy machinery is costly, while monitoring with smart sensors equipped with AI is much cheaper.
Reliability: It is impossible to have reliable communication in extreme offshore environments. Using Edge AI, monitoring of equipment status can continue even if daily connections are interrupted.
Privacy: This is not a primary consideration for this use case.
Human and Biometric Recognition
The biological world is complex, chaotic, and rapidly changing. The ability to monitor and analyze them in real-time holds immense value. This category includes technologies that study humans, such as fitness tracking watches and smart toys, as well as systems for monitoring nature, agriculture, and the microscopic world.
These applications help bridge the gap between biology and technology, allowing our rigid computer systems to interact with the dynamic and flexible living world on Earth. As our understanding of biology improves, this field will continue to evolve.
Human-related Edge AI application scenarios are as follows.
Besides humans, the world is filled with plants, animals, and other organisms, as summarized in the following image for their specific application scenarios.
This field involves application scenarios for humans and organisms, utilizing various aspects of the BLERP model. Applications in this area may have particularly good privacy requirements. Many applications using server-side AI are technically feasible, but will only be socially accepted when completed on the device.
The most common example is digital personal assistants, such as Apple’s Siri or Google’s Google Assistant. Personal assistants continuously listen for wake words by using device-side models. Only when a wake word is detected does the audio stream get transmitted to the cloud. Without components on the device, the assistant would have to continuously stream audio to the service provider. This contradicts most people’s privacy requirements.
By moving functionality to the device and avoiding data transmission, we unlock a multitude of possibilities, especially in visual aspects, such as self-driving cars making real-time decisions through Edge AI.
Let’s analyze the example of “using tracking cameras to discover rare wildlife” in depth:
Tracking cameras or camera traps are a special type of camera designed specifically for monitoring wildlife. They have a robust weatherproof casing, high-capacity batteries, and motion sensors. Equipped with a trail view, they automatically take photos whenever motion is detected.
Researchers install camera traps in remote areas to monitor specific species, placing them for months at a time. When they return, they download photos from the cameras to better understand the target species. For example, they may try to estimate how many individuals are present.
Camera traps face several significant issues that require substantial time and funding:
1-Most of the photos taken do not feature the target species. Instead, captures are triggered by non-target species or random movements in the field of view.
2-Due to the high number of false positives, the reliability of sending capture notifications via network connection is poor. Researchers must physically travel to remote areas to collect saved photos, which is very labor-intensive. If the storage card is full, it may lead to data loss. If nothing valuable is captured, unnecessary trips may also occur.
3-Researchers must sift through thousands of useless photos to find a few valuable ones.
Using Edge AI, camera traps can identify target species through specially trained deep learning visual models, ignoring non-valuable images. Researchers no longer need to worry about useless images filling up the storage card. The cameras can also be equipped with low-power or radio transmitters, allowing them to report animal sightings in the wild without human access. This can significantly reduce research costs and increase effective scientific workload.
Using the BLERP model to analyze the above application.
Bandwidth: Camera traps are often deployed in remote areas where network connectivity is not common, and expensive low-bandwidth satellite is the only option. Using Edge AI can reduce the number of photos taken, allowing all photos to be transmitted.
Latency: Without Edge AI, researchers need to send personnel to collect photos from camera traps, a process that can take months. With Edge AI and low-power radio connections, photos can be analyzed immediately without waiting, providing useful information.
Economics: Avoiding field trips can save a significant amount of money, as can avoiding unnecessary use of expensive satellite broadcasts.
Reliability: Discarding useless photos can extend the lifespan of the storage card.
Privacy: Edge AI cameras can discard photos of humans on trails, protecting the privacy of other passersby.
Signal Generation and Transformation
For computers, the real world consists of signals: time series of sensor readings, each data point describing a small part of the environment. Previous application categories primarily focused on interpreting these signals and responding accordingly. Receiving sensor data and constructing a simple output that is convenient for human interpretation or can serve as control signals for automated systems.
The last category is slightly different. Sometimes, we do not want to convert raw signals into instantaneous decisions, but rather just want to convert one signal into another. Digital signal processing is an important component of embedded applications. In these use cases, signal processing is the end goal rather than a byproduct, and achieving signal processing through Edge AI is more effective than traditional DSP pipeline processing methods.
Some use cases regarding signal transformation are summarized below.
Since digital signals are expressed over time, applications in this field often benefit from the latency advantages of Edge AI. Bandwidth is also particularly important, as access to the original signal is required. Transmitting transformed signals typically requires the same amount of bandwidth, if not more.
Let’s analyze the use case of “blurring backgrounds during remote work meetings” in depth:
With the rise of remote work and video conferencing, employees have had to get used to showing their previously private home spaces to colleagues. To help maintain a certain level of privacy, many video conferencing tools now support blurring the background of video streams while keeping the video subject unchanged.
These tools rely on a technique called “segmentation,” which uses deep learning models to identify pixels in the video stream that belong to one category or another. In this case, the model is trained to distinguish between a person and their background. The input is the raw video stream from the camera. The output is a video stream with the same resolution, but the background pixels are blurred together, making it difficult to see the background content.
To protect privacy, using Edge AI for this technology is crucial. Otherwise, unblurred videos would be transmitted outside the user’s home, rather than performing segmentation and blurring on the device before data transmission.
Using the BLERP model to analyze the above application.
Bandwidth: If the transformation occurs on a high-resolution raw video stream, its transformation effect is optimal. However, video transmission typically involves compressed low-resolution versions that may contain visual artifacts. Therefore, the blurring transformation must be completed on the device before compression transmission.
Latency: Performing the transformation on a remote server may introduce additional latency compared to directly sending the video stream to the other party. Performing the operation on the device eliminates this extra step.
Economics: Performing the required computations on the device sending the video is cheaper than performing the computations in the cloud, as the service provider must pay for this in the cloud.
Reliability: Using a cloud server as the conversion endpoint complicates the video stream transmission channel, increasing the probability of interruptions. By processing on the device, the transmission path is simpler, reducing the likelihood of failure.
Privacy: When data is converted on the local device, it ensures that other users will never see the original video.
Another interesting application of transforming data is virtual sensors. In some cases, engineering or cost constraints may prevent devices from being equipped with all the required sensors. For example, a product design may require a particularly precise sensor to enhance performance, but that sensor may be too expensive for the budget.
To address this issue, a virtual sensor can be created, which is an artificial data stream that provides signals almost as good as those from real objects. To achieve this, Edge AI algorithms can reconstruct the signal of the required sensor by processing other signals; for instance, it can combine data from several inexpensive sensors to simulate a high-precision sensor.
For example, in monocular depth estimation, a model can be trained to estimate distance using data from a simple imaging sensor. If a high-precision sensor is used, it typically requires more expensive solutions, such as stereo cameras or laser-based distance sensors.
In summary, this article analyzes the practical application scenarios of Edge AI and deeply examines the important role of Edge AI in different types of applications using the BLERP model.