Gongbang·New Ecology of Industrial Interconnection
Bangxun
01
Introduction
Edge Artificial Intelligence (Edge AI) refers to the development and deployment of artificial intelligence (AI) algorithms and programs on edge devices. It is a form of edge computing characterized by analyzing and processing data near the location where it is generated or collected. In contrast to Edge AI, cloud AI requires data to be transmitted over the internet to remote servers for processing.
Bangxun
02
Training and Deployment of Machine Learning Models
In machine learning (ML), data is input into the training process. For supervised learning, each sample is accompanied by its true label. The training algorithm automatically updates the parameters (also known as “weights”) in the machine learning model.
At each step of training, we evaluate the model based on the accuracy of predicting the correct labels for the given data. Over time, we hope the model’s accuracy will gradually improve, ultimately reaching an acceptable level. In most cases, the computational load of training machine learning models is substantial, so it does not need to be performed on edge devices. We can leverage powerful acceleration hardware, such as graphics processing units (GPUs), to complete model training in the cloud.
Once the model performance meets expectations, we can deploy it to the terminal devices. At this point, the model receives new data, performs data analysis, and produces corresponding output results. For supervised learning and classification tasks, the output is the label that the model believes best represents the input data. For regression tasks, the output is one or more numerical values. The process of making predictions based on new data after the model training is completed is called inference.
In traditional cloud-based machine learning model deployment, inference runs on remote servers. Clients connect to the inference service, attaching new data in the request, and the server processes it and returns the results. This cloud-based inference process is referred to as prediction service.
In most cases, the computational load of inference is much smaller than that of training. Therefore, we can run inference on edge devices instead of on powerful cloud servers.
Since the computational capabilities of edge devices are typically far less than those of cloud servers, machine learning models trained for them often need to reduce complexity. Nevertheless, Edge AI still has several advantages over cloud AI.
Bangxun
03
Advantages of Edge Artificial Intelligence
Assuming that machine learning models can run on edge devices such as laptops, smartphones, single-board computers, or embedded IoT devices, Edge AI has the following advantages compared to cloud-based solutions:
-
Reduced Bandwidth Usage
– There is no need to transmit raw data over the network; inference is completed directly on the edge device. As a result, only the inference results need to be transmitted, which is usually much smaller in data volume than the original input data.
-
Lower Latency
– Transmitting data over the network takes time (including the internet), as it must pass through multiple switches, routers, and servers. The round-trip latency while waiting for a response from the cloud server is typically hundreds of milliseconds. In contrast, Edge AI performs inference at or near the data collection point, resulting in virtually no network latency.
-
Better Energy Efficiency
– Most cloud servers require containerized operating systems and various abstraction layers, leading to significant system overhead. Running inference on edge devices often eliminates these intermediate layers and additional overhead, thereby improving energy efficiency.
-
Higher Reliability
– In environments with poor or no network connectivity, edge devices can continue to operate normally. This is particularly important for critical application scenarios such as remote areas or autonomous driving.
-
Better Data Privacy
– Although IoT devices must be particularly cautious when implementing security policies, raw data (such as users’ facial images) does not leave the device or local network, effectively reducing the risk of interception by malicious attackers.
Similar to edge computing, these advantages can be summarized with the acronym BLERP: Bandwidth, Latency, Energy usage, Reliability, and Privacy.
Bangxun
04
Limitations of Edge Artificial Intelligence
Edge AI has some limitations that need to be considered, weighing its pros and cons against cloud deployment.
-
Resource Constraints
– Typically, edge devices have fewer computational resources than cloud servers. Cloud servers can be equipped with powerful processors and large amounts of memory. If a machine learning model cannot be optimized or simplified to fit edge devices, then a cloud solution should be considered.
-
Limited Remote Access
– Cloud prediction services can be easily accessed from any connected device. However, remote access to edge devices often requires special network configurations, such as running a VPN service.
-
Scalability
– Scaling cloud model prediction services typically only requires cloning servers and paying the service provider for additional computing power. In contrast, scaling edge computing requires purchasing and configuring additional hardware.
Bangxun
05
Use Cases of Edge Artificial Intelligence
Edge AI has integrated into our daily lives and extended into industrial IoT applications, saving costs. One of the most typical use cases of Edge AI in home automation is smart speakers.
Smart speakers continuously listen for a specific keyword or phrase (such as “Alexa” or “Hey Google”). This process is called keyword spotting, which utilizes a small machine learning model to perform inference on the input audio data, specifically trained to recognize certain keywords or phrases. The latency requirements here are very high—the speaker must respond to user voice commands within milliseconds. This method also saves bandwidth, as the raw audio does not need to be continuously transmitted over the network.
Once the speaker recognizes the keyword, it “wakes up” and begins streaming audio to powerful cloud servers over the internet for more complex intent analysis to determine the user’s request. Smart speakers are a perfect combination of Edge AI and cloud AI, providing users with a unique interactive experience.
Many smartwatches also rely on Edge AI.
Some devices can perform keyword detection directly on the watch hardware or stream audio to a connected smartphone for analysis. In either case, the processing is done on the edge device. They can also work in conjunction with smartphones to analyze sleep patterns and track fitness activities.
Factories and industrial facilities are beginning to adopt Edge AI to help monitor equipment and workflow. For example, the Lexmark Optra is a single-board computer that can serve as an IoT Hub and perform critical analytical tasks, such as automatic optical inspection of production line components.
Finally, a typical application of Edge AI is in autonomous vehicles. These cars, trucks, and buses are expected to transport people and goods without human drivers.
Since vehicles cannot rely on a continuous and stable internet connection, the data processing from numerous sensors must be done locally within the vehicle. This means engineers must balance computational power, device size, and the complexity of machine learning models.
Bangxun
06
Market Size
The International Data Corporation (IDC) predicts that by 2025, 41.6 billion IoT devices will generate nearly 80 zettabytes of data. Additionally, Gartner predicts that by 2025, 55% of all data analysis performed by artificial intelligence and machine learning algorithms will occur on the devices that collect the raw data. This figure shows a significant increase in edge AI capabilities compared to just 10% of device-side processing in 2021. Gartner also forecasts that by 2027, revenues from dedicated AI processors (such as GPUs and neural processing units, NPUs) will reach $137 billion, with a compound annual growth rate (CAGR) of 26.5% over five years.
The rapid proliferation of AI technology and the large-scale deployment of IoT devices indicate that the market is expanding into areas that include Edge AI solutions. It is important to note that this does not replace cloud AI; cloud solutions will continue to grow in parallel with edge deployments.
(Source: Gongbang)