This article is about6057words, and it is recommended to read in12 minutes。This article comprehensively introduces edge computing and edge AI, as well as the challenges they face. This September, AI chip unicorn Horizon released its self-proclaimed strongest edge AI chip, Horizon “Rising Sun 3”, which has caused a sensation. Compared to the second-generation chip, the AI performance of “Rising Sun 3” has greatly improved, achieving an equivalent standard computing power of 5TOPS with a power consumption of only 2.5W. Especially for the high-end market, the Rising Sun 3M has stronger encoding and decoding capabilities, making it highly competitive.
In recent years, Horizon has focused primarily on the Rising Sun and Journey series, with its product line mainly concentrated in the field of edge AI chips, making Horizon the highest-valued AI chip unicorn in the world. It is well known that AI chips are divided into cloud and edge. Cloud chips require high performance, while edge chips have more requirements regarding environment and energy consumption due to the numerous application scenarios. In fact, edge AI chips are no longer a niche market. In addition to Horizon, companies like Google, NVIDIA, Intel, Qualcomm, Huawei, and Cambricon have all launched edge AI chips in the past two years, indicating that the battle for AI chips has spread from the cloud to the edge. According to data from global technology market consultancy ABI Research, it is expected that by 2025, the revenue of the edge AI chip market will reach $12.2 billion, while the cloud AI chip market will reach $11.9 billion. The edge AI chip market will surpass the cloud AI chip market. Chief analyst Lian Jye Su stated that as the industry develops, companies increasingly need to address issues related to data privacy, power efficiency, low latency, and robust on-device computing performance, and edge AI will be the solution. It is expected that in the next five years, AI training and inference will occur at gateways or various edge devices, even down to sensor nodes. So, what exactly is edge AI, and how can it solve these industry pain points? 1. Edge Computing and Edge AI In recent years, the number of IoT devices has shown a linear growth trend. According to Gartner’s prediction, by 2020, the number of global IoT devices will exceed 20 billion. At the same time, the devices themselves are becoming increasingly intelligent. The integration of artificial intelligence and the Internet of Things in practical applications will drive human society into the era of “intelligent interconnection of all things”, and the data generated will explode. Over the past decade, cloud computing has successfully alleviated issues related to storage and management brought by the growing data, but currently, the growth rate of network bandwidth lags far behind the growth rate of data, and the decline in network bandwidth costs is much slower than that of hardware resources like CPUs and memory. Additionally, complex network environments make it difficult to achieve breakthroughs in network latency. Therefore, traditional cloud computing cannot meet the high demands for response time and security. For example, in the case of autonomous vehicles, cars traveling at high speeds need to respond within milliseconds; any delay due to data transmission or network issues could lead to serious consequences. Moreover, cloud computing also faces bandwidth limitations. If all the data generated by edge devices were to be transmitted to cloud computing centers, it would put tremendous pressure on network bandwidth. For instance, a Boeing 787 generates over 5GB of data every second, but the bandwidth between the aircraft and satellites is insufficient to support real-time data transmission. In summary, relying solely on cloud computing, which is a centralized computing processing method, is inadequate to support applications running and processing massive amounts of data based on IoT perception. In this application context, edge computing has emerged. It can effectively solve big data processing issues between cloud centers and network edges when combined with existing centralized processing models of cloud computing. Edge computing is a supplement and optimization of cloud computing. If cloud computing is centralized big data processing that occurs “in the cloud”, then edge computing can be understood as big data processing that occurs at the edge, close to the terminal (such as mobile phones, smart voice interaction devices, etc.). In many cases, edge computing and cloud computing coexist. A vivid analogy is to compare cloud computing and edge computing to the organs of an octopus, which may be easier to understand. As the most intelligent invertebrates in nature, octopuses possess “conceptual thinking” abilities, which are inseparable from their two powerful memory systems. One is the brain memory system, which has 500 million neurons, and the other is the suckers on their eight tentacles. In other words, the eight legs of the octopus can think and solve problems. Cloud computing is like the octopus’s brain, while edge computing is similar to the octopus’s small tentacles, with each tentacle representing a small data center close to the specific physical objects. Edge computing is closer to the device side and the user. Specifically, edge computing has several obvious advantages: Bandwidth: Edge devices process some of the generated temporary data, eliminating the need to upload all data to the cloud, only valuable data needs to be transmitted, which greatly reduces the pressure on network bandwidth and decreases the demand for computing storage resources; Latency: Processing data close to the data source significantly reduces system latency and improves service response times; Economics: For certain applications, even if cloud-based solutions can technically solve bandwidth and latency issues, executing computations at the edge may be more cost-effective; Reliability: For certain applications, even if cloud-based solutions can technically solve bandwidth and latency issues, the network connection to the cloud is not always reliable, and applications may need to operate continuously. In such cases, edge computing is necessary. For example, a facial recognition door lock must continue to function normally even if the network connection is interrupted; Privacy: For certain applications, even if cloud-based solutions can technically solve bandwidth, latency, reliability, and economics issues, many applications still need to perform local processing due to privacy considerations. Edge computing provides the infrastructure for storing and using critical privacy data, enhancing data security. Due to its outstanding advantages, edge computing meets the demands of the future IoT, rapidly gaining attention globally since 2016. Of course, edge computing is a continuously evolving concept, and the integration of different technologies leads to ongoing innovation in the core of edge computing. For example, the application of artificial intelligence and neural networks is empowering the realization of “edge AI”. Thanks to the extensive data computing capabilities of cloud servers, significant progress has been made in the fields of artificial intelligence and machine learning, developing more comprehensive artificial neural networks to tackle challenging tasks. As network architectures and tools for machine learning and neural network training continue to adapt and integrate into embedded systems, more and more AI applications can run directly on edge devices, making “edge AI” a hot topic of discussion today. Edge AI refers to AI algorithms that are processed locally on hardware devices and can handle data without a network connection. This means that operations such as data creation can occur without streaming or storing data in the cloud. To achieve these goals, edge computing can generate data in the cloud using deep learning and perform model inference and prediction at the data origin—i.e., the device itself (the edge). Currently, there is no established standard architecture or unified algorithm for edge artificial intelligence internationally, but major manufacturers have begun exploring relevant fields. Traditional cloud service providers like Google, Amazon, and Microsoft have launched edge artificial intelligence service platforms, enabling machine learning inference by running pre-trained models locally on terminal devices, pushing intelligent services to the edge. Additionally, various edge artificial intelligence chips have emerged in the market, such as the Horizon Rising Sun 3 mentioned at the beginning of this article, Google’s Edge TPU, Intel’s Nervana NNP, Huawei’s Ascend 910, and Ascend 310. 2. Edge Computing and 5G Edge computing and 5G can be said to be mutually complementary. The International Telecommunication Union 3GPP defines three major scenarios for 5G. Among them, eMBB refers to high-bandwidth mobile broadband services such as 3D/ultra-high-definition video, mMTC refers to large-scale IoT services, and URLLC refers to services requiring low latency and high reliability, such as autonomous driving and industrial automation. The 5G communication network is more decentralized, requiring the deployment of small-scale or portable data centers at the network edge for local processing of terminal requests to meet the ultra-low latency requirements of URLLC and mMTC. Therefore, edge computing is one of the core technologies of 5G. Edge computing can precisely solve the problems faced by the three major scenarios of 5G. First, edge computing devices will provide connectivity and protection for new and existing edge devices; second, although 5G will offer better connectivity and lower latency for cloud-based applications, there are still costs associated with processing and storing data. Hybrid edge computing/5G solutions will reduce these costs; finally, edge computing allows more applications to run at the edge, shortening delays caused by data transmission speed and bandwidth limitations and enabling preliminary analysis of local data, relieving some of the workload from the cloud. On the other hand, 5G and edge computing complement each other to some extent. On one hand, the development of 5G itself will directly promote the development of edge computing; on the other hand, since 5G promotes the Internet of Things, it will also indirectly promote the development of edge computing. 3. Use Cases for Edge AI Due to its low traffic consumption, low latency, and strong privacy features, edge AI has wide application prospects across various industries. Smartphones This may be the edge AI device we are most familiar with. Siri and Google Assistant are the best examples of edge AI on smartphones, as the technology powers their voice UIs. AI on phones allows data processing to occur on the device (the edge), meaning there is no need to send device data to the cloud. This helps protect privacy and reduce traffic.
Smart Home With the proliferation of the Internet of Things, home life will introduce more and more smart applications, such as smart lighting control, smart TVs, smart air conditioning, etc. These applications require the deployment of numerous sensors and controllers in homes. To protect home data privacy, data processing must rely on edge computing, limiting most computing resources within the home gateway and preventing sensitive data from leaking. By optimizing indoor positioning and home intrusion detection through edge AI, higher accuracy and lower latency can be achieved compared to cloud computing. For example, smart speakers like Amazon Echo and Google Home are widely used; they receive user commands and respond by interacting with third parties (services or home appliances). However, smart speakers rely on the cloud for voice recognition and language semantics understanding and processing, which can lead to home data privacy breaches, while applying edge AI can effectively solve these issues. Home entertainment will also benefit from edge AI, as there is no need to upload user preferences to the cloud; the system can autonomously recommend personalized services, providing users with a better entertainment experience. Drones Drones can fly to remote and dangerous areas for exploration and capture aerial images in unique ways. The applications of drones are expanding, including applications in agriculture and mining. However, these devices must be able to “phone home” to respond to the data they collect. Edge computing enables drones to check data and respond in real-time. For example, Baidu Smart Cloud collaborated with MaiFei Technology to deploy the BIE-AI-Board (which has built-in detection and operation models) on drones, using hyperspectral and visible light cameras to collect crop information data, which is uploaded to the BIE-AI-Board. The BIE-AI-Board loads detection models to assess the location and severity of crop pests and diseases and, based on the detection results, calls operation models to control agricultural protection machines to execute agricultural protection tasks. This solution has already been implemented in several farmlands in China. Smart Agriculture Agriculture will also benefit from edge AI. In addition to the application of drones, there are also applications for precision agriculture using agriculture IoT based on edge computing. For example, an agricultural technology company based in Australia, The Yield, uses sensors, data, and artificial intelligence to help farmers make informed decisions regarding weather, soil, and plant conditions.Public Safety Facial recognition systems are the direction of development for surveillance cameras, allowing them to learn to recognize individual human faces. In November 2019, WDS Ltd. released the AI camera module Eeye, which analyzes facial features in real-time through edge AI. The use of edge computing enhances the computational capability of camera terminals, allowing facial recognition functions to operate independently of cloud servers, avoiding the time-consuming process of uploading images and saving a significant amount of bandwidth resources. Moreover, by completing facial recognition directly on local devices, the recognition process can be shortened to within 1.5 seconds. Real-time video analysis is also one of the most important application scenarios for edge AI. Previously, video analysis was generally conducted in the cloud, facing high traffic consumption and significant latency issues. With the development of edge computing, some video analysis tasks can be shifted to edge nodes. At the endpoint, video capture devices are responsible for video capture, compression, and image/video preprocessing. At the edge layer, multiple distributed edge nodes collaborate with each other. In the cloud, because the distributed model training of edge nodes may not have been well-trained due to limited local knowledge, when the edge cannot provide services, the cloud can use global knowledge for further processing and help update training models for edge nodes. A report by Tractica estimates that by 2025, the shipment of AI edge devices will increase from 161.4 million units in 2018 to 2.6 billion units globally each year. In terms of unit numbers, the top AI edge devices will include smartphones, smart speakers, personal computers/tablets, head-mounted displays, automotive sensors, drones, consumer and enterprise robots, and security cameras. Wearable health sensors and building or facility sensors will also see more AI applications. Internet of Vehicles The Internet of Vehicles enhances safety and efficiency by interconnecting vehicles, reducing traffic accidents and avoiding congestion. Initially, cellular networks, edge computing, and AI served the Internet of Vehicles as independent technologies. Edge computing can provide high-speed data transmission and low latency services for the Internet of Vehicles, making autonomous driving possible. AI is widely applied in various fields of smart transportation. The combination of the two will further optimize the performance of the Internet of Vehicles, achieving accurate vehicle positioning, target tracking, intelligent perception, and decision-making through edge AI. Healthcare Healthcare has entered the data age. The widespread awareness of health has led to an increasing variety of medical devices and terminals, and edge artificial intelligence will have multiple application scenarios in smart healthcare. First, in pre-hospital emergency care, during the transfer of emergency patients to hospitals or between hospitals, current emergency medical services are mostly deployed in the cloud, which can be affected and limited by mobile environments and extreme weather. Edge AI can establish a bidirectional real-time communication channel between ambulances and hospitals, enabling real-time natural language and image processing, thereby improving timeliness and efficiency. Secondly, regarding smart wearable devices, current smart wearable devices are limited by computing power and primarily serve data collection purposes. Research is ongoing into deploying edge AI on smart wearables, such as the work by researcher Jaime Andres Lincoln at the University of Valencia, who is deploying edge AI to monitor human emotions. In the future, more lightweight intelligent algorithms will be executed on terminals for real-time monitoring, analysis, and prediction of health data, helping humans better understand their physical conditions. Smart Cities As urban scales continue to expand, data exhibits geographical distribution characteristics, requiring edge artificial intelligence models to provide monitoring and intelligent control for latency-sensitive devices. Supporting large-scale infrastructure computing and services for smart cities through edge computing can enable low-latency applications for terminal devices, deploying latency-sensitive tasks to execute at edge nodes. Using AI to coordinate and schedule urban infrastructure can be applied to public safety, healthcare, urban management, transportation, smart communities, and other fields, achieving optimal resource utilization across the entire city. For example, regarding traffic congestion, Singapore has addressed significant traffic challenges through connected transportation solutions. Its Intelligent Transportation System (ITS) has pioneered an electronic road pricing system, which increases road charges with rising traffic volumes. ITS also allows for real-time traffic information through taxis equipped with GPS, integrating public transportation structures, and enabling buses to be more punctual. Urban sanitation conditions can also be improved through smart technologies, such as using internet-connected garbage bins or IoT-supported fleet management systems for waste collection and garbage processing, or applying sensor technology to smart waste containers (which automatically sense when garbage reaches the top of the container, notifying urban sanitation departments for timely collection). Manufacturing In the industrial manufacturing sector, AI and edge applications are expected to play an increasingly important role in the development of smart factories. Driven by the Industry 4.0 model, the next generation of smart factories will integrate advanced robotics and machine learning technologies into software services and industrial IoT to improve capacity and maximize production efficiency. Edge computing and artificial intelligence use local sensors to control and manage outputs, significantly increasing efficiency and reducing errors. Edge systems can respond to inputs within milliseconds, either adjusting to fix issues or shutting down production lines to prevent severe problems.
4. Challenges Facing Edge AI In recent years, the ecosystem for edge intelligence has gradually been constructed; however, behind the promising situation, edge intelligence still faces various challenges. First, due to the decentralization of cloud computing services, some traffic directly flows out through local edge intelligence platforms. In traditional core networks, computation, control, security, etc., are all completed within the core network. Research is needed on how to bill and control traffic on edge intelligence platforms; at the same time, edge intelligence has various deployment plans in actual network architectures, such as deployments in wireless access clouds, edge clouds, and aggregation clouds, and different architectures face slightly different issues.Second, the objects and scenarios that edge intelligence serves are quite diverse, and how to make a single edge intelligence platform adaptable to diverse third-party applications is currently a problem.Third, due to the fragmentation of services, edge intelligence may be deployed for a single scenario or two or three scenarios, requiring clear definitions of overall deployment capabilities and characteristics, as well as considerations for agile and intelligent operations.Fourth, research is needed on how to better integrate artificial intelligence and leverage the overall advantages of edge computing.Fifth, coordination is needed on how to further lighten the platform management subsystems based on OpenStack within the edge intelligence platform and consider quickly integrating some microservice architectures into the edge intelligence platform to reduce management overhead. Additionally, issues such as the business operation model of edge intelligence, deployment locations, self-healing, and automatic scaling still need to be clarified and resolved. Currently, although edge intelligence is still in its early development stages, it is expected to play a significant role in the next wave of computing, from the telecommunications industry and internet industry to the industrial field. There are high expectations for the significant role edge intelligence will play in enabling the three typical application scenarios of 5G, extending IT service environments and cloud computing capabilities to the mobile network edge, and it will undoubtedly work in synergy with cloud intelligence to support the digital transformation of various industries. With the popularization of 5G, it is believed that we will soon see a decrease in global edge AI service costs and an increase in demand.——END——
