Think Tank Highlights #Global Defense Dynamics #US Military Dynamics #Russian Military Dynamics #Taiwan Affairs #WeChat Store Available
#South Korea #Raytheon #Japan #Electronic Warfare #Northeast Asia Military Dynamics #Drones
The following article is sourced from Giant Vision Security, reprinted from Unmanned Competition
Unmanned Aerial Vehicles (UAVs), commonly known as drones, face numerous challenges and applications that can be enhanced through Artificial Intelligence (AI), edge computing, and edge AI.
AI “describes the workflow of machines that would require intelligence if performed by humans.” Recently, robots/drones have been utilizing AI to accomplish various intellectual tasks. Compared to traditional cloud computing, edge computing brings computing services (such as storage and processing) closer to the end users at the network edge. Computation is not performed on remote cloud servers but can be done on nearby edge servers or devices (such as drones/unmanned aircraft), or a combination of both. Edge AI can be summarized as the integration of AI with edge computing.
The key technical challenges for drones include autonomous navigation, power management, security and privacy, formation control, computer vision, and communication.
Autonomous navigation can be further divided into localization and mapping, path planning, and collision avoidance systems. Important applications of drones include delivery systems, precision agriculture, civil infrastructure inspection, search and rescue operations, serving as aerial wireless base stations, and drone light shows.
Traditional cloud AI computing occurs on cloud servers, providing more processing power and storage space compared to edge AI. However, edge AI has many advantages over cloud AI. Notably, it offers lower latency, higher reliability, improved security and privacy, lower costs, and reduced energy consumption.
The identified technical challenges for drones have certain particularly important advantages of edge AI. For example, autonomous navigation (especially collision avoidance systems) heavily relies on the lower latency of edge AI. Given the flight speed of drones, if obstacles cannot be detected and avoided within fractions of a second, collisions are likely to occur. The improved latency provided by edge AI compared to cloud AI could be the difference between a collision and no collision. Different applications also rely more on specific technical challenges. For instance, precision agriculture (such as crop management) primarily depends on the technical challenges of computer vision.
While the benefits of edge AI for drones and their technical challenges/applications are undeniable, there are still many challenges in implementing edge AI for drones. Key challenges in implementing edge AI for drones include developing distributed training algorithms, security and privacy, resource allocation, and real-time requirements. However, potential solutions can be identified for each implementation challenge. For example, the implementation challenge of “real-time requirements” outlines that drones often need feedback to operate “in real-time” (e.g., when tracking fast-moving cars).
However, sometimes collecting data, training models, and deciding actions can take too long. One solution is to reduce inference time through “model pruning” and “knowledge distillation.” Future research directions for edge AI applications in drones can also be defined by the challenges faced and lessons learned.
Drones and edge AI are exciting and active research fields.
The key technical challenges of drone systems can be categorized into six main areas: 1) Autonomous Navigation; 2) Formation Control; 3) Power Management; 4) Security and Privacy; 5) Computer Vision; 6) Communication
A. Autonomous Navigation
1) Introduction:
Autonomous navigation can refer to vehicle navigation that is remotely operated by humans but has some simple onboard algorithms to take over and prevent collisions, or it can refer to fully automated vehicle navigation that reaches point A to point B without any human intervention (e.g., drones used for package delivery). Depending on the application, vehicles autonomously navigate by employing localization and mapping, path planning, and/or collision avoidance. For example, in the case of remotely operated drones using only some simple onboard algorithms, only collision avoidance is employed. In fully autonomous cases, localization and mapping, path planning, and collision avoidance are required. Therefore, this article introduces autonomous navigation from the perspectives of “localization and mapping,” “path planning,” and “collision avoidance systems.”
a) Localization and Mapping:
For robots, mapping is the process of constructing a map of a specific area (2D or 3D), while localization is the process of determining the robot’s position and orientation relative to a reference frame. Accurate localization is a daunting task for any robot, and for aerial robots like drones, localization is particularly challenging due to the three-dimensional nature of the environment. To make this accurate localization task easier, drones often heavily rely on the Global Positioning System (GPS), where GPS position measurements are fused with onboard Inertial Measurement Unit (IMU) measurements to accurately estimate the drone’s attitude (position and orientation). This method works well because GPS data can compensate for the cumulative errors of the IMU (caused by drift in its measurements).
However, in many cases, GPS services are unavailable/unreliable, such as indoors (factories, warehouses, etc.), in emergency/disaster situations, near tall trees/buildings, or near bodies of water. These environments are aptly termed GPS-denied environments, and navigating drones in such environments using only IMU measurements to achieve precise localization is very challenging due to the aforementioned cumulative errors of the IMU.
The most common solution to this problem is vision-based solutions, which accurately localize UAVs without GPS by combining IMU and visual sensor measurements. When the two measurements (IMU and visual sensor measurements) are fused, improved attitude estimation can be achieved. The most notable of these vision-based solutions are Visual SLAM (Simultaneous Localization and Mapping) and Visual Odometry (VO). SLAM algorithms aim to estimate the robot’s pose while constructing a representation of the explored area (visual SLAM algorithms rely on visual sensors). VO predicts the UAV’s pose incrementally by analyzing the changes caused by motion across a series of images.
b) Path Planning:
Drone path planning is the problem of determining the path for a drone from a starting point to a target point. There are many types of path planning techniques, but they all attempt to find the optimal (or near-optimal) path based on performance metrics (such as shortest time, shortest route, or lowest operational cost).
As expected, the integration of path planning and collision avoidance is crucial. They are often used together, so collision avoidance is often referred to as “local path planning,” while path planning is referred to as “global path planning.” The idea is that global path planning generates the best route while considering the entire environment, while local path planning performs local processing when environmental changes are detected and executes collision avoidance accordingly. Once collision avoidance is executed, an attempt is made to return to the global path.
Drone path planning (global path planning) techniques can be categorized into various types. One example is the research by Aggarwal and Kumar, which categorizes path planning techniques into representative techniques, cooperative techniques, and non-cooperative techniques.
c) Collision Avoidance Systems:
Due to the high-speed flight of drones, they are at a high risk of collision, whether they collide with other objects or other objects collide with them. Collision avoidance systems are crucial for drones to avoid such collisions and ensure safe flight.
Yasin et al. categorize collision avoidance systems into two main categories: 1) Perception and 2) Action, where perception is primary. In the perception phase, sensors perceive the environment and detect obstacles, while in the action phase, collision avoidance methods utilize this information to avoid collisions. Perception is subdivided into active sensors (sensors that emit/launch and then receive their own source) and passive sensors (sensors that only read energy released by other sources). Active sensors are further divided into sonar, LiDAR, and radar, while passive sensors are divided into cameras and infrared.
In the action phase, collision avoidance methods are divided into “four main methods”: 1) Geometric; 2) Optimization; 3) Force Field; 4) Perception and Avoidance. The “Geometric” collision avoidance method uses the speed and position information of the drone and obstacles to ensure that minimum distance violations do not occur, often involving trajectory simulation. The “Optimization” collision avoidance method uses known obstacle parameters to find the best (or near-best) route. The “Force Field” collision avoidance method manipulates attractive/repulsive forces, while the “Perception and Avoidance” method makes avoidance decisions based on obstacle detection in real-time.
2) The Role of Artificial Intelligence:
The greatest contribution of AI to autonomous navigation is through computer vision algorithms, which are crucial for three major challenges: 1) Localization and Mapping; 2) Path Planning; 3) Collision Avoidance Systems. Recent advancements in computer vision algorithms (especially deep learning algorithms) and camera technologies (particularly RGB cameras) enable drones to effectively detect objects using high-resolution, lightweight, and inexpensive onboard cameras. In terms of localization and mapping, especially in the absence of GPS data, the use of computer vision algorithms is heavily relied upon in visual methods (as discussed in Section III-A1a). Additionally, in path planning and collision avoidance systems, computer vision algorithms are essential for quickly detecting oncoming objects to avoid collisions.
3) The Role of Edge Artificial Intelligence:
Compared to traditional AI under cloud models, using edge AI can bring many benefits to autonomous navigation. While all general advantages of edge computing (lower latency, higher reliability, improved security and privacy, reduced costs, and lower energy consumption) and all general additional advantages of edge AI (further improvements in energy consumption/privacy/communication) apply here, the lower latency and higher reliability provided by edge AI are particularly important for autonomous navigation issues. Therefore, this section briefly discusses the advantages and disadvantages of edge AI.
a) Lower Latency:
Using AI with traditional cloud computing may involve considerable transmission delays in sending and receiving data between the drone and the central cloud server. Edge AI allows most processing to occur on the terminal device or nearby edge servers, resulting in much shorter associated transmission delays compared to remote clouds. Even in cases described in Section II-B, where certain applications require edge servers to communicate with the core cloud for data synchronization, the transmission delay is still not significant. This is because preprocessing at the edge reduces the overall traffic load (i.e., less data sent from each device/drone).
Collision avoidance systems are a typical example of real-time latency-sensitive challenges that particularly rely on low latency, making edge AI potentially essential. The process of detecting and avoiding oncoming objects (unless the drone and the oncoming object are moving very slowly) needs to be completed within fractions of a second. If this process takes too long, it could lead to crashes, resulting in significant economic losses or worse (e.g., fatal consequences). For path planning, localization, and mapping, the latency of traditional AI may be sufficient in most static environments, especially in sparsely populated or high-altitude areas. That said, considering the limitations of high latency on flight speed, the improved latency of edge AI remains a significant benefit.
b) Improved Reliability:
Running AI under traditional cloud computing models relies on the reliability of central cloud servers. If the central cloud server is inaccessible/fails, it can lead to catastrophic consequences (e.g., crashes). The likelihood of this occurring with edge AI is much lower because processing occurs closer to the user (on edge servers/devices), significantly reducing the chances of network interruptions. If an edge server does fail, another server should be able to provide service; even if not, the terminal device can handle a significant number of requests on its own and still access some locally stored data.
When using AI under traditional cloud computing models, the inaccessibility/failure of the central cloud server can lead to crashes because the drone’s video source cannot be analyzed, and oncoming objects that could collide with the drone cannot be detected. With traditional cloud AI, path planning, localization, and mapping can cope without a central cloud server for a slightly longer time, but if this connection loss persists for too long, hovering in the air without completing tasks or crashing remains inevitable.
c) Disadvantages:
Edge AI has two significant disadvantages that affect autonomous navigation: the noticeable lack of network devices in certain areas and the fewer technicians available to implement, repair, or manage network devices in those areas. In other words, in sparsely populated areas and/or areas with limited financial or technical resources, there may be fewer edge servers on the network, and in many such areas, there may be fewer technicians capable of implementing, repairing, and managing edge networks. This can affect autonomous navigation as it may rely on edge servers, meaning the positive impacts of edge AI may diminish, especially in terms of latency and reliability.
Additionally, another disadvantage of autonomous navigation is the need to offload drone tasks between different edge servers. When a drone leaves the service range of a specific edge server, that server may need to migrate the tasks being offloaded from the drone to other edge servers. Given the high speed of drone flight, this can limit the task performance of the drone.
B. Formation Control
1) Introduction:
When performing drone tasks, multiple drones working collaboratively are often preferable to a single drone completing tasks alone (see the end of Section II-A). Driven by the advantages of multiple drones working together (such as transporting large payloads or searching for objects/people over large areas), extensive research on formation-related studies has been conducted in recent decades, with formation control being one of the most active research topics.
Formation control is inspired by self-organization in nature, such as flocks of birds, and is the coordinated control of multiple robots in a “formation.” Formation is defined as a network of agents that are interconnected through controllers, where each agent must maintain relationships with adjacent agents. Formation control can be broadly divided into two categories: 1) Leader-Follower and 2) Non-Leader.
2) The Role of Artificial Intelligence:
a) Reinforcement Learning:
Reinforcement Learning (RL) is a field of machine learning and thus a field of artificial intelligence. It can be described as learning how to maximize a numerical reward signal, where the learner is not given actions to take but must try these actions and infer which actions will yield the greatest reward. RL is particularly useful for the coordinated movement and communication of drones. Therefore, many papers, especially recent ones, have studied the application of RL in drone formation control. For example, Knopp et al. proposed an RL algorithm using GQ (λ) for leader-follower formation control scenarios.
b) Deep Reinforcement Learning:
A challenge faced by RL is that learning from each state and determining the optimal reward path (i.e., policy) can be very difficult. RL can be combined with deep learning to produce another subfield of machine learning known as Deep RL (DRL). DRL can use neural networks to handle higher-dimensional state/action spaces. For example, Conde et al. used DRL to drive multiple drones to achieve formation, allowing the use of deep neural networks to estimate the quality of specific states.
c) Multi-Agent Systems and Reinforcement Learning:
Multi-Agent Systems (MAS) are a method for solving complex problems by breaking them down into smaller tasks. Each individual task is assigned to an autonomous entity called an “agent,” and each agent selects appropriate actions to solve the task based on multiple inputs, including its operational history, interactions with neighboring agents, and its goals. RL can be effectively applied to such MAS. For example, Liu et al. proposed a distributed model-free solution based on RL to address the leader-follower formation control problem in MAS.
3) The Role of Edge Artificial Intelligence:
Similar to the case of autonomous navigation, when adopting edge AI instead of traditional AI under cloud computing models, there are many benefits for formation control, so most general advantages of edge computing (lower latency, higher reliability, etc.) and general additional advantages of edge AI (e.g., further improvements in energy consumption) apply. Likewise, this section will focus on particularly interesting and important advantages, which for formation control issues is cost reduction.
a) Cost Reduction:
When referring to drone formations, we are talking about swarms of drones that generate large amounts of streaming data that are continuously transmitted from the drones for processing. Edge AI can reduce the amount of data that needs to be sent to servers through onboard preprocessing and can also reduce the distance data is sent by sending it to edge servers (rather than more remote centralized cloud servers). The reduced amount of data is particularly helpful in lowering operational costs, as cloud computing costs can be quite high when processing large amounts of data. That said, the costs of deploying and maintaining edge devices distributed over vast areas can be quite high and should be considered.
C. Power Management
1) Introduction:
A well-known issue with drones is their limited battery life, which in turn leads to limited flight time. In addition to improving the energy efficiency of drones, more energy can be obtained through innovative means (i.e., energy harvesting technologies) to extend battery life. Common energy harvesting technologies include solar energy, wireless charging, and battery swapping.
a) Solar Energy:
Solar energy is cost-effective and environmentally friendly, making it an ideal energy source. Solar cells are particularly useful for drones as they require long flight times at high altitudes. Solar-powered drones typically use fixed batteries as auxiliary power sources, relying less on batteries during the day, but battery power becomes very important at night or in adverse weather conditions. That said, surface area, weight limitations, and dependence on light intensity are major limiting factors for commercial solar-powered drone applications.
b) Wireless Charging:
Wireless charging has not yet become a standard feature for drones. However, in recent years, extensive research has been conducted to make wireless charging for drones feasible. Explored technologies include capacitive coupling, magnetic resonance, and even charging from power lines through wireless power transmission.
2) The Role of Artificial Intelligence:
Artificial intelligence can address power management issues through planning and optimization algorithms. For example, algorithms that find the optimal route for drones to minimize power consumption or algorithms that select the best charging/battery swapping stations, taking into account the current battery level and deviations from the current route. For instance, Zhang et al. used DRL to find the fastest path to a charging station.
Additionally, Convolutional Neural Networks (CNNs) can be used to identify charging stations and their landing points, while Recurrent Neural Networks can accurately predict when a drone will run out of battery, and AI can also help improve the efficiency of drones, allowing them to achieve more objectives with less power.
3) The Role of Edge Artificial Intelligence:
As with Sections III-A3 and III-B3, the general advantages of edge computing and the general additional advantages of edge AI also apply here. The advantage of edge AI in reducing energy consumption is particularly important and will be discussed.
a) Reducing Energy Consumption:
As mentioned in Section III-B3, edge AI can reduce the amount of data that needs to be sent to servers through onboard preprocessing and can also reduce the distance data is sent by sending it to edge servers (rather than more remote centralized cloud servers). In addition to lowering costs, this can also reduce energy consumption, as transmitting data to remote clouds incurs significant energy costs. By reducing data size and transmission distance, overall energy consumption can be lowered, which is especially important for drones as batteries are a scarce resource. Furthermore, when using edge AI, distributed federated learning within drone swarms can further reduce the data sent to servers and the associated energy consumption. For example, Zeng et al. proposed a drone swarm framework based on distributed federated learning, which features a “leading drone” and several “following drones.” All following drones locally train models based on their collected data, and then each following drone sends this locally trained model to the leading drone. This drone aggregates all models to create a global model, which is then sent to the subsequent drones.
D. Security and Privacy
1) Introduction:
Security and privacy are very important issues for all digital systems, especially for drones. Compared to other privacy-invading devices, drones possess unique flexible access technologies (which are attractive to criminals) and are also vulnerable to attacks targeting wireless links, network elements, physical elements, and the interfaces between network and physical elements.
a) Security:
While many papers discussing drone security address the same issues, the approaches taken differ.
Shakhatreh et al. categorize the attack vectors of drone systems into attacks on communication links, the drone itself, ground control stations (GCS), and humans. They also define three major cybersecurity challenges for drone systems: 1) Confidentiality challenges (referring to protecting information from unauthorized access); 2) Availability challenges (referring to ensuring that drone system services and data work as expected and are accessible to authenticated users); 3) Integrity challenges (referring to ensuring the authenticity of information). Ullah et al. briefly explain the classification of attackers (internal/external, malicious/reasonable, active/passive, and local/extended) and attacks (e.g., spoofing, denial of service, and GPS spoofing). However, this section analyzes drone attacks from the perspective of sensors and communication links.
The information acquired by sensors can alter the behavior of drones and have significant implications for security. GPS sensors are a common attack vector, as they are often used to obtain accurate location information. The most common ways to attack drone GPS include “jamming” or “spoofing.” Jamming refers to unauthorized parties broadcasting interference signals to prevent the reception of various signals. Spoofing can involve unauthorized parties recording satellite signals and transmitting them to the drone (relay-type spoofing) or using certain programs to generate signals based on real signals (generation-type spoofing). One method to address GPS jamming is to adopt alternative navigation methods, such as using visual and inertial navigation systems employing SLAM or VO (see the localization and mapping section, Section III-A1a). Solutions to GPS spoofing include authenticating GPS signals, such as checking GPS observables that represent signal travel time or detecting sudden changes in signal power/observables that may indicate the onset of spoofing attacks. Other commonly vulnerable sensors include stereo vision sensors that may be spoofed (e.g., by using lasers pointed at the ground to induce features) or MEMS gyroscopes that may be attacked by ultrasound (e.g., causing the drone to lose balance).
In terms of communication links, the interaction between drones and ground stations relies on communication links, which are vulnerable to attacks if not secured. For example, one way drones communicate with ground terminals is through WiFi, which is susceptible to Deauth attacks, leading to disconnection between the drone and terminal, and the drone control password being compromised. Solutions include: asking the user whether to automatically return after the drone has lost contact for 10 seconds; using radio signals instead of WiFi signals.
b) Privacy:
Drone privacy invasion is both easy and difficult to capture invading drones. Preventing drones from infringing on personal privacy primarily has two solutions. One is to register home addresses in no-fly zone databases, but this still does not ensure that invading drones will not fly into no-fly zones. The second solution is to use technologies/systems to detect, track, and deploy drones in space.
Additionally, malware can exploit drones to collect personal information. For example, Snoopy malware can be installed on drones to collect personal information and track/analyze individuals using smartphones. Drones need to continuously evolve to counter this ever-evolving malware.
2) The Role of Artificial Intelligence:
Artificial intelligence can be used to help prevent drones from being attacked and can also be used in the systems briefly discussed above to prevent drones from infringing on people’s privacy (by flying over/near no-fly zones). Regarding the latter, by 2026, the “global anti-drone market” is expected to reach $2.597 billion. An example is the monitoring system from Japan Electric Company, which uses acoustic, thermal, infrared, and/or radio communication sensors/detectors to perceive invading drones and provides options for tracking system owners to discard/capture drones. Detection algorithms for drones particularly employ artificial intelligence, for example, Zhang et al. proposed a drone detection algorithm based on Artificial Neural Networks (ANN).
For the former (preventing drones from being attacked), Challita et al. argue that it is crucial to detect potential attacks by discovering bad/abnormal drone movements. Challita et al. use RNNs as an example, where RNNs can predict drone movements, thus detecting abnormal movements of drones. Additionally, AI protection can utilize models trained against malicious files to protect drones from zero-day attacks.
3) The Role of Edge Artificial Intelligence:
As with the “Role of Edge AI” sections for other technical challenges, the general advantages of edge computing and the general additional advantages of edge AI also apply. Clearly, the security and privacy advantages of edge AI are partially relevant and will be discussed.
a) Improved Security:
Typically, using artificial intelligence in traditional cloud computing models requires transmitting all data to a central server. This is considered “highly vulnerable” because a single attack (e.g., a DDoS attack) can cause severe damage. Using edge AI means distributing data processing across multiple devices/servers. While it must be acknowledged that this distributed data processing increases the potential attack surface, the risks are more dispersed, thus reducing the impact of successful attacks (such as DDoS attacks). Additionally, since edge AI supports processing at the edge, less data is sent, resulting in less data being intercepted.
In terms of shutting down attacks, the distributed and decentralized nature of edge computing means that the vulnerable parts of edge AI systems are easier to shut down than traditional cloud AI systems, which typically require shutting down the entire network.
Integrating blockchain into drone systems can bring many security advantages. For example, blockchain can ensure that each drone has a copy of the blockchain, reducing the vulnerability of the drone network to signal interference. In this case, individual drones use the detailed information of other drones’ flight paths contained in their blockchain copies to determine their own paths. Another example is how blockchain can detect malicious drones that change network information. Malicious drones may initially be part of the drone network and later be hijacked or enter the network later. Blockchain can help prevent such attacks through blockchain consensus algorithms, where any drone can report suspicious activity. If the number of entries contradicting a drone exceeds a certain threshold, that drone is deemed malicious.
b) Improved Privacy:
Edge AI enables real-time computing. If it is inevitable to capture/record privacy-invading images, data does not need to be sent and stored on a remote centralized cloud server but can be processed locally on the drone or edge server. The significance of this is that such data is less likely to be hacked and does not need to be stored on a remote centralized cloud server.
Federated learning and blockchain can both enhance the security of drone communications. Federated learning can avoid sending any raw data from devices/drones (only sending local model updates). Blockchain can encrypt data and store it on the blockchain, making it inaccessible to anyone without the correct decryption key. Additionally, blockchain can protect four main types of data in the drone network (drone identifiers, flight path control, sensor data, and flight schedules) by writing and updating them within blockchain blocks.
c) Disadvantages:
As noted in the security section (Section III-D3a), it should be noted that distributing data/data processing across multiple edge nodes has the disadvantage of increasing the potential attack surface. Even with a blockchain-based system, where each drone has a copy of the distributed ledger, some sensitive information will be spread across all drones in the system. Additionally, it should be noted that in a blockchain-based drone swarm, if more than half of the drones in the swarm are hacked, the entire swarm can be controlled (this type of attack is known as a 51% attack).
E. Computer Vision
1) Introduction:
The purpose of computer vision is to enable computers to understand the environment from visual information (whether from a single image or a series of images). In recent years, there has been growing interest in the automatic understanding of visual data collected by drones, as computer vision plays a crucial role in most drone applications (from aerial photography to SAR operations).
From the perspective of computer vision, the core task of such applications is scene parsing. Different applications require different levels of scene parsing, from locating objects to determining the precise boundaries of objects to recognizing objects. Drone computer vision applications include object detection, object recognition, object tracking, collision avoidance, autonomous navigation, and 3-D reconstruction. Such image processing can be performed remotely on servers (edge or central cloud) or locally on the drone (embedded).
a) Remote Computer Vision Processing:
Drones typically lack the processing power to handle images captured by their cameras, so processing needs to occur at different locations. From a latency perspective, the ideal processing location is an edge server, although computer vision can also be processed on more distant centralized servers.
b) Real-Time Embedded Computer Vision Processing:
If the goal is to enable drones to achieve true autonomy and reliability, real-time embedded computer vision processing is preferable to remote computer vision processing, as remote processing requires high bandwidth, minimal latency, and extremely reliable wireless links, which cannot always be guaranteed.
The most prominent limitation of real-time embedded computer vision processing for drones is onboard computational capability. Van Beeck et al. point out that the computational requirements of state-of-the-art drone computer vision algorithms often conflict with hardware resource limitations.
2) The Role of Artificial Intelligence:
In computer vision technology, the use of artificial intelligence is not mandatory; for example, Petricca et al. can perform rust detection based on the number of pixels containing certain red components. Nevertheless, despite the fact that AI-based techniques may require large datasets to achieve optimal results, computer vision applications extensively utilize artificial intelligence. The area of artificial intelligence that intersects most with computer vision is deep learning.
a) Deep Learning:
According to research by Lecun et al., deep learning enables computational models composed of multiple processing layers to learn data representations with multiple levels of abstraction. The purpose of the earlier layers is to learn low-level features such as edges, while the later layers aim to combine features into more complete representations. An example of the application of deep learning in drone computer vision is the method proposed by Ye et al., which employs a deep learning classifier to detect and track other drones. Another example is the method proposed by Padhy et al., which uses a CNN model to help drones navigate autonomously in indoor corridor environments without GPS.
3) The Role of Edge Artificial Intelligence:
The role of edge AI in computer vision involves computer vision processing on edge servers and drones. When AI processing can be performed on edge servers or devices (rather than under traditional cloud models), all general advantages of edge computing and general additional advantages of edge AI apply. The advantage of low latency is particularly important for computer vision applications, and thus will be emphasized. As with the “Role of AI” section (Section III-E2), this section will also particularly highlight deep learning, but with a focus on embedded deep learning, as this has not been discussed previously.
a) Lower Latency:
As mentioned in Section III-A.6, drones using traditional cloud AI experience considerable transmission delays when sending and receiving data between themselves and central cloud servers. Compared to remote clouds, edge AI can perform most processing on terminal devices or nearby edge servers, resulting in significantly shorter transmission delays. Even in cases described in Section II-B, where certain applications require edge servers to communicate with the core cloud for data synchronization, the overall traffic load is smaller due to preprocessing at the edge, so transmission delays are not significant.
Most computer vision applications (from object detection to object recognition to object tracking to collision avoidance to self-navigation) require low latency, as objects can be lost/collide if certain latency standards are not met.
b) Deep Learning:
According to Van Beeck, at the “UAVision2020” workshop focused on real-time image processing onboard drones, all accepted workshop papers (covering a wide range of different applications) utilized deep learning. Castellano et al. and Zhao et al. are good examples of using deep learning, both describing the use of CNNs for crowd counting or understanding. Other examples include Stad and Zhang using deep learning for object tracking, and Peralta et al. using deep learning for 3-D reconstruction. Recent examples include Onishi and Ise using CNN methods to build tree recognition and mapping systems using drone RGB images, and Kung et al. proposing a CNN model for automatic detection of building defects (such as cracks).
Reprint Statement
This article is sourced from publicly available information on the internet or has been reprinted with authorization from the original public account. The copyright belongs to the original author. The content of the article does not represent the views of this platform or its authenticity, and the purpose of reprinting is to convey information and for online sharing. If there are any copyright issues, please contact us, and we will delete it immediately.
