Understanding Autonomous Driving Technology

In general, autonomous driving systems can be divided into perception layer, decision layer, and execution layer.

Perception Layer is defined as the collection and processing of environmental and in-vehicle information, involving multiple technologies such as road boundary detection, vehicle detection, pedestrian detection, etc. It can be considered an advanced sensor technology, with sensors including LiDAR, cameras, millimeter-wave radar, ultrasonic radar, speed, and acceleration sensors. Due to the limitations of a single sensor in perception, which cannot meet precise perception under various working conditions, autonomous vehicles need to utilize multi-sensor fusion technology to achieve smooth operation in various environments, which is also a key technology of the perception layer.

Decision Layer can be understood as making decisions based on perception information, determining appropriate working models, and formulating corresponding control strategies to replace human drivers in making driving decisions. The functions of this part are similar to assigning corresponding tasks to the autonomous vehicle. For example, in systems such as lane keeping, lane departure warning, distance keeping, and obstacle warning, it is necessary to predict the states of the vehicle and other encountered vehicles, lanes, pedestrians, etc., in the future. Advanced decision-making theories include fuzzy reasoning, reinforcement learning, neural networks, and Bayesian network technologies. Since human drivers face various road conditions and scenarios, and different individuals respond with different driving strategies to various situations, optimizing human-like driving decision algorithms requires very sophisticated and efficient artificial intelligence models and a large amount of effective data. This data needs to cover as many rare road conditions as possible, which is also the biggest bottleneck in the development of driving decision-making.

Execution Layer refers to the system controlling the vehicle according to the decision results after making decisions. Each control system of the vehicle needs to connect to the decision system through a bus and accurately control acceleration, braking, steering, light control, and other driving actions according to the bus instructions issued by the decision system to achieve autonomous driving.

Figure 1 Overview of Autonomous Driving System

Traditionally, the decision control software system of autonomous driving systems includes functional modules such as environmental prediction, behavior decision-making, action planning, and path planning.

❖ Environmental Prediction Module

The environmental prediction module serves as one of the direct data upstreams of the decision planning control module. Its main function is to predict the behavior of objects recognized by the perception layer and convert the predicted results into trajectories in the time-space dimension for transmission to subsequent modules. Typically, the object information output by the perception layer includes physical attributes such as position, velocity, and direction.

Using these output physical attributes, an “instantaneous prediction” can be made for the objects. The environmental prediction module is not limited to predicting objects based on physical laws but can also predict the behavior of perceived objects more “macro” by combining the objects with their surrounding environment and accumulated historical data. For example, in Figure 2, by recognizing a pedestrian’s historical movements on the sidewalk, it predicts that the pedestrian may cross the intersection on the sidewalk, while the historical trajectory of a vehicle can determine that it will turn right at the intersection.

❖ Behavior Decision Module

The behavior decision module plays the role of the “co-driver” in the entire autonomous driving decision planning control software system. This layer gathers all important information surrounding the vehicle, including not only the real-time position, speed, and direction of the autonomous vehicle itself but also all relevant obstacle information and predicted trajectories within a certain distance around the vehicle. The problem that the behavior decision layer needs to solve is to decide the driving strategy of the autonomous vehicle based on this information.

Since it needs to consider various types of information, behavior decision problems are often difficult to solve with a single mathematical model. Instead, some advanced software engineering concepts are needed to design rule engine systems. For example, in the DARPA Grand Challenge, Stanford’s autonomous vehicle system used a series of cost designs and finite state machines to design the trajectory and control instructions of the autonomous vehicle. Currently, Markov decision process models are increasingly being applied in the decision-making algorithms of autonomous driving systems’ behavior layers. In short, the behavior decision layer needs to combine the results of the environmental prediction module to output macro decision instructions for subsequent planning modules to execute more specifically.

❖ Action Planning Module

The planning module of autonomous vehicles includes action planning and path planning. The action planning module mainly plans short-term or even instantaneous actions, such as turning, obstacle avoidance, and overtaking; while the path planning module plans the vehicle’s driving path over a longer period, such as designing or selecting a route from the starting point to the destination.

The design idea of the autonomous driving system is to establish several driving states and trigger state switches based on different conditions. This design idea has a problem of poor smoothness in the switching process. In the actual system design process, the method of describing both real and non-real targets in the road as virtual particles is mainly adopted to enhance the smoothness of vehicle driving. Here, real targets refer to factors such as vehicles and pedestrians; non-real targets include speed limits, red lights, stop points, road curvature, weather conditions, etc. The advantage of the virtual particle model approach is that it unifies the algorithm model and effectively avoids the problem of sudden acceleration and deceleration of vehicles caused by switching targets or control modes in traditional control algorithms.

❖ Path Planning Module

The path planning module of autonomous vehicles refers to the planning of an effective path that avoids collisions and can safely reach the target point after giving the starting point and target point of the autonomous vehicle based on a certain environmental model. Path planning mainly includes two steps: establishing an environmental map that includes obstacle areas and free areas, and selecting an appropriate path search algorithm in the environmental map to quickly and real-time search for feasible paths. The result of path planning plays a navigational role in vehicle driving, guiding the vehicle from its current position to the target location. The representation methods of environmental maps are mainly divided into metric map representation and topological map representation.

The development of artificial intelligence machine learning, deep neural networks, and connected communication technologies further enriches the technical path for the development of autonomous vehicles and promotes the transition of autonomous driving technology from a single prototype demonstration to a direction capable of certain practical application and achieving autonomous positioning in typical traffic scenarios.

❖ Artificial Intelligence

Artificial intelligence is a new technical science that studies and develops theories, methods, technologies, and application systems used to simulate, extend, and enhance human intelligence. Its aim is to explore the essence of intelligence and produce a new type of intelligent machine that can react similarly to human intelligence. One of its major application areas is autonomous driving, with the main goal of enabling autonomous vehicles to have a certain degree of autonomous learning ability and to form memory-based cognitive recognition of simple traffic environments. Currently, the main applications of artificial intelligence technology in the field of autonomous vehicles are reflected in the following aspects.

1. Achieving Recognition and Cognition of Environmental Objects

By utilizing multi-view vision, LiDAR, millimeter-wave radar, and other sensors and recognition algorithms, accurate recognition of multi-surface objects in real road environments can be achieved. After integrating deep learning technology, iterative classification of the three-dimensional spatial dimensions and feature information of each object can be formed, enabling autonomous vehicles to have the ability to recognize and understand various environmental objects.

2. Achieving Detection of Drivable Areas

Using advanced sensor-based map collection technology, detailed annotations of the road (signs, markings, traffic lights, etc.) and high-precision location (longitude, latitude, altitude, etc.) can be extracted, enabling autonomous vehicles to extract the planar features of the road. At the same time, based on deep learning, the recognition of drivable and non-drivable areas of the road can be achieved.

3. Achieving Planning and Decision-Making of Driving Paths

Decision planning processing is another important application scenario of artificial intelligence technology in autonomous driving. Currently, mainstream artificial intelligence methods include state machines, decision trees, Bayesian networks, etc. With the development of deep learning and reinforcement learning technologies, decision-making for complex working conditions has been achieved, and online optimization learning can be conducted. Due to the many factors that affect driving path planning on actual roads, it will inevitably consume a lot of computing resources. To improve computing efficiency, Japanese researchers proposed the idea of a “safe field,” forming typical traffic scenarios as inputs for deep learning neural networks to improve the decision-making efficiency of autonomous vehicles and enhance path planning capabilities.

Figure 3 Framework for Unstructured Road Detection Based on Machine Learning

❖ Intelligent Connectivity

Combining the development of communication technology, using real-time communication technology between vehicles, roads, people, and clouds can further support the three major elements of data, computation, and algorithms in the application of artificial intelligence technology in autonomous driving. It can also provide solutions to the problems faced by collaborative driving systems for multiple vehicle types and scenarios. The specific architecture of the vehicle-cloud collaborative autonomous driving system based on intelligent connectivity is shown in Figure 4.

Figure 4 Schematic Diagram of the Construction Plan for Vehicle-Cloud Collaborative Autonomous Driving System Based on Artificial Intelligence

This architecture plan is divided into two parts: AI-based intelligent terminals for autonomous driving and cloud systems for autonomous driving based on big data analysis, jointly forming a vehicle-cloud collaborative autonomous driving system that integrates precise environmental perception, intelligent decision-making, and optimized driving control execution. The vehicle-cloud collaborative technology can achieve precise driving environment perception, intelligent decision-making, and optimized driving action control under different driving conditions and application scenarios, as well as information data interaction and collaboration between the vehicle and the cloud.

The intelligent connectivity technology of the autonomous driving system mainly addresses the issues of insufficient multi-source heterogeneous data fusion and inadequate computing power of front-end devices. It transmits sampling data from vehicle sensor nodes (such as GPS/INS data, millimeter-wave radar data) and multimedia data (such as camera images) to the cloud database at a certain frequency, while conducting online processing, offline processing, traceability processing, and complex data analysis. Based on the intelligent driving control model of integrated application algorithms of artificial intelligence, it provides reliable and efficient collaborative control solutions for vehicle decision-making.

The cloud platform for the application of artificial intelligence algorithms is the core part of the autonomous driving cloud system. It analyzes perception fusion information using machine learning, data mining, and related technologies, providing decision basis for vehicle control planning. It integrates large-scale scalable computing, storage, data, and application distributed computing resources using virtualization and networking technologies to complete the learning and training of artificial intelligence model algorithms in the cloud, and deploys them to embedded platforms through vehicle-cloud collaborative technology, allowing deep application of artificial intelligence algorithms in the vehicle’s autonomous driving system.

Currently, the application of connectivity technology in the field of autonomous driving mainly focuses on information services and top-level monitoring. Achieving highly automated driving through intelligent connectivity technology still requires addressing challenging issues such as information security, transmission delay, and network coverage to truly realize practical applications.

❖ Intelligent Computing Platform

Autonomous vehicles are gradually transforming from transportation tools into new mobile intelligent terminals. The change in the functions and attributes of vehicles leads to changes in their electronic and electrical architecture, which in turn requires stronger computing, data storage, and communication capabilities as a foundation. The onboard intelligent computing platform is an important solution to meet these requirements.

The onboard intelligent computing platform mainly completes the function of taking environmental perception data, navigation positioning information, vehicle real-time data, cloud intelligent computing platform data, and other V2X interaction data as inputs, and outputs control instructions for driving, transmission, steering, and braking based on core control algorithms such as environmental perception positioning, intelligent planning decision-making, and vehicle motion control, achieving automatic control of the vehicle. It also outputs data to the cloud intelligent computing platform and V2X devices, and can achieve human-computer interaction of vehicle driving information through a human-computer interaction interface.

(By/Yuan Jianhua, Lu Wenjie, Luo Weiming, Fan Zhixiang, National Engineering Laboratory for Integrated Optimization and Safety Analysis Technology of Road Traffic, Autonomous Driving Testing Technology R&D Team)