Understanding High-Precision Maps: Past, Present, and Future

Understanding High-Precision Maps: Past, Present, and Future

▍01 Introduction

In Cooperative, Connected and Automated Mobility (CCAM), the more capable intelligent driving vehicles are in perceiving, modeling, and analyzing their surrounding environment, the better they can understand, make decisions, and safely and efficiently execute complex driving scenarios. High-Definition (HD) maps represent road environments with centimeter-level accuracy and lane-level semantic information, making them a core component of intelligent mobility systems and a key driver of CCAM technology. These maps provide automated vehicles with a significant advantage in understanding their surroundings. High-precision maps are also seen as hidden or virtual sensors, as they aggregate knowledge from physical sensors (maps), such as LiDAR, cameras, GPS, and IMUs, to model the road environment. High-precision maps are rapidly evolving into a comprehensive representation of smart city digital infrastructure, encompassing not only road geometries and semantic information but also real-time perceptions of road participants, weather conditions, work zones, and accident updates.

The large-scale deployment of autonomous vehicles requires a massive fleet to build and maintain these maps, working collaboratively to continuously update the maps, enabling the autonomous vehicles in the fleet to operate correctly. This article provides a comprehensive overview of the various applications of these maps in highly automated (AD) systems. We systematically review the latest advancements in different methods and algorithms for building and maintaining high-precision maps. We also discuss and synthesize the data, communication, and infrastructure requirements for high-precision map distribution. Finally, we review current challenges and discuss future research directions for next-generation digital map systems.

▍02 History of High-Precision Maps

A. Digital Maps

The advent of modern satellite systems and imaging technologies has revolutionized the creation of precise and detailed digital representations of the world, resulting in what we now refer to as digital maps, such as Google Maps, OpenStreetMap, Apple Maps, Garmin, and Mapbox. Digital maps encode road structures and basic semantic information along with Points of Interest (POIs). Several methods and techniques can extract and identify geographic features needed to build these maps from satellite imagery. Digital maps have become indispensable tools in our daily lives, especially when integrated with GPS. In fact, this integration has been a core component in building a multitude of digital services, most notably navigation and routing. These maps were primarily developed to assist humans and are now utilized in the latest vehicles to aid human drivers. However, the accuracy and precision of these maps, as well as their update times required for AD, are limited, where vehicles need a certain level of positional accuracy and detailed lane-level information.

B. Enhanced Digital Maps

Digital maps have been significantly improved to meet the requirements of Advanced Driver Assistance Systems (ADAS) functions, such as lane-keeping assistance and adaptive cruise control (ACC). Typical features of these enhanced digital maps include speed limits, road curvature and slopes, lane information, and traffic signs and signals. Enhanced digital maps, also known as ADAS maps, are now an essential part of most modern vehicles, enabling ADAS functionalities. Although enhanced digital maps introduced lane-level information, their geometric accuracy and level of semantic detail limit their applicability at higher levels of autonomy. In AD systems, vehicles are required to achieve high-precision localization relative to their environment, understand the current situation, and plan collision-free trajectories. To achieve this level of autonomy, automated vehicles not only need maps with centimeter-level positional accuracy and lane-level geometric information but also 3D models of the environment, including all static and dynamic features of the road environment.

C. High-Precision Maps

The aforementioned requirements led to what we now refer to as high-precision maps, or simply HD maps. Figure 1 highlights the evolution of maps, their features and uses, and the information contained within the maps along with their levels of accuracy and detail.

D. Scalable Maps

Over the past decade, both academia and industry have made significant research and development efforts to push the limits towards affordable, self-maintaining, and scalable high-precision maps. However, there are various outstanding issues in scaling high-precision maps. These challenges hinder the realization of the full potential and ultimate goals of high-precision maps in autonomous mobility. These challenges can be categorized into the following:

1) Data Collection: Collecting data for high-precision maps can be a time-consuming and labor-intensive process. It typically involves a combination of sensors such as GPS, IMUs, LiDAR, and cameras to collect detailed information about the environment.2) Data Communication: Data communication involves transmitting map data from the collection site to the processing site to build high-precision maps, and finally to the consumer site, such as autonomous vehicles. Surveying vehicles generate vast amounts of data from different sensors, which need to be processed to build and update maps. Real-time processing of this data from a large number of surveying vehicles is indeed a challenge.3) Data Processing: Data processing is the step of creating high-precision maps by extracting the elements and features needed to build them. This can be a very complex task, especially for large maps, as it involves aggregating and aligning data from multiple sources and ensuring the map is accurate and up-to-date. In cases where a large number of mapping vehicles are involved in the mapping process, precise time synchronization must be ensured to avoid data misalignment. Using the Pulse Per Second (PPS) signal generated by GPS for synchronization is often the most common method to synchronize all onboard sensors.4) Map Maintenance: Map maintenance refers to the ongoing process of updating high-precision maps based on changes in the road environment, such as construction sites, road closures, and modifications to road connections. Since the road environment is highly dynamic and can change, this process requires frequent data collection and processing work.5) Data Privacy and Security: Data security and privacy are crucial for high-precision maps as they often contain sensitive information, such as the locations of buildings and infrastructure. Ensuring that this data is protected from misuse is a significant challenge.6) Mapping Costs: The cost of mapping is an important factor in creating high-precision maps. Large-scale mapping requires the use of numerous surveying vehicles, each equipped with an expensive suite of high-precision sensors. This cost becomes very significant when mapping large areas. While mapping high-precision maps using consumer-grade sensors is possible, it comes at the cost of using complex mapping algorithms.

E. Major Contributions

This article provides an in-depth overview of high-precision maps, including a unified model of their layered architecture. Additionally, this article emphasizes the importance of high-precision maps in modular AD systems and synthesizes how they are used in various AD core functionalities. Given the aforementioned challenges regarding map data collection, communication, processing, security, and costs, this article extensively reviews past works on building and maintaining high-precision maps, including cost-effective solutions and the communication and map data requirements from generation to distribution. Furthermore, this article discusses the current challenges faced in building and maintaining high-precision maps. Finally, we provide some insights for the future and next-generation mobile high-precision maps. The main contributions of this work can be summarized as follows:

– An independent overview of high-precision maps as a backdrop for the broader intelligent transportation system community.

– A detailed review of state-of-the-art technologies used in various core functionalities of AD systems.

– A comprehensive investigation of different methods, approaches, and algorithms to maintain high-precision maps at different levels and keep them up to date.

– A discussion of the major challenges and future prospects of high-precision maps in integrated administration and other aspects.

▍03 High-Precision Maps: Overview

Early high-precision maps were merely extensions of enhanced digital maps used in ADAS, referred to as early maps. The term HD map has only recently emerged but is widely accepted in the CCAM industry, including by Tier 1 automotive companies, map providers, and original equipment manufacturers.

High-precision maps encapsulate all necessary information for automated vehicles to understand the driving environment with very high accuracy. While high-precision maps are widely regarded as the core drivers of CCAM, there are no clear guidelines or standards on what information constitutes high-precision maps and how this information should be represented. Nevertheless, high-precision maps available on the market share common functionalities. Centimeter-level positional accuracy and the availability of lane-level geometric and semantic information are fundamental features in most high-precision maps.

At a basic level, high-precision maps can simply be a set of points and line segments whose accurate positions represent road signs, lane markings, lane boundaries, and lane separators. Due to the requirements of AD systems, today’s high-precision maps are becoming increasingly complex, with data from different sources constituting multi-layered information about the driving environment. Breaking down high-precision maps into multiple layers allows for a more structured data representation of the road environment.This aids the accessibility of different components of AD systems that require modeling the environment at different levels of detail. Furthermore, layered data representation makes the construction, storage, retrieval, and maintenance of maps easier. Figure 2 provides examples of these layers.

Understanding High-Precision Maps: Past, Present, and Future

As mentioned above, there are several ways to represent the map information used in AD systems, including lane-level details such as lane boundaries, lane marking types, traffic directions, driveable area polygons, and intersection annotations. Although the driving environment is highly dynamic, the data represented in these three layers is static. The overall representation of the environment should also include real-time traffic information about observed speeds, weather conditions, congestion areas, and road blockage areas (construction). This section attempts to provide a global overview of the information stored in these layers in a unified manner. While most high-precision map providers have their definitions and formats, and there is no unique standard for high-precision maps, we categorize the information contained in high-precision maps into six distinct layers, as shown in Figure 2.

A. Base Map Layer

The base map layer is the foundation of HD maps and serves as the reference layer for building all other layers. It contains a highly accurate three-dimensional geospatial representation of the environment, such as the locations and shapes of roads, buildings, and other structures. The three-dimensional geospatial model of the road environment is becoming an important source of information for autonomous vehicles. Currently, HD maps typically include a three-dimensional representation of the environment. The base map layer is usually created using point clouds from LiDAR and/or images from one or more cameras, sometimes assisted by GPS/IMU. This set of sensors constitutes a Mobile Mapping System (MMS) that can create a highly accurate and detailed representation of the environment in 3D point clouds. Geometric and semantic features of roads and lanes are extracted from this layer to construct other layers in HD maps. Since this layer contains a dense data representation of the environment, it plays a crucial role in the precise localization of autonomous vehicles. Several techniques for point cloud registration allow for estimating vehicle poses by matching raw sensor data with the point cloud of this layer. In terms of data processing and communication requirements, constructing and updating this layer is challenging.

B. Geometric Map Layer

Although the base map layer provides a precise and dense representation of the environment, its ability to support understanding of the environment is limited due to a lack of meaningful features in its representation. The geometric layer in HD maps provides detailed information about the geometric shapes of the road environment, including the locations and shapes of roads, lanes, curbs, and other features. The geometric layer typically includes information about road widths, the number of lanes, center lines for each lane, lane boundaries within each road, and pavement elevations. It also includes precise locations and shapes of curbs, sidewalks, crosswalks, and vertical and horizontal traffic signs. Each of these features is represented using basic geometric primitives, namely points, lines, polylines, and polygons. For instance, the position of a vertical traffic sign can be represented as a point. The center line or boundary of a lane can be represented by a set of interconnected line segments, such as a polyline. Similarly, crosswalks can be represented as polygons. The geometric features of this layer are created by processing the data from the base map layer. Constructing the geometric layer based on the base map data typically involves several processing steps, including road segmentation, lane information extraction, traffic signs, poles, traffic signals, curbs, obstacles, and pavement features. The geometric features in HD maps are crucial for various core components of AD, especially for accurate motion prediction of dynamic road participants and safe planning of geometrically feasible trajectories.

C. Semantic Map Layer

The semantic map layer defines the significance of the road features provided by the geometric map layer. The data in this layer provides context and meaning to the features represented in the map. For example, the semantic map layer in HD maps contains information such as road types (e.g., highways, residential roads) and lanes (e.g., whether they allow left or right turns), their numbering, traffic directions, and whether a lane is for turning or stopping. It also includes information about speed limits, lane boundaries, intersections, crosswalks, traffic signs, traffic signals, parking spaces, bus stops, and many other features that are important for constructing contextual representations of the environment. The semantic map layer allows automated vehicles to construct detailed contextual representations of their environment and understand traffic rules, enabling them to make correct and safe decisions in various traffic scenarios. Simply put, the semantic map layer assigns semantic labels to the road features and objects defined in the geometric map. For instance, points in the geometric layer are merely a set of ordered coordinates in the map coordinate reference system. Only the semantic layer defines whether a point corresponds to a traffic light, yield sign, or stop sign. It is well-known that high-precision maps contain rich semantic information. The semantic layer also associates metadata with road features, such as road curvature, recommended driving speeds, and unique identifiers for each semantic feature. In fact, semantically rich high-precision maps enable automated vehicles to better understand driving situations, thus making complex decisions in intricate scenarios. However, constructing a reliable and high-fidelity semantic map of the road environment is not a simple process. Several processing steps are required, including scene segmentation, object detection, classification, pose estimation, and mapping. With recent advancements in computer vision, deep learning, sensor fusion, and semantic SLAM algorithms, constructing accurate semantic maps has become feasible.

D. Road Connectivity Layer

The road connectivity layer describes the topological structure of the road network and how various geometric elements are connected. In contrast to the standard definition of digital maps that only contain road-level information and road-level connections, high-precision maps include lane-level geometric and semantic information, making the connections between roads complex, as it defines the connections between two or more sets of lanes. More accurately, this layer provides the layout and connectivity of roads, including lane boundaries and center lines as well as intersections. Lane-level connectivity information is necessary for planning legal transitions between roads and lanes, as well as planning allowed maneuvers at each intersection, which is crucial for path planning of autonomous vehicles. Simply put, this layer defines how the geometric elements that constitute the geometric layer are interconnected. These connections are established by defining ordered pairs of geometric and semantic elements. Assigning a unique identifier to each geometric and semantic element can represent this information using a graph data structure, where each element is represented by an edge and its connection as nodes. Graph structures allow for quick querying and searching of maps and efficient route planning.

E. Prior Map Layer

This layer is also referred to as the prior map layer, as it represents and learns information from past experiences. It involves geometric and semantic elements in the map whose states change over time. Learning the state of traffic flows and accident areas from fleet data allows for more effective and predictive driving behavior. This layer also acquires and learns information that helps predict human driving behavior and the dynamic states of intersection traffic lights. It also adapts to temporary road setups, such as parking commands, occupancy rates, and schedules. For example, roadside parking availability may change on certain weekdays in some cities, predicting the probability of occupancy, and managing time rules for a given parking spot is derived from previous sensor readings of different fleet vehicles passing by that parking spot. Learning and predicting the driving behaviors of road users can be challenging due to sociocultural differences among various societies. Modeling these behaviors based on experience is crucial for universal and scalable AD systems.

F. Real-Time Map Data

The real-time layer in high-precision maps is a dynamic layer that provides real-time information about the environment, such as traffic conditions, road closures, and other events that may affect the navigation of autonomous vehicles. This layer is usually created by combining data from various sources, such as cameras, sensors, and other connected devices installed on vehicles or located at the roadside. Real-time data is collected through crowd-sourcing from participating vehicles or using smart infrastructure with specific communication networks to update high-precision maps. The real-time layer can include information about the positions and speeds of other vehicles, the status of traffic signals, and whether there are construction zones or other obstacles and blockages on the road. This information is crucial for autonomous vehicles to make safe and efficient driving decisions in real-time to optimize traffic flow and reduce congestion. Moreover, the real-time layer can enhance the accuracy and completeness of HD maps by providing the latest information about the environment that may not have been captured by the sensors used to create the maps. In simple terms, the real-time layer in high-precision maps provides a dynamic and up-to-date representation of the environment. Real-time updates of dynamic elements in high-precision maps are challenging and require complex intelligent communication infrastructures and cooperation among multiple participants. Data transmission between Intelligent Transportation Systems (ITS), high-precision map providers, and vehicles must be reliable and meet certain requirements discussed later in this survey.

▍04 High-Precision Maps in Autonomous Driving System Architecture

High-precision maps provide a detailed and precise representation of the road environment for AD systems. These maps contain lane-level geometric, topological, and semantic information necessary for autonomous vehicles to navigate safely and efficiently. The use of high-precision maps in autonomous vehicles allows them to better understand their surroundings, plan routes, and make more accurate driving decisions, ensuring the safety of passengers and other road users. This section discusses the significance and use of high-precision maps in AD systems. Ultra-precise map data is now a component of most different core components in AD systems. To discuss the importance and use of high-precision maps in AD, we briefly introduce the architecture and standard components of a typical modern AD system. Figure 5 illustrates the standard components of an AD system, showcasing those components that rely on HD maps. This section first briefly introduces the architecture of the AD system and how it works along with its various components. The remainder of this section extensively reviews the latest technologies for AD components that depend on high-precision maps.

A. Autonomous Driving System Architecture

Automated vehicles are complex cyber-physical systems where different components must work together to complete the overall driving task robustly, reliably, and safely. While there is no unique architecture for AD systems, we rely on a generic architecture in this work that helps us understand how to use high-precision maps to enhance different functionalities of AD systems.

Similar to any robotic system, autonomous vehicles can be viewed as cognitive agents with three main components: (1) sensors, (2) perception, and (3) behavior. Breaking these elements down into an industry-level AD system results in several components, as shown in Figure 5.

The sensing components in modern AD system architectures typically include different sensors like IMUs, GPS, cameras, LiDAR, and radar. A subset of these sensors allows the vehicle to know its position relative to the environment, which is used for localization, while the remaining sensors are used for perceiving the environment itself. The role of the sensing component is to read and preprocess raw sensor data and provide it to other parts of the AD system. In its simplest form, the sensing component consists of a set of sensor drivers that read raw sensor data in real-time.

The localization component is one of the most critical parts for the reliable operation of the entire AD system. Its role is to accurately estimate the vehicle’s position. Errors in localization propagate to the rest of the AD processing pipeline. Localization is merely a state estimator that fuses raw sensor data from the sensing component. Moreover, the availability of maps allows for improved and robust localization, especially in areas where some sensors may fail or degrade in performance.

The role of perception is to generate an intermediate-level representation of the current state of the environment, including information about obstacles and road factors. This representation also includes detailed information about lanes (position, boundaries, markings, and types), traffic signs, traffic signals, and drivable areas. Computer vision and deep learning techniques are widely used for segmentation, clustering, and classification tasks. Additionally, object-level fusion is also an important part of this component. The output of perception is a list of tracked objects and a semantic segmentation of images for scene understanding. Geometric and semantic information from HD maps can also be used to improve object detection and fusion. Accurate perception is critical for safety, as perception errors can affect the quality of information used throughout the AD system. Therefore, using redundant sources of sensor data can enhance confidence in perception accuracy, thereby improving the robustness of the entire system.

The scene understanding component serves as a bridge between the intermediate-level state representation of the environment provided by the perception component and the high-level cognitive components of the AD system. This component aims to provide a higher-level contextual understanding of the driving scene by integrating high-precision maps and data provided by the perception component. In the later sections of this paper, we will discuss how to fuse these two information sources to build scene representations for understanding the driving environment. Another component in the AD pipeline that relies on high-precision maps is the motion prediction component. It builds on the high-level spatiotemporal representation of the environment provided by scene understanding to predict the behaviors of road agents surrounding the vehicle.

The role of HD maps in motion prediction is to provide the previous trajectories of each road agent in the scene. Motion prediction is a highly multimodal problem in which HD maps play a critical role, and this section will discuss it in detail.

The motion planning component aims to compute feasible, collision-free, and safe trajectories for the autonomous vehicle. This is achieved by optimizing the globally shortest path obtained from routing algorithms running on HD map data and the predicted trajectories of road agents. Motion planning also includes behavioral planning functionalities that rely on the state of the current scene defined by detected objects and HD maps.

The control component receives planned trajectories and computes control commands for steering, braking, and accelerating the drive system. The control component does not explicitly rely on map data, so it will not be considered in this survey.

Finally, as shown in Figure 5, a special component serves all other components by processing requests that provide map data.

High-precision map data is typically stored in a database queried by map servers (local or cloud) to route, tile, and update map clients in vehicles. Since routing elements require special algorithmic processing, we will consider it in the investigation of HD map applications in AD systems.

B. Localization

The localization component in AD systems is designed to estimate the vehicle’s position and orientation relative to a global reference coordinate system. Its key role is to maintain the high precision and robustness of the estimates required by the continuous components in the system. The accuracy of localization algorithms determines the reliability of the entire AD system. Robustness of localization under adverse weather conditions is a key requirement for modern AD systems, as degraded estimation performance can lead to serious consequences and potential harm. Over the past two decades, significant research efforts have been made to achieve localization, resulting in a wide variety of methods. To ensure normal operating conditions and achieve global system safety, autonomous vehicles need localization within 10-centimeter accuracy.

C. Perception

The perception component in AD systems is typically associated with processing raw camera images and LiDAR point clouds, not only for detecting and tracking static targets (such as traffic signs and road markings) but also for detecting and tracking dynamic obstacles, such as surrounding vehicles, pedestrians, and cyclists. Perception is one of the core functionalities of AD systems. Ensuring its reliability and real-time performance is crucial for ensuring collision-free navigation. Integrating perception data with the detailed and precise geometric and semantic information contained in the layers of high-precision maps can potentially improve perception by focusing on the most relevant Regions of Interest (ROIs). More accurately, the geometric shapes of HD maps allow for defining ROIs to filter point clouds, leaving only the point clouds that the perception function is particularly interested in, thus simplifying and improving the computational efficiency of non-urgent detections.

D. Scene Understanding

Understanding the driving environment is vital for autonomous vehicles to make correct and safe decisions. One of the early motivations for high-precision maps was to provide accurate and detailed information to assist autonomous vehicles in understanding their environment. This information enables AD systems to comprehend the current driving situation and interpret all entities that constitute the scene. The geometric structures and semantics contained in the map facilitate the systematic construction of compact data models and environmental representations, allowing vehicles to handle complex driving scenarios. More accurately, the scene understanding component in AD systems, supported by the geometric and semantic information of HD maps, can consistently provide meaningful perceptual context. Beyond raw object detection, scene understanding aims to extract and estimate critical safety information and make it available for subsequent processing stages.

E. Routing

Road-level digital maps assist human drivers in navigation. The route calculations in these maps cannot exceed the bounds of using road-level connections, as these maps do not include lane-level details. Accurate and optimized driving routes are necessary to save time and energy and enhance global vehicle safety. Efficient and low-cost driving route calculations must consider the lane-level models of the environment. Moreover, in highly dynamic environments, when autonomous vehicles navigate through the environment, detailed information about traffic conditions and lane occupancy is crucial for dynamically adjusting routes. Considering the detailed and accurate lane-level information in the static layers of high-precision maps, as well as the prior and real-time layers, effective dynamic route calculations become possible. For the routing subsystem in autonomous vehicles to compute a drivable path from the current location to the set destination, the system must obtain the latest maps from the HD map server, as shown in Figure 5. Alternatively, similar to digital maps, route calculations can also be provided as a service. After sending its accurate location to the HD map server, the best route can be computed and fed back to the vehicle to support other core components of the system. In recent years, these routing services have involved considering real-time traffic conditions and energy factors (e.g., the most energy-efficient routing). For autonomous vehicles, other factors can also be considered, such as routes that avoid complex urban environments that are difficult for ADS to navigate or routes with good network coverage to ensure continuous connectivity for online services, including real-time high-precision map services.

F. Motion Planning

The role of motion planning in AD systems is to generate feasible, safe, collision-free, and energy-efficient trajectories. Motion planning tasks typically include trajectory generation and behavioral planning. Behavioral planning is a high-level decision-making function used to determine transitions between different driving states, such as lane changes, following vehicles within a lane, decelerating, and stopping. To safely perform these transitions, the behavioral planner requires local maps and vehicle perception to establish a transition model of the vehicle’s environment. Unlike navigation in mobile robotics, road environments are highly structured, and all road users must adhere to traffic rules. The generated AD trajectories are strictly required to ensure compliance with traffic rules, and motion must occur within drivable road areas. Various methods exist for motion planning in autonomous vehicles, all of which rely to some extent on the geometric and semantic information provided by high-precision maps to comply with traffic regulations. In sample-based motion planning methods, the lane geometries of HD maps are used to limit the search space by rejecting infeasible candidate trajectories.

G. Motion Prediction

The driving environment is highly dynamic and involves different road participants, such as pedestrians, vehicles, and cyclists. Predicting the future movements and behaviors of these road participants is crucial for autonomous vehicles to build context-aware representations of their interactive environments, thereby predicting potential hazardous situations. From an abstract perspective, these traffic participants can be viewed as a complex multi-agent system. In fact, developing reliable solutions for motion and behavior prediction of road agents will enhance the safety and capability of autonomous vehicles to adapt to human-like behaviors in real-world traffic conditions. Predicting the behaviors of these traffic participants is crucial for AD systems, primarily for risk assessment and safe and comfortable motion planning. Motion prediction refers to estimating the future behaviors based on the current states of the road agents and the model of their surrounding environment. Various research works have addressed the problem of predicting the future movements of road participants.

H. Third-Party Applications

High-precision maps can provide accurate and reliable ground truth (GT) data that can serve as a reference for calibrating sensor outputs. For example, LiDAR can be calibrated using the high-precision coordinates of geometric elements in high-precision maps, achieving perfect alignment with IMUs. By comparing sensor measurements with HD map data, any errors or discrepancies can be identified and corrected, thereby improving sensor calibration. Furthermore, high-precision maps can be used for online (self) calibration. The availability of high-precision maps, real-time raw sensor data, and algorithms for performing comparisons allows for the calculation of errors between sensor measurements and GT in real-time. This enables continuous correction of sensor calibration errors in real-time. Consequently, AD systems can become more robust and reliable in adapting to changing environmental conditions and variations in sensor performance. Compared to offline calibration methods, online calibration can achieve more accurate and robust sensor calibration. Recently, high-precision maps have also been used to enhance road annotations to create large datasets for traffic landmark detection.

▍05 Building High-Precision Maps

A. Mobile Mapping Systems

Building high-precision maps is a complex process that requires several steps. The first step in the high-precision map building program is to dispatch specialized vehicles equipped with a set of high-precision and well-calibrated sensors to survey and collect data about the environment. The data collection vehicles used for mapping may be equipped with connections to correction services with RTK (Real-Time Kinematic) positioning accuracy of up to a few centimeters or achieve high-precision GNSS. GNSS positioning measurements are typically fused with high-performance IMU (Inertial Measurement Unit) measurements and wheel odometry. Several commercial products integrate global navigation satellite systems and IMUs into a single unit as an inertial navigation system. Mapping vehicles are also equipped with one or more high-resolution LiDARs and cameras to collect raw 3D/2D data of the road environment. There are two approaches to setting up the data collection tools used for mapping. The first is to purchase the aforementioned sensors, select a suitable configuration, and install them in the vehicle. While this method provides flexibility in predefined sensor configurations, calibrating several different sensors to the accuracy required for mapping is not trivial and time-consuming, especially for cameras. Alternatively, some manufacturers provide a complete set of sensors in one package, referred to as a Mobile Mapping System (MMS). Examples of commercial MMS are shown in Figure 6.

▍06 High-Precision Map Maintenance

Having the latest HD maps is crucial for the proper functioning of various core components in AD. Errors in HD maps can lead to severe damage due to incorrect decisions made by the system. Frequent updates by mapping vehicles can prevent erroneous decisions. Due to new infrastructure construction, road maintenance, and lane expansions, the road environment is highly dynamic and may change frequently. Mapping vehicles must be able to detect changes in the environment and send them to update the maps. The map update process involves complex processing steps, including handling data from multiple sources and sensors of varying scales, identifying deviations between the stored maps and newly collected environmental data, and ultimately integrating these deviations to update the different layers of the map. Several methods and approaches have been developed in the literature to capture changes in high-precision maps and update them. Below, we review different methods and approaches for detecting changes in high-precision maps and how this information can be applied to update the maps. We investigate the approaches previously followed for maintaining high-precision maps based on analyzing the layers maintained by each state-of-the-art method, as shown in Table 3.

A. Change Detection in Maps

Change detection in high-precision maps refers to the process of identifying changes in the environment, such as new buildings, road closures, etc. Subsequently, the respective layers of the map are updated accordingly. High-precision maps undergo regular changes, and having a map that autonomous vehicles can trust is crucial for ensuring safe navigation. Change detection is typically achieved using various sensors, such as cameras, LiDAR, and radar, and combined with computer vision algorithms and machine learning techniques. Even before high-precision maps emerged, change detection algorithms had found applications in many scenarios. Remote sensing was one of the early applications for map change detection and updating. It has also been successfully applied in urban monitoring, forest change detection, crisis monitoring, 3D geographic information updates, construction progress monitoring, and resource surveys. At the most basic level of these applications, the problem equates to comparing raw sensor data, primarily 3D point clouds, 2D images, or a combination of both.

B. Map Data Updating

The second phase of HD map maintenance involves updating map elements based on the results of change detection. In simple terms, map updating equates to a probabilistic data fusion problem. Continuously monitoring changes in a dynamically changing environment in near real-time and fusing different data patterns in time and space, as well as updating multiple layers from different sources, is indeed a challenging task. Cadena et al. pointed out in their survey paper that the distributed process of updating and maintaining high-precision maps created and used by large fleets of autonomous vehicles is a compelling topic for future research. In this direction, Kim et al. proposed a solution to keep the new feature layers updated from crowdsourced point cloud data. This new feature layer forms the basis for constructing various semantic and geometric features of HD maps.

▍07 Data and Communication Infrastructure for High-Precision Maps

Building and maintaining high-precision maps at scale is a data exchange issue among multiple stakeholders, such as governments as owners of ITS roadside infrastructure, map providers, and vehicles, as shown in Figure 7. The collection, construction, maintenance, and distribution of map data require reliable communication and distributed computing infrastructure. This section discusses the data and communication infrastructure required to extend the creation, maintenance, and distribution of high-precision maps.

▍08 Challenges and Future Outlook

Despite significant advancements in CCAM over the past decade, achieving full autonomy for vehicles remains an unresolved challenge. Scalable solutions for high-precision maps are crucial for autonomous vehicles to be deployed at scale. In this section, we elucidate various challenges that need to be addressed to fully realize the potential of high-precision maps in CCAM. Undoubtedly, the availability of cost-effective and flexible solutions for building, maintaining, and distributing map data among stakeholders will greatly enhance the scalability of CCAM in the future generations of smart cities. Additionally, we discuss the future prospects and applications of high-precision maps.

A. Challenges

– Standardization and Data Representation: The concept of high-precision maps has been widely accepted as a key technology in CCAM. However, there is no consensus on how to represent map data, how many layers are needed, what map data must be stored in each layer, and in what data format the map data should be stored. Due to the complexity of high-precision maps and the vast amount of data and information they contain, defining a universal standard for them is challenging, thus creating a comprehensive, understandable, and effective standard for storing, maintaining, updating, and distributing maps is a challenge. Defining a universal standard for map data will provide greater data compatibility, facilitate data access, while lowering development and integration costs. Furthermore, this will enhance the quality, consistency, and privacy of data, thereby improving road safety for all participants, including automated vehicles.

– Scalability: Scalable high-precision map solutions are essential for the large-scale deployment of autonomous vehicles. Building and maintaining high-precision maps across cities, regions, and nationwide remains a significant challenge, especially when dealing with different standards used to represent geometric road features and traffic signs, which vary by region. Mapping algorithms must be universal and capable of functioning across different regions and countries. Mapping should be a continuous data collection and processing process to fix modified areas. This process becomes challenging in large geographic areas where a large number of vehicles must be part of the mapping process. The cost of mapping directly depends on how large the area to be mapped is and how many vehicles are needed to service it. As discussed earlier in this article, mapping vehicles are quite expensive. Furthermore, using separate vehicles equipped with consumer-grade sensors requires complex algorithms that are not yet mature.

–Network and Computing Infrastructure: In the case of building and updating scalable high-precision maps, processing and handling large amounts of data require reliable network and computing infrastructure that should be coordinated and close to real-time. With the advent of 5G/6G cellular communications, the Internet of Things (IoT), and edge computing architectures, many opportunities for in-vehicle communication are widely available, making processing solutions for building high-precision maps commercially viable. These communication and computing infrastructures are designed to handle such data-intensive applications and meet their latency and bandwidth requirements. Large-scale crowdsourced maps with a vast number of connected vehicles will be one of the primary applications of these infrastructures.– Limitations of Mapping Algorithms: Despite significant R&D efforts in automating the process of building high-precision maps, recent research results on high-precision maps clearly indicate that the mapping algorithms used to extract features for high-precision maps and build road and lane topologies remain limited to simple features. Current state-of-the-art algorithms can detect simple geometric features but struggle with high-curvature features such as roundabouts. Moreover, most of these methods require several post-processing steps to obtain features in a suitable vector format. The semantic features of maps remain limited to a few easily detectable traffic signs. Recently, there has been little work addressing the problem of building lane topologies to create simplified road/lane connectivity networks. Developing a universal mapping pipeline that can build a fully functional high-precision map containing geometric, semantic, and topological information remains a challenge.– Ownership, Privacy, Integrity, and Distribution of Map Data: The future of building and maintaining high-precision maps will involve the automation and distribution of millions of vehicles. Collecting, processing, and storing large amounts of distributed data raises concerns about data ownership, privacy, integrity, and distribution. The raw map data is generated within vehicles and aggregated with other data sources from public authorities, processed, and distributed by map providers. Ownership of map data from collection to distribution may require resolution in large-scale high-precision maps. Additionally, protecting the privacy of individuals and vehicles is crucial and must be considered during the mapping process. Map data may include sensitive user information, such as the precise location of vehicles and detailed descriptions of the vehicle’s environment. Ensuring the integrity of high-precision map data is essential to avoid errors and fatal decisions, especially when used by autonomous vehicles. Building accurate and reliable high-precision maps remains an ongoing research issue. Commercial high-precision maps typically undergo manual checks and verification. Generating accurate and reliable high-precision map data from multiple data sources, such as crowdsourcing, presents several technical issues that need to be resolved. The ownership, privacy, and integrity of scalable high-precision maps have recently begun to attract the attention of researchers. On the other hand, blockchain has proven to be a promising solution for ensuring data integrity due to its distributed nature and security. The use case of building and updating scalable high-precision maps while maintaining data traceability, privacy, and integrity is a perfect application for blockchain. This technology is expected to play a central role in building and distributing the next generation of high-precision maps.

B. Future Directions

– Precision: Precise localization has always been one of the primary motivations for introducing high-precision maps into autonomous vehicles. The existence of dense and compact representations of the road environment forms the basis of high-precision maps, especially in localization. There has always been a trade-off between the information density contained in HD maps and the computational workload required to process this information. Recent advancements in neural 3D scene representations have made it possible to reconstruct realistic 3D scenes in a very compact representation. Using Neural Radiance Fields (NeRF) to represent the base map layer allows for benefiting from the compact and realistic representation of this layer. This technology may usher autonomous vehicles into a new era.

– Applications Beyond Autonomous Driving: The development of high-precision maps was primarily aimed at assisting autonomous vehicles in understanding their environment and navigating safely within it. However, due to the detailed and precise environmental representations provided by high-precision maps, they can also be used to enhance the quality of various services offered by traditional digital maps. Furthermore, high-precision maps play an important role in digital assistive technologies for individuals with disabilities. If visually impaired individuals are equipped with appropriate sensors and can obtain highly precise, detailed, and semantically rich environmental representations, their mobility and safety can be significantly improved. With precise localization, digital assistive devices can interpret and understand the environment, generating voice navigation messages for safe navigation. The real-time status of traffic lights and other traffic information in pedestrian high-precision maps relates to enhancing the functionalities of these devices. Currently, most high-precision map providers only offer maps representing the vehicle environment. Today’s high-precision maps still do not depict routes for participants other than vehicles, such as sidewalks for pedestrians and bike lanes. Constructing and updating high-precision maps for all participants will pave the way for widespread autonomous and non-autonomous navigation, as well as some useful digital services.

– Towards Digital Twins: Digital twins of the environment represent a comprehensive digital representation of the environment, including all its physical and functional characteristics. City-scale digital twins are an emerging concept in CCAM aimed at constructing a data-driven model that integrates data from IoT sensors, connected vehicles, buildings, smart infrastructure, traffic networks, and all other data sources to help create a comprehensive, real-time urban model to improve road services. This concept encapsulates high-precision maps as digital models of connected and autonomous vehicles to support various functionalities and services in smart cities. Digital twins can even be used to model the behaviors of different entities in the environment, even at micro-detail levels. High-precision maps will be a single module of digital twins, supporting various functionalities and services of connected and autonomous vehicles in our smart cities. Since high-precision maps can be used to simulate complex driving scenarios, digital twins will be utilized to simulate and analyze the impacts of new development projects or changes in traffic patterns, helping urban planners and decision-makers analyze and optimize city performance by predicting future scenarios and identifying opportunities for improvement. Building a city-scale digital twin is indeed a significant challenge that requires vast amounts of data and can be a complex and time-consuming process. Cross-validation, integrity, and credibility of distributed large-scale data remain challenges in creating digital twins. Crowdsourced road mapping by vehicles will be replaced by a unified process of outdoor and indoor mapping using large amounts of data from heterogeneous connected sensors.

▍09 Conclusion

High-precision maps remain a rapidly evolving aspect of real-world CCAM applications, driving innovation and progress in the field. Despite extensive research and development efforts in the applications of high-precision maps in AD systems and the algorithms and infrastructure for building and maintaining high-precision maps, there is little literature summarizing and providing a foothold for these works. This article extensively reviews past works on building and maintaining high-precision maps, including cost-effective solutions and the communication and map data requirements from generation to distribution. Furthermore, this article discusses the current challenges faced in building and maintaining high-precision maps. More accurately, we provide an independent overview of high-precision maps as a backdrop for the broader intelligent transportation system community. We also discuss and analyze the latest technologies used in various core functionalities of AD systems utilizing high-precision maps. Additionally, we extensively discuss and review different methods, approaches, and algorithms for building different layers of high-precision maps and keeping them up to date. Finally, we provide some insights into the future development of high-precision maps for next-generation mobile applications.

↓↓ Learn More About Industry News ↓↓

Set me asStarred★

Source: Beijing High-Level Autonomous Driving Demonstration Zone(Click “Read the Original” at the end) Edited by: Li Juan Reviewed by: Yu Qing

Column:Geographic Information Industry Welcomes New Year

>>New Year Message from the China Geographic Information Industry Association 2024

>>Looking Back at 2023 | Top Ten Highlights of China’s Geographic Information Industry in 2023

>> Association Notice

Notice on the Ranking of the Top 100 Enterprises, Most Dynamic SMEs, and Most Growth-Oriented Enterprises in the Geographic Information Industry in 2024
Notice on Hosting a Series of Training Courses on Real-Scene 3D City Construction and Applications (Qingdao Station)
Notice on the Construction of a Geographic Information Industry Investment and Financing Information Platform
Notice on Collecting the Group Standards of the China Geographic Information Industry Association for 2024 (First Batch)
Notice on the Application for the 2024 Geographic Information Science and Technology Progress Award
Notice on the Application for Excellent Projects in the Geographic Information Industry in 2024
Notice on the Evaluation of Geographic Information Science and Technology Achievements
Recruitment Announcement of the China Geographic Information Industry Association in 2024
Call for Contributions to the “Voice of Innovation” Themed Promotion Activity by the China Geographic Information Industry Association
Notice on Learning and Implementing the Opinions of the Ministry of Natural Resources on Accelerating the Transformation and Upgrading of the Surveying and Geographic Information Industry to Better Support High-Quality Development

Notice on Paying Membership Fees

Notice on Organizing Attention to the Association’s Public Account, Designated Correspondents and Contributions to the Public Account

>> Recent News

The Ministry of Natural Resources: The General Implementation Plan for the Construction of the New Generation Geographic Information Public Service Platform (Tianditu) Issued
18 Departments Including the Ministry of Industry and Information Technology: Jointly Deploying the “Together to Benefit Enterprises” Service Action for Small and Micro Enterprises
Voice of Innovation | Aerospace Hongtu Wang Yuxiang: Dreaming of the Stars “Perceiving” the Earth, Fully Layouting the Space-Earth Integrated Satellite Internet
Two Sessions Feature | Real-Scene 3D View of Representatives and Committee Members: Digital New Infrastructure Builds a New “Base” for Beautiful China
President Li Weisen and his team investigated geographic information companies in Kunming, Yunnan