This article provides an overview of the current mainstream simulation software for autonomous driving. The content is extensive, and interested readers can bookmark it for later as it is filled with valuable information.
The existing simulation software mainly includes CarSim, CarMaker, PreScan, PTV Vissim, SUMO, VIRES VTD, rFpro, Cognata, 51Sim-One, Pilot-D GaiA, Metamoto, Tencent TAD Sim simulation platform, Baidu Apollo, AirSim, NVIDIA Drive Constellation, Waymo Carcraft, PanoSim, etc.
In the future, we will also introduce specific usage of some simulation software. Friends interested can follow me! Creating content is not easy, so remember to give a thumbs up!
1. CarSim
CarSim, along with related TruckSim and BikeSim, is a powerful dynamics simulation software developed by Mechanical Simulation Corporation, widely used by OEMs and suppliers worldwide. CarSim is targeted at four-wheeled vehicles, while TruckSim is focused on multi-axle and dual-tire trucks, and BikeSim is for two-wheeled motorcycles. CarSim is a vehicle dynamics simulation software that mainly simulates from a complete vehicle perspective. It has a considerable number of vehicle mathematical models, all of which come with rich empirical parameters, allowing users to use them quickly without the complex modeling and parameter tuning process. CarSim models run on computers at speeds up to 10 times faster than real-time, simulating the vehicle's response to driver control, 3D road surfaces, and aerodynamic inputs, with simulation results closely approximating real vehicles. It is mainly used to predict and simulate the handling stability, braking, ride comfort, power, and economy of the entire vehicle. CarSim comes with a standard Matlab/Simulink interface, enabling convenient co-simulation with Matlab/Simulink for control algorithm development, while generating a large amount of data results for subsequent analysis or visualization using Matlab or Excel. CarSim also provides an RT version that supports mainstream HIL testing systems, such as dSpace and NI systems, facilitating HIL simulations.
CarSim also supports ADAS-related functions, allowing the construction of parametric road models with over 200 moving traffic objects, using scripts or external control through Simulink to manage their movements, while adding up to 99 sensors to detect moving and stationary objects. The latest version of CarSim has enhanced capabilities in ADAS and autonomous driving development, adding more 3D resources such as traffic signs and pedestrians, as well as high-precision map import processes. Additionally, CarSim provides a plugin for the Unreal Engine, enabling co-simulation with Unreal Engine.
2. CarMaker
CarMaker, along with related TruckMaker and MotorcycleMaker, is a dynamics, ADAS, and autonomous driving simulation software launched by the German company IPG. CarMaker is primarily an excellent dynamics simulation software that provides accurate vehicle body models (engine, chassis, suspension, transmission, steering, etc.). In addition, CarMaker has built a closed-loop simulation system that includes vehicles, drivers, roads, and traffic environments. IPG Road can simulate various forms of roads, including multi-lane and intersections, and can generate cone-shaped and cylindrical obstacles through the configuration GUI. It allows arbitrary definitions of road geometry and surface conditions (roughness, roughness). IPG Traffic is a traffic environment simulation tool that provides a rich array of traffic object models (vehicles, pedestrians, road signs, traffic lights, construction sites, etc.), enabling real traffic environment simulations. Test vehicles can recognize traffic objects and trigger actions accordingly (e.g., speed limit signs can trigger vehicles to slow down). IPG Driver is an advanced, self-learning driver model that can control vehicles under various driving conditions, performing operations such as starting uphill, parking, and counter-steering. It can adapt to the vehicle's power characteristics (driving form, transmission type, etc.), road friction coefficients, wind speed, and traffic environment conditions to adjust driving strategies. As a platform software, CarMaker can integrate with many third-party software, such as ADAMS, AVLCruise, rFpro, etc., leveraging the advantages of each software for co-simulation. Meanwhile, the hardware that accompanies CarMaker provides a large number of board interfaces, allowing easy HIL testing with ECUs or sensors.
3. PreScan
PreScan is an ADAS testing simulation software developed by Tass International, acquired by Siemens in August 2017. PreScan is a simulation platform consisting of a GUI-based preprocessor for defining scenes and an execution environment for running scenes. Engineers use MATLAB and Simulink as the main interface for creating and testing algorithms. PreScan can be used for applications ranging from Model-in-the-Loop (MIL) control design to real-time testing using Software-in-the-Loop (SIL) and Hardware-in-the-Loop (HIL) systems. PreScan can operate in open-loop, closed-loop, and offline and online modes. It is an open software platform with a flexible interface that can connect to third-party vehicle dynamics models (e.g., CarSIM and dSPACE ASM) and third-party HIL simulators/hardware (e.g., ETAS, dSPACE, and Vector). PreScan consists of multiple modules, and its usage is mainly divided into four steps: building the scene, adding sensors, adding control systems, and running simulations. Scene building: PreScan provides a powerful graphical editor that allows users to construct rich simulation scenes using road segments, including traffic signs, trees, and buildings, as well as a library of traffic participants like vehicles, bicycles, and pedestrians, modifying weather conditions (e.g., rain, snow, and fog) and light sources (e.g., sunlight, headlights, and streetlights). The new version of PreScan also supports importing high-precision maps in OpenDrive format to create more realistic scenes. Adding sensors: PreScan supports a wide variety of sensors, including ideal sensors, V2X sensors, LiDAR, millimeter-wave radar, ultrasonic radar, monocular and binocular cameras, and fisheye cameras. Users can add them according to their requirements. Adding control systems: Control models can be established through MATLAB/Simulink, or closed-loop control can be performed with third-party vehicle dynamics models (like CarSim, VI-Grade, dSpace ASM). Running experiments: The 3D visualization viewer allows users to analyze experimental results and provides image and animation generation capabilities. Furthermore, the interface with ControlDesk and LabView can be used to automate the running of experimental batches and hardware-in-the-loop simulations.
4. PTV Vissim
Vissim is a world-leading microscopic traffic flow simulation software provided by the German company PTV. Vissim can easily construct various complex traffic environments, including highways, large roundabouts, parking lots, etc., and can simulate interactions between motor vehicles, trucks, rail traffic, and pedestrians within a simulation scene. It is an effective tool for planning and evaluating urban and suburban traffic facilities, and it can also simulate the impact of localized emergency situations and the evacuation of large numbers of pedestrians. Vissim's simulations can achieve high accuracy, including microscopic individual following behavior and lane-changing behavior, as well as group cooperation and conflicts. Vissim has built-in multiple analysis methods, enabling the acquisition of various specific data results under different conditions, and providing intuitive understanding from a high-quality 3D visualization engine.
5. SUMO
SUMO is an open-source microscopic continuous traffic flow simulation software developed by the German Aerospace Center. It comes with a traffic simulation road network editor that allows users to add roads, edit lane connections, handle intersection areas, and edit traffic light timing interactively. It can also convert road networks from Vissim, OpenStreetMap, and OpenDrive using a separate conversion program. Users can specify routes for each vehicle through editing routing files or use parameters for random generation. During runtime, it can handle continuous traffic simulation demands for several square kilometers with tens of thousands of vehicles simultaneously and provides a real-time display of traffic simulation results based on OpenGL visualization. Additionally, SUMO offers convenient C++ and Matlab interfaces for flexible integration with third-party simulation programs. SUMO is primarily used for traffic flow, timing, prediction, and other simulations in the traffic field, and has recently begun to be applied in autonomous driving simulations, providing random complex dynamic environments for autonomous driving algorithms.
6. VIRES VTD
VTD (Virtual Test Drive) is a complete modular simulation toolchain developed by the German company VIRES for ADAS, active safety, and autonomous driving. VIRES was acquired by MSC Software Group in 2017. VTD currently runs on the Linux platform, covering road environment modeling, traffic scene modeling, weather and environmental simulation, simple and physically realistic sensor simulation, scene simulation management, and high-precision real-time rendering. It supports the entire development process from SIL to HIL and VIL, and its open modular framework facilitates co-simulation with third-party tools and plugins. VIRES is also a major contributor to the widely used open formats for autonomous driving simulation, including OpenDrive, OpenCRG, and OpenScenario, and VTD's functionality and storage rely on these open formats. The simulation process of VTD mainly consists of three steps: road network construction, dynamic scene configuration, and simulation operation.1) VTD provides a graphical interactive road network editor (ROD) that allows users to construct complex road simulation environments containing various traffic elements while simultaneously generating high-precision OpenDrive maps.
2) For dynamic scene establishment, VTD offers a graphical interactive scenario editor that allows users to add user-defined behavior-controlled traffic bodies or continuously running traffic flows based on OpenDrive.
3) Whether for SIL, HIL, real-time, or non-real-time simulations, whether in single-machine or high-performance computing environments, VTD provides corresponding solutions. During runtime, VTD can simulate real-time high-quality lighting effects, road reflections, vehicle rendering, rain, snow, fog weather rendering, sensor imaging rendering, and headlight visual effects.
7. rFpro
rFpro is a British company founded in 2008, initially as an internal track reconstruction and simulation project for an F1 team, which set high demands for speed, real-time performance, and accuracy from the beginning. rFPro uses high-precision phase-based laser radar scanning data to scan road surfaces and shoulders, generating high-resolution digital models of road surfaces with a resolution of 1cm. It also uses TOF laser radar to scan roadside streets and scenes, providing highly matched virtual scenes for dynamics simulation, ADAS, and autonomous driving testing. rFpro has created numerous high-precision virtual scenes of tracks and test environments, including F1, NASCAR, IndyCar, etc.In dynamic scene simulation, rFpro can integrate with SUMO or Vissim to generate continuous traffic flows to fill the entire scene, or it can co-simulate with CarMaker, providing more realistic sensor and road surface inputs for CarMaker’s test scenes. rFpro also offers a physically realistic lighting and weather system, effectively simulating changes in daylight and weather conditions such as rain and fog.
8. Cognata
Cognata is an Israeli autonomous driving simulation startup founded in 2016, which completed $18.5 million in Series B financing at the end of 2018. Cognata combines artificial intelligence, deep learning, and computer vision to recreate urban environments on its 3D simulation platform, providing customers with various simulated real-world test driving scenarios. Cognata's technology mainly consists of three aspects: in static environments, Cognata's TrueLife 3DMesh engine uses computer vision and deep learning algorithms to automatically generate virtual simulation environments, including buildings, roads, lane markings, and traffic signs based on maps and satellite images. In dynamic simulations, Cognata establishes accurate and scalable traffic simulation models and weather lighting models based on historical street traffic data, simulating various vehicles and pedestrians in real-world environments. The entire virtual simulation engine integrates static and dynamic simulation models, simulating the interactions between sensors and various changes in the simulated environment, providing a complete feedback loop for the autonomous driving systems under test.Cognata’s simulation technology is powered by NVIDIA DGX Station. In March 2019, Cognata announced a partnership with NVIDIA to leverage its powerful computing capabilities on NVIDIA’s platform to simulate multiple virtual vehicles for large-scale testing in virtual environments.
9. RightHook
RightHook is a startup based in California, USA, providing simulation solutions for the autonomous driving industry. RightHook offers a complete set of toolchains, including RightWorld, RightWorldHD, and RightWorldHIL. RightWorld provides a process for automatically reconstructing high-precision maps into richly detailed virtual scenes, while also offering an easy-to-use test case creation process. After case creation, the AI algorithm can organically expand the cases. RightWorld also includes deterministic intelligent traffic simulation models that involve vehicles, pedestrians, and bicycles. RightWorldHD simulates dynamics, weather, time changes, and sensors (including cameras, Lidar, Radar, IMU, and GPS), while supporting rich interfaces including NVIDIADriveWorks, LCM, and ROS. RightWorldHIL provides support for HIL testing that mixes software, algorithms, and hardware.
10. Parallel Domain
Parallel Domain is a startup founded in California in 2017. At the end of 2018, Parallel Domain received investment from Toyota. Parallel Domain focuses on the automatic generation of high-quality virtual environments. Its developed software can automatically generate the required testing city blocks in a short time. The Parallel Domain platform uses real-world map data, can accept various map formats, and uses additional elements where the maps do not provide sufficient data, relying on a procedural generation engine to automatically create virtual worlds. A notable feature is that all elements of the virtual world are adjustable and programmable, such as the number of lanes, terrain types, mountain locations, and road curvature. Parallel Domain also provides dynamic traffic scenarios for automatically generated scenes.
11. 51Sim-One
51Sim-One is an integrated autonomous driving simulation and testing platform developed by 51VR, encompassing multi-sensor simulation, traffic flow and agent simulation, perception and decision-making simulation, and autonomous driving behavior training. This simulation platform is based on physical characteristic modeling, featuring high precision and real-time simulation, used for the R&D, testing, and validation of autonomous driving products, allowing users to quickly accumulate autonomous driving experience, ensuring product performance safety and reliability, and improving product R&D speed while reducing development costs. In terms of scene construction, users can quickly create OpenDrive-based road networks from scratch through WorldEditor or restore road network information using real data such as point cloud data and map images. It supports importing existing OpenDrive format files for secondary editing, ultimately allowing 51Sim-One to automatically generate the required static scenes. Users can freely configure global traffic flows, independent traffic agents, competing vehicles, pedestrians, and other elements in the scene to build dynamic scenes, along with simulating lighting, weather, and other environmental conditions to present a rich and varied virtual world.
In terms of sensor simulation, 51Sim-One supports multi-channel simulation of general types or custom sensors, meeting the testing and training needs for perception system algorithms. For camera simulation, 51Sim-One provides annotated image datasets including semantic segmentation maps, depth maps, and 2D/3D bounding boxes, simulating monocular, wide-angle, and fisheye cameras. For radar simulation, it can provide raw LiDAR point cloud data, annotated point cloud data, bounding box data for recognized objects, and target-level millimeter-wave radar detection data.
12. Pilot-D GaiA
GaiA is an autonomous driving and ADAS development verification simulation tool developed by Peidai (Shanghai). It can restore complex roads by integrating road network databases and can recreate realistic driving environments using an environmental building model library. GaiA provides rich C++ and MATLAB interfaces, suitable for various vehicles and systems under test. GaiA can generate numerous traffic participants and manually or automatically set their traffic behavior planning, even changing the aggressiveness of driving behavior. GaiA also provides high-fidelity environmental perception sensors, including millimeter-wave radar, LiDAR, and cameras.
13. Metamoto
Metamoto, founded in 2016, is a Silicon Valley startup. Metamoto provides "Simulation as a Service" for autonomous driving companies, aiming to help them achieve iterative development through an accelerated feedback loop. Its products mainly consist of three parts: the designer, cloud platform, and analyzer. The designer can be used to add road networks, other environmental vehicles, pedestrians, and traffic lights to construct test scenes, generating multiple test cases by controlling the range of various parameters. The cloud platform is responsible for scheduling hardware resources based on the test case conditions, running test cases in parallel, and generating a large amount of test data. After completion, the analyzer can replay the simulation's sensor data and various simulation information of the vehicle to debug the autonomous driving system. Metamoto supports precise simulation of various sensors, including LiDAR, cameras, millimeter-wave radar, ultrasonic radar, GPS, IMU, etc., able to respond differently to various materials. One notable feature of Metamoto is that it provides a rapid method for adjusting and overriding test parameters, allowing for the execution of a large number of tests in a short time, effectively improving testing efficiency.
14. ESI Pro-Sivic
ESI Group's sensor simulation analysis solution Pro-SiVIC can help manufacturers in the transportation industry conduct virtual tests of the operational performance of various onboard or airborne perception systems, accurately reproducing factors such as lighting conditions, weather, and other road users. Pro-SiVIC can establish high-realism 3D scenes comparable to actual scenarios and perform real-time interactions within the scene for simulation analysis, reducing the need for physical prototypes. Clients can quickly and accurately simulate the performance of each embedded system under typical and extreme operating environments, offering sensor models based on various technologies, such as cameras, radar, LiDAR (laser scanners), ultrasonic sensors, GPS, odometers, and communication devices. For example, in the automotive industry, Pro-SiVIC provides multiple environmental directories that represent different roads (urban roads, highways, and rural roads), traffic signs, and lane markings.
15. NVIDIA Drive Constellation
NVIDIA Drive Constellation is an autonomous driving simulation platform launched by NVIDIA, primarily composed of two parts in hardware: one is a DGX server running the Drive Sim software system, relying on DGX's powerful graphical computing capabilities to realistically simulate lighting, nighttime, and various weather changes in actual environments, while the other server is equipped with a DRIVE AGX Pegasus onboard computer for running the full-stack algorithms of autonomous driving, forming a complete HIL simulation feedback loop.
16. PanoSim
PanoSim is a simulation software platform that integrates complex vehicle dynamics models, 3D driving environment models, vehicle driving traffic models, onboard environmental sensor models (cameras and radars), wireless communication models, GPS and digital map models, Matlab/Simulink simulation environment auto-generation, and graphical and animation post-processing tools. It is based on physical modeling and aims to balance precision and efficiency in numerical simulation, realistically simulating various environments and conditions for vehicle driving, supporting the R&D, testing, and validation of technologies and products in automotive dynamics and performance, automotive electronic control systems, intelligent driving assistance and active safety systems, environmental sensing and perception, and autonomous driving.PanoSim includes not only complex vehicle dynamics models, chassis (braking, steering, and suspension), tires, drivers, and powertrains (engines and transmissions) but also supports modeling and simulation analysis of various typical drive types and suspension forms for large, medium, and small passenger cars. It provides 3D digital virtual test scene modeling and editing capabilities, supporting modeling and editing of road and road textures, lane markings, traffic signs and facilities, weather, and nighttime conditions for vehicle driving environments.
17. AAI
AAI (Automotive Artificial Intelligence) is a startup founded in Berlin in 2017. AAI builds a set of complex high-fidelity virtual environments based on high-precision maps, integrating traffic participants into the virtual simulation environment using artificial intelligence technology, and utilizing driving behavior data from real life, training participant behaviors using machine learning algorithms, resulting in profiles for aggressive, mild, and defensive drivers. Its goal is to replicate the real world, realistically simulating all road users and environmental factors. AAI supports various sensor simulations and also provides analyzers for in-depth analysis of the data generated by simulations.
18. AirSim
AirSim is an open-source research project for drones and autonomous driving developed on the Unreal Engine by Microsoft Research. AirSim is implemented as a plugin for Unreal Engine, fully leveraging Unreal Engine's capabilities in creating highly realistic virtual environments, simulating shadows, reflections, and other real-world conditions, while also providing the ability to generate large amounts of annotated data in virtual environments, along with simple and convenient interfaces for drones and autonomous driving algorithms to conduct extensive training. AirSim's primary goal is to serve as a platform for AI research to test end-to-end reinforcement learning algorithms for deep learning, computer vision, and autonomous vehicles. The latest version of AirSim also offers a version for the Unity engine, adding support for LiDAR.
19. CARLA
CARLA is an open-source simulator developed under the guidance of the Computer Vision Center of the Autonomous University of Barcelona for the development, training, and validation of autonomous driving systems. Like AirSim, CARLA is also developed based on the Unreal Engine, using a server and multi-client architecture. In terms of scenes, CARLA provides open digital resources for creating scenes for autonomous driving (including city layouts, buildings, and vehicles) as well as several scenarios built from these resources for autonomous driving testing and training. At the same time, CARLA can also use VectorZero's road-building software RoadRunner to create scenes and accompanying high-precision maps, and it also offers a simple map editor. CARLA supports flexible configuration of sensors and environments, including multiple cameras, LiDAR, GPS, and can adjust lighting and weather conditions. CARLA provides simple automatic behavior simulation for vehicles and pedestrians and offers a complete set of Python interfaces to control vehicles, traffic lights, etc., facilitating co-simulation with autonomous driving systems for decision-making systems and end-to-end reinforcement learning training.
20. LGSVL Simulator
LGSVL Simulator is an open-source autonomous driving simulator developed by LG's Silicon Valley lab based on the Unity engine. It provides integration with the open-source autonomous driving platforms Autoware and Baidu Apollo. Users can annotate and export high-precision map formats matching autonomous driving systems based on 3D scenes within Unity. It also supports sensor simulations including LiDAR, millimeter-wave radar, GPS, IMU, and cameras, allowing simultaneous output of raw sensor results and ground truth.
21. Baidu Apollo
The Baidu Apollo simulation platform, as an important component of the Baidu Apollo platform, is designed to support internal Apollo system development and iteration, while also providing cloud-based decision system simulation services for developers in the Apollo ecosystem. The Apollo simulation platform is built on Baidu Cloud and Azure cloud services, allowing simulation testing in the cloud using user-specified Apollo versions. Apollo simulation scenes can be divided into Worldsim and Logsim. Worldsim consists of pre-set roads and obstacles, serving as efficient unit tests for autonomous vehicles, while Logsim is composed of scenes extracted from road test data, reflecting the complex and variable obstacles and traffic conditions in actual environments. The Apollo simulation platform also provides a comprehensive scene discrimination system that can evaluate autonomous driving algorithms from aspects such as traffic rules, dynamic behavior, and comfort.Apollo has also established a partnership with Unity to develop realistic virtual environment simulations based on Unity, offering 3D virtual environments, as well as variations in roads and weather. Recently, Baidu also proposed a new data-driven approach for end-to-end simulation in autonomous driving: Enhanced Autonomous Driving Simulation (AADS). This method uses simulated traffic flows to enhance real-world images, creating photo-realistic simulated scenes similar to real-world renderings. Specifically, it suggests using LiDAR and camera scans of street scenes, decomposing input data into background, scene lighting, and foreground objects. At the same time, it proposes a new view synthesis technique that can change the viewpoint on a static background, with foreground vehicles equipped with computer-generated 3D models. By accurately estimating outdoor lighting, it can reposition 3D vehicle models, computer-generated pedestrians, and other movable subjects, rendering them back onto the background images to create realistic street scene images. Furthermore, simulated traffic flows, placement and movement of synthesized objects, and capturing real-world vehicle trajectories can appear natural and capture the complexity and diversity of real-world scenes.
22. Waymo Carcraft
Representing the world-leading level of Waymo's driverless vehicles, a key secret is its Carcraft simulator, which is crucial for Waymo's driverless vehicles to travel billions of miles annually. At the beginning of Carcraft's development, this system was initially used for visual playback of roadside vehicles' situations on the road, later playing an increasingly important role. Carcraft can test each new software version using replay data from real-world driving, verifying algorithm improvements, discovering new issues, and constructing entirely new virtual scenes for testing. Every day, 25,000 virtual Waymo driverless vehicles travel over eight million miles in the simulator, reinforcing existing autonomous driving skills and testing new skills. The greatest advantage of simulation testing is the ability to quickly repeat tests of important scenarios that rarely occur in reality, such as at five-way intersections and merging roundabouts. The simulator allows the autonomous driving system many opportunities to practice these specific scenarios to master the corresponding skills. Moreover, within the simulator, changes can be made to specific participants or traffic signals in a particular test scenario, such as adding additional pedestrians, thus constructing a large number of derivative scenarios for more thorough testing of the driverless algorithms.
23. Tencent TAD Sim Simulation Platform
Efficient closed-loop verification is needed to build the core competitiveness of autonomous driving using data. Tencent’s autonomous driving virtual simulation platform TAD Sim is designed differently from traditional simulation systems, specifically developed for autonomous driving testing and verification, built with centimeter-level high-precision maps, creating a true digital twin system with dynamic and static elements, testing the completeness of autonomous driving algorithms in various scenarios.
This article is largely based on the 2019 Autonomous Driving Simulation Blue Book for academic dissemination, and if there are any infringements, please inform us.
There are many groups related to the automotive industry, including complete vehicles, key components, new energy vehicles, intelligent connected vehicles, aftermarket, automotive investment, autonomous driving, vehicle networking, etc. Please scan the administrator’s WeChat to join the group (please indicate your company name).There is also a startup financing group, welcome to angel round and Series A companies to join.