“

Simulation is one of the most important tools in the process of robot learning and research, which refers to the technology of simulating actual physical systems through computers. Robot simulation is based on interactive computer graphics technology and robotics theory, generating geometric shapes of robots and displaying them in 3D, used to describe the dynamic changes of robots and their working environment.In this technical article, we invited Xiaomi engineer—— Xie Yan, focusing on the robot simulation platform as the main discussion topic, introducing the necessity of simulation to evaluation metrics, combined with specific physics engines and simulation software, providing a multi-layered introduction to robot simulation platforms.

”

1. Necessity of Simulation

Why is simulation necessary? This may be the first question for most beginners. The traditional product development process usually requires going through product definition -> concept design -> program review -> detailed design -> prototype trial production -> testing -> program optimization, etc. However, as the complexity of products increases, especially for robots, which are multi-degree-of-freedom complex systems, the disadvantages of directly adopting the above processes become increasingly significant.

First, when problems are found during testing, it is necessary to iterate and optimize the program; especially for hardware design, processing, and assembly, the process is time-consuming and labor-intensive. Once problems arise during this process, the iteration cycle can easily be prolonged.

Figure 1 Traditional Development Process

Secondly, without simulation, software development work can easily lag behind. Although the software structure can still be designed while waiting for hardware finalization, the lack of effective feedback means that the development during this process can easily become disconnected from the actual system. By the time testing on the actual system is conducted, some components may even need to be rebuilt, wasting a portion of the previous workload.

To improve product development efficiency, many product developments now adopt a simulation-driven development process. After the program review phase, the program is evaluated through simulation, iteratively optimizing unreasonable parts of the program before proceeding to detailed design. This can shorten development cycles, reduce costs, and minimize design risks.

Figure 2 Simulation-Driven Development Process

Finally, many individuals and teams do not have the conditions to build hardware platforms. For some industrial robotic arms, prices can reach hundreds of thousands or even higher, which is unaffordable for some robotics enthusiasts or beginners. Simulation can provide a low-cost learning environment where users can easily build their robot hardware in a simulated environment to implement and test their software algorithms.

2. Introduction to Simulation

Simulation refers to the technology of simulating actual physical systems through computers. For robots, robot simulation is based on interactive computer graphics technology and robotics theory, generating geometric shapes of robots and displaying them in 3D, used to describe the dynamic changes of robots and their working environment.

Figure 3 Composition of Simulation Platform

A complete robot simulation platform is mainly responsible for three things: model modeling, dynamics calculation, and visualization display. These correspond to the three components of the simulation platform: model description, dynamics engine, rendering engine.

▍Model Description

It is mainly responsible for modeling the robot model, using computer language to describe how the robot is composed to the simulation platform. The basic composition of a robot can be described using links and joints. In addition, it is necessary to know some physical information about these basic components, such as the mass properties of the link, inertia properties, color, and the type of joint (whether it is a rotational joint or a translational joint).

Currently, the industry mainly uses URDF (Unified Robot Description Format) and SDF (Simulation Description Format) files to describe robot models, both of which are XML syntax frameworks used to describe robots. Although URDF is the standard format in ROS, it lacks many functions to describe more complex scenarios. For example, URDF can only specify the kinematic and dynamic characteristics of a single robot; it cannot specify friction characteristics and handle closed-chain structures. Additionally, it cannot specify environmental information such as lighting, height maps, etc. The introduction of SDF aims to solve the shortcomings of URDF, allowing for a complete description of all aspects of the robot, with greater scalability suitable for complex simulations.

▍Dynamics Engine

It can calculate motion, rotation, and collision based on the physical properties of objects. The basis of the dynamics engine is system dynamics, which is a mathematical description of the laws of motion of objects in the real world, primarily based on ordinary differential equations, and the core task of the physics engine is to solve these ordinary differential equations.

Commonly used robot simulation physics engines include ODE, Bullet, MuJoCo, etc. Bullet and ODE were originally developed for 3D movies and games. Therefore, the numerical solver uses the first-order semi-implicit Euler method, primarily pursuing speed and stability rather than accuracy. MuJoCo is a recently open-sourced physics engine aimed at promoting research and development in robotics, biomechanics, graphics and animation, and other fields that require fast and accurate simulation. MuJoCo provides a unique combination of speed, accuracy, and modeling capabilities, mainly applied in reinforcement learning and optimal control.

Figure 4 Physics Engine Workflow

▍Rendering Engine

It uses computer graphics technology to render the 3D graphics of the robot based on the robot model and texture, allowing us to observe the correctness of the robot’s operation through the visualization platform. Currently, the robot rendering engine mainly uses OpenGL.

OpenGL (Open Graphics Library) is a cross-language, cross-platform application programming interface for rendering 2D and 3D vector graphics. This interface consists of nearly 350 different function calls, used to draw complex three-dimensional scenes from simple graphic bits. OpenGL is commonly used in CAD, virtual reality, scientific visualization programs, and video game development, with strong versatility and portability.

Figure 5 Rendering Engine Workflow

3. Simulation Evaluation Metrics

To objectively evaluate the quality of simulation software, unified standards need to be established. Early researchers identified the common standards for evaluating robot simulation platforms: realism, scalability, ease of development, and cost. These four standards can be used to judge any robot simulation software. However, with the development of simulation software and the richness of application scenarios, the evaluation criteria for simulation software may vary slightly in different development environments.

Specifically, for simulation of legged robots, their structure is a multi-degree-of-freedom open or closed-chain structure, which makes their dynamic model more complex than that of general wheeled robots, and during robot movement, real-time multi-point contact with the environment is required, which places high performance requirements on the simulator in terms of multi-object collision detection. Using precise mathematical models to describe dynamic models and collision models is certainly good, but it undoubtedly increases the burden on the solver. Therefore, the simulator needs to reasonably balance between computational accuracy and computational speed. In addition, the simulator also needs to consider factors such as scalability, maintainability, and compatibility, which are crucial for the development, deployment, and later maintenance of developers’ projects. The table below lists some more comprehensive evaluation metrics.

Table 1 Simulation Evaluation Metrics

Evaluation Metrics	Metric Description
Computational Speed	Efficiency of simulation computation
Computational Accuracy	The degree of closeness between simulation results and theoretical values
Contact Model	Accuracy of the contact model, ensuring that dynamics simulation results are close to reality
Programming Environment	Supported programming languages
Compatibility	Supported operating systems
Scalability	Supported third-party tools, including matrix computation, numerical optimization, data analysis, etc.
Open Source	Degree of source code openness, degree of API interface openness
Cost	Including financial expenditure and time expenditure

4. Introduction to Physics Engines

There are two common types of physics engines: high-precision physics engines and real-time physics engines.

High-precision physics engines require more processing power to calculate very precise physics, focusing on accurate calculations, with high precision in simulation results, typically used in scientific research (computational physics) and computer animation film production, such as ADAMS, ANSYS, etc.

Real-time physics engines are usually used in the field of video games and simplify computations, reducing precision to decrease computation time, thus achieving acceptable processing speeds in video games, such as Unreal Engine, Bullet, etc. The following will introduce and analyze several physics engines currently widely used in robotics and machine learning.

>>>> RaiSim

Introduction: RaiSim[1] is a cross-platform multi-body physics engine developed by RaiSim Tech Inc. for robotics and AI, designed to provide accuracy and speed for simulating robotic systems. However, it is a general rigid body simulator that can effectively simulate any rigid body. It is closed source and fully supports Linux, Mac OS, and Windows systems, and users need to purchase a commercial license to use it, while academic licenses are free.

Features: Supports the following functions——unique contact solving methods, efficient multi-body dynamics algorithms, calculating the forward dynamics of the entire system with O(n) complexity, the easiest to learn/use C++ simulation library, and minimal dependencies (only relying on STL and Eigen).

>>>> Bullet Physics (Bullet)

Introduction: Bullet[2] physics engine is an open-source, professional dynamics library that integrates rigid body, soft body, and collision detection. Developed by Erwin Coumans, it was originally designed for game development but is now widely used in robotics, virtual reality, computer graphics, and other fields. Bullet3 supports various physical simulations, including rigid bodies, soft bodies, vehicles, water, gas, etc.

Features: Open-source powerhouse, with discrete and continuous collision detection for rigid and soft body simulations; collision shapes include: spheres, boxes, cylinders, cones, convex hulls using GJK, non-convex meshes, and triangle meshes; supports soft body simulations: cloth, ropes, and deformable objects; provides rich rigid body and soft body constraints; features velocity-level MLCP and simplified coordinate Featherstone ABA algorithms; rich plugins: supports plugins for Maya, Softimage, can be integrated into Houdini, Cinema 4D, LightWave 3D, Blender, Godot, and Poser; Collada 1.4 physics content import; directed optimization for PlayStation 3 Cell SPU, CUDA, and OpenCL.

>>>> Open Dynamics Engine (ODE)

Introduction: ODE[3] (Open Dynamics Engine, an open-source dynamics engine) is a well-known free library for simulating joint-connected rigid body dynamics, developed by Russell Smith with the help of some contributors. Verified application scenarios include simulating ground vehicles, legged robots, and mobile objects in virtual display environments. It is characterized by being fast, flexible, robust, and has built-in collision detection capabilities.

Features: Supports rigid bodies with arbitrary mass distribution; joint types: spherical, revolute, prismatic, fixed, rotary motors, linear motors, universal joints; collision primitives: spheres, boxes, cylinders, capsules, planes, and triangle meshes, convex bodies; collision spaces: quadtrees, hash spaces, and simple spaces; simulation methods: derived motion equations based on the Lagrange multiplier velocity model of Trinkle/Stuwart and Anitescu/Potra; uses first-order integrators, fast but not accurate enough, with higher-order integrators to be released later; time step method selection: can use standard “big matrix” methods or newer iterative QuickStep methods; contact and friction models: based on Baraff’s Dantzig LCP solver, although ODE implements a faster approximation for the Coulomb friction model; open native C interface (though ODE is mostly written in C++), and C++ interface.

>>>> Multi-Joint Dynamics with Contact (MuJoCo)

Introduction: MuJoCo[4] is a physics engine developed by Emo Todorov for Roboti LLC, aimed at promoting research and development in robotics, biomechanics, graphics, animation, and other fields that require fast and accurate simulation. It provides a unique combination of speed, accuracy, and modeling capabilities, but it is not just a better simulator. Instead, it is the first full-featured simulator designed from scratch, aimed at model-based optimization, especially optimization conducted through contact. MuJoCo makes it possible to scale computationally intensive techniques (such as optimal control, physically consistent state estimation, system identification, and automated mechanism design) and apply them to complex dynamic systems with rich contact behavior. It also has more traditional applications, such as testing and validating control schemes before deployment on physical robots, interactive scientific visualization, virtual environments, animation, and games.

Features: Simulations in generalized coordinates to avoid joint conflicts; can define inverse dynamics well even with contact; unified continuous-time representation of constraints through convex optimization; constraints include soft contacts, limits, dry friction, equality constraints; can simulate particle systems, cloth, ropes, and soft bodies; includes actuators for motors, cylinders, muscles, tendons, and slider-crank; optional Newton, conjugate gradient, or projected Gauss-Seidel solvers; optional pyramid or elliptical friction cones, dense or sparse Jacobian equations; optional Euler or Runge-Kutta numerical integrators; multi-threaded sampling and finite difference approximation; intuitive XML model format (called MJCF) and built-in model compiler; cross-platform GUI with interactive 3D visualization in OpenGL; runtime modules written in ANSI C and manually optimized for performance.

>>>> Dynamic Animation and Robotics Toolkit (DART)

Introduction: DART[5] (Dynamic Animation and Robotics Toolkit) is a collaborative, cross-platform open-source library created by the graphics lab and humanoid robot lab at Georgia Tech. This library provides data structures and algorithms for kinematics and dynamics applications in robotics and computer animation. Because DART uses generalized coordinates to represent articulated rigid body systems and Featherstone’s articulated body algorithm to calculate motion dynamics, its accuracy and stability have been significantly improved. DART has a wide range of applications in robotics and computer animation, as it includes a multi-body dynamics simulator and various kinematic tools for control and motion planning.

Features: Supports multiple platforms: Ubuntu, Archlinux, FreeBSD, macOS, and Windows; provides an extensible API to solve various optimization problems, such as nonlinear programming and multi-objective optimization.

Supports various collision detectors: FCL, Bullet, and ODE; supports various collision shapes, including primitive shapes, concave meshes, and probabilistic voxel grids; through customizable inertia and material properties, supports numerous primitive and arbitrary body shapes; provides comprehensive access to motion states of arbitrary entities and coordinate systems (e.g., transformation, position, velocity, or acceleration).

Plug-and-play hierarchical whole-body inverse kinematics solvers; uses Lie group representations and Featherstone hybrid algorithms for articulated dynamic systems to achieve high performance; provides comprehensive APIs for dynamic quantities and their derivatives, such as mass matrix, Coriolis forces, gravity, other external and internal forces; uses implicit LCP to handle contact and collision, ensuring no penetration, directional friction, and approximate Coulomb friction cone conditions; provides multiple constraint solvers: Lemke method, Dantzig method, and PSG method; supports “island” techniques to subdivide constraint handling for improved performance.

The basic characteristics of these five physics engines can be summarized in the table below[6]:

Table 2 Basic Features of Five Physics Engines

RaiSim	Bullet	ODE	MuJoCo	DART
Initial release	Unreleased	2006	2001	2015	2012
Author	J. Hwangbo D. Kang	E. Coumans	R. Smith	E. Todorov	J. Lee et al
License	Proprietary	Zlib (open-source)	GPL / BSD (open-source)	Apache (open-source)	BSD
Main purpose	Robotics	Game, Graphics	Game, Graphics	Robotics	Robotics
Language	C++	C / C++	C++	C	C++
API	C++	C++ / Python	C	C	C++
Contacts	Hard	Hard/Soft	Hard/Soft	Soft	Hard
Solver	Bisection	MLCP	LCP	Newton / PGS / CG	LCP
Integrator	Semi-implicit Euler	Semi-implicit Euler	Semi-implicit Euler	Semi-implicit Euler / RK4	Semi-implicit Euler

To evaluate the performance of each physics engine more fairly, we introduce the speed-accuracy curve. As shown in the figure below, the ideal physics engine is located at the upper right corner, while most physics engines are represented as curves sloping down from left to right. Better physics engine curves are located higher up; thus, the physics engine represented by curve 1 is superior to that represented by curve 2.

Figure 6 Speed-Accuracy Curve

The table below describes the performance results of each physics engine based on the speed-accuracy curve evaluation[7].

Table 3 Performance Results of Five Physics Engines

RaiSim	Bullet	ODE	MuJoCo	DART
friction model test	++	+++	–	+	–
single-body elastic collision test	++++	++	+++	–	+
single-body hard contact test	+++	+	++	+	+
single-body energy test	++++	++	+++	–	+
articulated-robot-system speed test for quadrupedal robot	+++++	+++	+	++++	++
articulated-robot-system momentum test	++++	+++	++	+++++ (RK4)	+
+++ (Euler)

• +: Good simulation result, more + indicates better performance

• -: Unable to simulate due to inaccurate or anomalous models

From Table 3, it can be seen that ODE generally handles objects of single primitive shapes well, while MuJoCo and DART have advantages in multi-body simulations. The multi-body system simulation of ODE is very time-consuming, but it is worth noting that it performs excellently in the articulated-robot-system momentum test: despite its obvious inefficiency, it has good momentum retention capability using maximum coordinate representation. Bullet can accurately and efficiently simulate the sliding of objects with frictional contact. RaiSim, with its unique contact solving methods and efficient multi-body dynamics algorithms, typically performs well in both single-body and multi-body tasks.

However, these engines also have their limitations:

• ODE and DART’s LCP contact models cannot simulate the sliding of objects under certain conditions;
• DART is not suitable for multi-object simulation scenarios;
• Due to the lack of a solver error correction mechanism, Bullet suffers from significant position-level drift;
• MuJoCo’s soft contact model cannot control contact elasticity.

5. Introduction to Simulation Software

Robot system design cannot be separated from the support of simulation tools. Robot simulation allows us to quickly validate algorithms or improve safety without physical hardware, avoiding experiments that could damage our equipment (for example, in reinforcement learning, a lot of random exploration is required). Generally speaking, robot simulation tools are built on top of physics engines, such as those based on ODE, Bullet, etc. In some cases, we only need to use the physics engine to meet our needs, but generally, we also want to observe the correctness of the robot’s operation through a visualization platform. Below, we will introduce and analyze some well-known simulation platforms in the field of robot simulation.

>>>> Webots

Webots[8] is a user-friendly robot simulator released by the Swiss company Cyberbotics. It can run on Windows, Linux, or macOS and supports most programming languages, such as C/C++, Java, URBI, MATLAB, and Python. The Webots simulator can also be externally controlled using an API or any standard TCP/IP network. The Webots simulator uses ODE to simulate physics in a realistic 3D environment. It also supports accurate modeling of collisions and contact points, allowing virtual robots to test various scenarios. Webots even comes with a robot model library that includes PR2, Atlas, Aibo, and some NASA robots. Webots also supports VRML97, so robot designs can be imported using most modeling software (Solidworks, AutoCAD, Blender, and Inventor). It can also import maps and terrain data to create accurate world representations in the testing environment.

However, Webots also has many drawbacks. First, the modeling process in Webots is relatively cumbersome, and it lacks robust support for closed-chain structure robots, making it cumbersome to simulate parallel robots. Secondly, the data from the Webots simulation process cannot be automatically saved, and the visualization is not very well done, making it inconvenient to use. Finally, the Webots console does not support Chinese, the software interface’s Chinese translation is incomplete, and the help documentation does not have a Chinese version.

Figure 7 Webots Simulator Interface

Image Source:https://blog.csdn.net/ZhangRelay

>>>> V-rep

V-REP[9] (now updated and named CoppeliaSim) is free for educational purposes, and if planning to use it for commercial projects, it can be upgraded to a professional version. V-REP can run on Windows, Linux, or macOS and can be programmed using six different programming languages. It also supports ROS & BlueZero, allowing the use of the 3D simulator and running code on another platform.

V-REP supports four different physics engines (Bullet, ODE, Newton, and Vortex Dynamics). It can accurately handle object interactions, such as collisions, contact points, and grasping. V-REP also supports particle physics to simulate air and water, allowing accurate modeling of jet engines and propellers. V-REP can also simulate various sensors and even add new sensors to the simulator through plugins.V-REP even supports path and motion planning, and can calculate both forward and inverse kinematics.It also supports mesh editing, allowing small adjustments to designs within the simulator, or easily importing models in any of five different 3D model formats (Obj, STL, URDF, etc.).

Figure 8 V-REP Simulator Interface

Image Source:https://www.coppeliarobotics.com/coppeliaSim_v-rep_iros2013.pdf

>>>> Gazebo

Gazebo[10] simulator is a free open-source platform for designing, developing, testing, and visualizing almost any type of robot. Gazebo runs on Linux, Windows, and Mac, and is the default simulator for ROS, allowing the invocation of various open-source algorithms and direct deployment to real machines. Gazebo supports various high-performance physics engines, such as ODE, Bullet, Simbody, and DART. The Gazebo simulator also comes with some robot models, such as PR2, DX, Irobot Create, and TurtleBot, so you can quickly get started even without your own robot models. It also supports a wide range of sensors and can simulate noise and sensor failures to accurately simulate real-world problems. Gazebo can even run in the cloud and interact with the simulator using a simple web browser.

Figure 9 Gazebo System Architecture

Image Source:https://www.mdpi.com/2075-1702/7/2/42

The model format in Gazebo is based on XML SDF, and building a model requires users to write XML files themselves, which can be challenging for beginners. Secondly, Gazebo’s support for closed-chain structure robots is not robust, and Gazebo’s high integration with ROS makes it difficult for users unfamiliar with ROS to get started.

Figure 10 Gazebo Simulator Interface

Image Source:https://alternativeto.net/software/gazebo-robot-simulator/about/

>>>> ISAAC

NVIDIA ISAAC[11] platform is a simulation platform developed by NVIDIA, a leading graphics card manufacturer, providing developers with a powerful end-to-end platform for developing, simulating, and deploying AI-supported robots. It can run on GPU-enabled desktops or in the cloud on AWS, Google Cloud, or Azure. It also supports NVIDIA Jetson AGX Xavier, Jetson TX2, and Jetson Nano. Since the ISAAC robotics platform is manufactured by NVIDIA, it offers the best graphics among all robot simulation platforms.

ISAAC is optimized for NVIDIA hardware (GPU), allowing for efficient platform-optimized cross-compiled applications. ISAAC integrates many CUDA-accelerated open-source libraries, such as NPP, OpenCV, ROS, PCL (wip), Eigen, etc. The ISAAC SDK comes with a series of high-performance algorithms known as GEM, in addition to algorithms for robot control, ISAAC also provides deep neural network module integration for those who play games and “refine elixirs” as a side job, such as stereo depth estimation, object detection, tracking algorithms, etc. ISAAC integrates the latest PhysX and RTX engines for physical simulation and image simulation, and introduces Pixar’s USD (Universal Scene Description) format as a way to describe robots and complex scenes.

Figure 11 ISAAC Simulator

Image Source:https://docs.nvidia.com/isaac/doc/overview.htmlhttps://analyticsindiamag.com/nvidia-releases-developer-toolbox-for-ai-powered-robotic-working-system/

>>>> Drake

Drake[12] is a C++ toolbox developed by the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory (CSAIL) Robot Locomotion Group. With the development of the core development team led by the Toyota Research Institute, the team has significantly expanded. It is a collection of tools for analyzing robot dynamics and building robot control systems, emphasizing optimization-based design/analysis.[13]

Drake is designed to simulate very complex robot dynamics (e.g., friction, contact, aerodynamics, etc.), while always emphasizing revealing structural characteristics (sparsity, analytical gradients, polynomial structure, uncertainty quantification, etc.) in control equations and using this information for advanced planning, control, and analysis algorithms.

Drake has three main components: 1) multi-body dynamics engine; 2) a “system framework” for organizing and combining system models from the library into block diagrams; 3) an optimization framework for mathematical programming. Each of these components is important for robotics research.

Figure 12 Drake System Framework

Image Source:https://medium.com/toyotaresearch/drake-model-based-design-in-the-age-of-robotics-and-machine-learning-59938c985515

Each of these components is crucial for robotics research, and Drake’s vision is to promote the powerful combination of these three components in a single application. Drake supports symbolic computation, which is very useful for optimization problems. Drake also provides a series of sensor models, actuator models, low-level controllers, and low-level perception algorithms. Drake provides a Python interface for rapid validation of new algorithms while also working to provide reliable open-source implementations for many state-of-the-art algorithms.

Figure 13 Drake Simulator

Image Source:https://www.youtube.com/watch?v=_8af5NgbKVc

>>>> MATLAB

MATLAB is a commercial mathematical software produced by MathWorks, USA. It is a versatile scientific computing platform that integrates many powerful functions such as algorithm development, data analysis, and matrix computation into an easy-to-operate window environment.

The Simulink[14] under MATLAB is considered capable of “simulating any system.” Due to the powerful capabilities of MATLAB/Simulink, it has been widely used in the field of robotics. The built-in Simscape toolbox can be used to design and analyze three-dimensional rigid mechanical structures (such as automotive platforms, robotic arms, etc.). In addition, it can model multi-domain systems using electrical, hydraulic, or pneumatic components and other components. The Robotics System Toolbox allows you to connect directly to sensors via ROS and perform calibration, denoising, and other operations on the returned information.

At the same time, for machine vision, Mathwork has developed a separate toolbox called the Computer Vision Toolbox. It supports calibration of monocular, stereo, and fisheye cameras, provides algorithms for object detection and tracking, feature extraction and matching, and can perform 3D reconstruction and 3D point cloud processing based on data. Simulink provides a series of embedded hardware support, preset hardware packages for Arduino and Raspberry Pi, supporting microcontrollers, PLCs, FPGAs, GPUs, and various devices, not only for simulation but also for direct hardware control. Simulink’s programming environment is diverse, supporting C/C++, and can connect directly to ROS systems. MATLAB Coder and Simulink Coder can automatically generate control codes based on the models you create, greatly reducing workload.

Figure 14 MATLAB/Simulink Simscape Simulator

Image Source: https://www.mathworks.com/content/dam/mathworks/mathworks-dot-com/solutions/automotive/files/jp-expo-2015/mbd-for-robotics-development.pdf

https://ww2.mathworks.cn/videos/modeling-and-simulation-of-walking-robots-1576560207573.html

>>>> PyBullet

PyBullet[15] is a simulation environment developed based on the Bullet physics engine and is a strong competitor to Gazebo. Developed by Erwin Coumans and other contributors, it allows users to control the Bullet3 physics engine using the Python programming language and perform physical simulations.

PyBullet is closely integrated with Python and is widely used in reinforcement learning (RL). This environment can combine with TensorFlow to implement RL training, such as DQN, PPO, TRPO, DDPG, etc.[16] Currently, it is commonly seen simulating multi-joint robots.

PyBullet is a fast and easy-to-use Python module for robot simulation and machine learning, focusing on Sim-to-Real conversion; using PyBullet, you can load articulated bodies from URDF, SDF, MJCF, and other file formats; PyBullet provides forward dynamics simulation, inverse dynamics computation, forward and inverse kinematics, collision detection, and ray intersection queries; the Bullet Physics SDK includes PyBullet robot examples, such as the simulated quadruped robot Minitaur, simulated humans running using TensorFlow for decision-making, and KUKA grasping objects; simplified coordinate multi-body, rigid body, and deformable bodies are handled by a unified LCP constraint solver.

In addition to physical simulation, it also has rendering bindings, including CPU renderers (TinyRenderer) and OpenGL visualization, and supports virtual reality such as HTC Vive and Oculus Rift; PyBullet also has the ability to perform collision detection queries (nearest points, overlaps, ray intersection tests, etc.) and add debugging rendering (debug lines and text); PyBullet has built-in client-server capabilities that are cross-platform, supporting shared memory, UDP, and TCP networks, and can run PyBullet on Linux connected to a Windows VR server; PyBullet wraps the new Bullet C-API, which is independent of the underlying physics engine and rendering engine, allowing us to easily migrate to newer versions of Bullet or use different physics engines or rendering engines.

Figure 15 PyBullet Simulator

Image Source: https://www.youtube.com/watch?v=aiWxIjtMMFI

https://twitter.com/erwincoumans/status/1002601540111118337

>>>> Mujoco

MuJoCo, full name Multi-Joint dynamics with Contact, is a physics engine for simulating multi-joint movements in contact with the surrounding environment, applicable in fields such as robotics actions, biomechanics, graphics and animation, and machine learning. Acquired and open-sourced by DeepMind in 2021, it can run on Linux, Windows, and Mac.

MuJoCo combines generalized coordinate simulation with optimized contact dynamics[17], allowing it to simulate complete physical motion. Even more stunning is MuJoCo’s ability to simulate the complex movements of human joints and muscles. Many robotic hand studies are first simulated and validated in MuJoCo. MuJoCo can flexibly break down simulation steps or execute only parts of the simulation process (e.g., not calculating inverse dynamics). It also supports the stability simulation of soft materials like ropes and fabrics.

To improve simulation performance, MuJoCo has made numerous optimizations, such as AVX instructions, and is one of the few modern physics engines that choose C language for implementation. The use of C language allows it to be easily converted to other architectures. Models are defined using the XML-based MJCF scene description language, compiled using a dedicated optimization compiler. In addition to MJCF, the engine also supports uploading files in the Unified Robot Description Format (URDF). MuJoCo also provides a graphical interface for interactive 3D visualization of the simulation process and results representation using OpenGL.

Figure 16 MuJoCo Simulator

Image Source: https://www.deepmind.com/blog/opening-up-a-physics-simulator-for-robotics

Figure 17 MuJoCo Simulator 2

Image Source: https://www.youtube.com/watch?v=3dlM1dvBtko

6. Conclusion

Simulation can effectively help improve development and testing efficiency in complex robot projects, allowing developers to identify as many errors as possible before physical testing and deployment, facilitating reproduction and resolution of issues after system problems arise, and also aiding in coordinating work progress between different small teams (such as software and hardware groups) within the same project. However, there is no free lunch; building and maintaining simulations also require additional workload, so development teams need to assess and choose suitable simulation platforms based on project needs to maximize benefits/costs.

References:

[1] https://raisim.com/

[2] https://pybullet.org/wordpress/

[3] http://www.ode.org/

[4] https://mujoco.org/

[5] https://dartsim.github.io/

[6] Erez, Tom, Yuval Tassa, and Emanuel Todorov. “Simulation tools for model-based robotics: Comparison of bullet, havok, mujoco, ode and physx.” 2015 IEEE international conference on robotics and automation (ICRA). IEEE, 2015.

[7] https://leggedrobotics.github.io/SimBenchmark/

[8] https://cyberbotics.com/

[9] https://www.coppeliarobotics.com/

[10] https://gazebosim.org/

[11] https://www.nvidia.com/en-us/deep-learning-ai/industries/robotics/

[12] https://drake.mit.edu/

[13] Posa, Michael, Scott Kuindersma, and Russ Tedrake. “Optimization and stabilization of trajectories for constrained dynamical systems.” 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2016.

[14] https://www.mathworks.com/products/simulink.html

[15] https://pybullet.org/wordpress/

[16] Mower, Christopher, et al. “ROS-PyBullet Interface: A framework for reliable contact simulation and human-robot interaction.” Conference on Robot Learning. PMLR, 2023.

[17] Todorov, Emanuel. “Convex and analytically-invertible dynamics with contacts and constraints: Theory and implementation in mujoco.” 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2014.