Click belowCard to follow the WeChat public account “New Machine Vision”
Important content delivered to you first
Author: Chen Bu Chen @ Zhihu
Editor: Dong An Yin Wei @ A Little Artificial Intelligence
Original: https://zhuanlan.zhihu.com/p/634848960
【Introduction】Many friends are very interested in robots, but do not know what parts make up a robot system and how it is implemented. Today, I will tell you how to design a robot from the perspective of a project, taking a remote-controlled robot as an example.
Remote control refers to the operator sending commands to the robot through an antenna or 5G. The robot uses various sensor fusion technologies, such as LiDAR, depth cameras, and ultrasonic sensors, to perceive the surrounding environment and sends the perception results to the main control unit through sensor fusion technology, enabling the robot to operate semi-autonomously or autonomously in an unknown environment.
This article aims to help everyone understand the technologies involved in a robot project and how they are implemented, without discussing specific technologies.
01 Components of a Robot System
We will take a mechanical dog as a carrier to elaborate. The technologies involved may not necessarily be present on the mechanical dog; this article uses the mechanical dog as a carrier to illustrate some technologies involved in unmanned equipment. In fact, the same applies to drones or unmanned vehicles.
The following diagram shows the system composition of a mechanical dog, listing some popular and major technologies, while some technologies are not listed, but they do not affect the design of a robot system.
A robot generally consists of three main components:
1. Mechanical structure design
2. Hardware circuit design
3. Control algorithms and perception decision-making algorithm design
This article starts from these three components to analyze some skills involved in robot design.
The process expressed in the system diagram of a robot is:
(1) The operator provides high-level commands by specifying the desired translational speed and steering rate.
(2) Upon receiving the high-level command, the CoM (Center of Mass) reference trajectory is generated and sent to the body and leg controllers.
(3) The controller uses the “Swing Leg Controller” if the legs are in the swing phase and the “Force Control Support Leg Controller” if the legs are in the support phase based on user input commands and the robot's state.
(4) Force and position commands are sent to the microcontroller to relay motor commands to each leg of the robot.
(5) If higher requirements are needed for the mechanical dog, such as autonomous navigation, algorithms like SLAM or AI can be run on the industrial computer.
02 Skills Involved in Robot Mechanical Design:
1) Dynamics;
2) Kinematics;
3) System Simulation
Software involved:
1) 3D design software (SolidWorks, UG, CATIA, etc.);
2) Dynamics and kinematics software (Adams, CoppeliaSim, etc.);
3) System simulation software (CoppeliaSim, Webots, etc.);
4) Finite element simulation software (Ansys or Abaqus, etc.)
When we design a robot body, we are usually concerned with two main issues: whether the body can withstand large loads (i.e., dynamics) and whether the body can move according to the trajectory I set (i.e., kinematics).
The first issue in dynamics is to consider the source of power, such as motors, hydraulic cylinders, electric cylinders, etc. For the mechanical dog, the first consideration is the power issue: whether the power is strong enough to meet the basic requirements of the mechanical dog. The following diagram shows the motor structure of the mechanical dog. You can easily guess the designer's intention; the designer integrated the motor, planetary gearbox, and the mechanical dog's casing into one unit, which reduces the weight of the mechanical dog, decreases inertia, and saves space, making it easier for the legs to perform various actions.
Once we have a basic design plan, we also need to verify the motor torque and analyze the force situation of the robot, which involves dynamics formulas such as Newton's laws, D'Alembert's principle, and Lagrangian dynamics equations. Often, one equation cannot solve all problems, so we need the cooperation of these equations. When using these formulas, we need to learn to simplify complex models for easier modeling. For example, the legs of the mechanical dog can be simplified as shown in the diagram below:
Next, let me give you an example to feel the role of different dynamics equations:
2.1 Example of D'Alembert's Equation
The total mass of the car and cargo is its center of mass distances from the front and rear wheels, respectively, and its height from the ground. When the car accelerates along a horizontal road, find the vertical reaction force on the front and rear wheels from the ground. The mass of the wheels is negligible.
D'Alembert's idea is that a previously unbalanced system can be treated as a static equilibrium system by introducing inertial forces.
The equations can be solved:
Without D'Alembert's principle, it would still be quite difficult to solve.
2.2 Example of Lagrangian Dynamics Equation
If all generalized coordinates are independent of each other, then the Lagrangian equation holds:
where L is the Lagrangian quantity, q is the generalized coordinates, t is a function of time, v is the generalized velocity, and F is the corresponding generalized force.
In mechanical modeling, the Lagrangian quantity is defined as kinetic energy minus potential energy, i.e.,
where T is the total kinetic energy in the system, and V is the total potential energy in the system.
Let me illustrate a common modeling process for an inverted pendulum. The inverted pendulum has a cart mass of m, a ball mass of m, and a rod length of l, with an external force acting on the cart:
First, analyze the generalized coordinates in the system, which are two: q1 and q2. The generalized coordinates mean that the selected variables can uniquely determine the state of the system, such as the position of the cart and the angle of the pendulum, which can determine the position of the system. Therefore, for the cart, we have:
For the rod, its horizontal and vertical displacements are x and y, respectively, so:
Thus, the Lagrangian quantity is:
So,
According to the Lagrangian equation,
Letting q1 and q2 be the state variables of the system, we solve the above equations for the unknown variables, and after tedious simplification and organization, we get:
Clearly, this is a nonlinear non-stationary differential equation. The pendulum can be treated as linear near the vertical state, and Laplace transforms can be used.
After obtaining the system's dynamics equation, various controls can be performed, such as LQR, MPC, etc.
The above are several types of dynamics calculation equations frequently used in robot design. With these calculated formulas, we can easily obtain the force situation of some components. However, if a component is connected to many other components and the force forms are very complex, then dynamics formulas will be of no help. Therefore, we need to use dynamics simulation software, such as Adams, to directly simulate the forces on various axes, motion speeds, angular velocities, etc.
Mechanical Arm Joint Angle Displacement and Angular Velocity Adams Simulation
Now we can calculate or simulate the forces and motion trajectories of each component, but we will also face some structural checks. For example, if I calculate that the supporting force of one leg of the mechanical dog is 100N, and the supporting force reaches 200N when jumping, can my leg components withstand such a large force? If the structure is relatively simple, I can calculate the force on the component using mechanics equations and compare it with the material yield strength to see if it meets the requirements. But what if the component shape is more complex? Formulas cannot resolve this, and we need to use finite element simulation, such as Ansys. Finite element simulation can not only simulate the force situation of components but also perform topology optimization, ensuring that the minimum mass meets the force requirements. For example, the leftmost is the designer's initial design shape, and through step-by-step optimization, the best component shape is obtained, reducing mass while ensuring component strength.
The leg structure of the mechanical dog has also undergone topology optimization; otherwise, such a large inertia would significantly affect the control algorithm of the mechanical dog.
After this round of analysis, we also need to verify the robot's kinematics, such as the walking trajectory of the mechanical dog. The walking trajectory of the mechanical dog generally requires simulation software for auxiliary analysis.
For example, CoppeliaSim acts as a physical entity; you can control it just like a real mechanical dog, serving to validate in advance and reduce trial and error costs.
03 Skills Required for Robot Hardware Circuit Design:
Analog/Digital Electronics; Hardware Schematics, PCB Design; Hardware Debugging; EMC/EMI Analysis;
Software to master: PCB Design (Altium Designer, Allegro Viewer), Circuit Simulation Software (PSPICE), Signal Simulation Software (HyperLynx);
The first step in hardware circuit design is to draw the circuit schematic based on the functions of the robot, then layout the PCB based on the schematic. Software such as Altium Designer or Allegro Viewer is used. For those focused on hardware technology, it is recommended to use Altium Designer or Allegro, which are feature-rich and customizable, supporting many plugins, though they have a steeper learning curve. For those who do not often draw boards, Lichuang EDA can be used, as it integrates a component library, making it convenient to find purchasable components while producing them without searching on Taobao or the Lichuang Mall.
After completing the PCB design, PSPICE circuit simulation must be performed to prevent short circuits and other issues; signal simulation is conducted to prevent signal interference.
The most commonly used microcontroller for low-level controllers is the single-chip microcontroller, such as STM32, used to control stepper motors, servos, etc. It can use pulse control or communication control, such as RS485 or CANopen (for example, the mechanical dog uses 2 STM32 as the main control and 12 STM32 as sub-control units to control the movements of the four legs of the mechanical dog). The microcontroller, combined with peripheral circuits, forms a complete PCB board, as shown in the following diagram.
Altium Designer Circuit Schematic
Mechanical Dog PCB
After the PCB design is completed, comprehensive testing must be conducted on the PCB board, using instruments such as soldering irons, multimeters, oscilloscopes, logic analyzers, bit error testers, transmission analyzers, Ethernet testers Smartbits/IXIA, calorimeters, attenuators, optical power meters, and RF signal strength meters. Especially for testing the power circuit, as chip voltages are diversifying, with lower voltages and higher currents, the strict requirements from operators for communication device power consumption and heat dissipation pose significant challenges for power design. It can be said that for hardware design, 40% of the work is related to the schematic/PCB design and subsequent testing and verification of the power circuit, which reflects the circuit capabilities of hardware engineers, as various passive components, semiconductor devices, protective devices, and typical topologies of DC/DC conversion have many parameters and formulas to consider and calculate. The following is a signal collection and analysis using an oscilloscope, including power ripple analysis.
Oscilloscope Signal Collection
In addition, EMC/EMI tests must be conducted to prevent external electromagnetic fields or static electricity from interfering with the PCB board, while also preventing the board itself from interfering with surrounding electronic devices.
EMC Interference Testing
Not only must you design the circuit board, but you must also be familiar with various CPU selections, such as M68k series, PowerPC860, PowerPC8240, 8260 systems, etc. In addition to the PCB control board that needs to be designed for the mechanical dog, existing processors will also be used, such as:
Boston Dynamics' Spot's built-in processor is entirely dedicated to the dog's movement and navigation, and does not support customer use. If customers wish to process and analyze data from Spot or any sensors it carries, they must add a computer or pay $3925 for an additional processor called Spot CORE provided by Boston Dynamics. The specifications of Spot CORE are as follows:
CPU=i5 Intel 8th Gen (Whiskey Lake-U) Core CCG Lifecycle
Operating System=Ubuntu Desktop 18.04 LTS 64-bit
RAM=16 GB DDR4 2666 MHz
SSD=512 GB SSD
If you want to run compute-intensive tasks like machine learning on Spot, you can pay $24,500 for an additional development environment called Spot CORE AI provided by Boston Dynamics, with the following specifications:
CPU=Intel Xeon E3-1515M V5
GPU=NVIDIA P5000
Operating System=Ubuntu Desktop 18.04 LTS 64-bit
CPU RAM=32 GB
SSD=480GB mSATA SSD
04 Robot Software Design
The software part is divided into low-level control software and upper-level software.
4.1 Low-Level Control Software
Skills to master: Embedded programming in C, freeRTOS operating system
Software to master: Microcontroller programming software (Keil, VSCode, Ozone, etc.), debugging software (serial port debugging assistant, etc.)
Currently, mainstream control schemes still use microcontrollers, but there is a trend towards unification with desktop systems. Various ROS-supported serial/FDCAN/Ethernet buses can run smoothly, and the industry is actively adapting support for x86 and arm64 platforms. The real-time performance of Linux can also be improved through real-time kernel patches, with typical response times achieving 0-50ns, indistinguishable from microcontrollers.
Taking the STM32 series as an example, many veterans are accustomed to using ARM's KEIL. Many online tutorials are based on Keil, but the theme support and intelligent code highlighting checks in Keil are outdated, and it even cannot set multi-threaded compilation. Using the HAL library can be quite torturous. After learning embedded development for some time, it is recommended to use CLion or VSCode to replace Keil for programming, utilizing Ozone for visual debugging, and using Segger RTT Viewer for log printing.
Additionally, microcontrollers are single-threaded program architectures. By utilizing the known freeRTOS real-time operating system, they can be transformed into multi-threaded systems, significantly improving response times for important tasks, allowing them to receive higher priority for execution.
FreeRTOS Real-Time Operating System Diagram
For microcontroller development, the recommended software is as follows:
Keil, low learning curve, suitable for beginners.
VSCode, highly customizable, but not as straightforward as Keil and requires some configuration.
Ozone, a debugger from SEGGER (the company that makes J-Link). Visual debugging is its major strength; unlike traditional serial debugging, its debugging information is sent via DBG, which is connected through several lines, allowing debugging without occupying system resources and enabling full-speed operation non-blocking.
Serial Port Debugging Assistant: Recommended Serial Port Debugging Assistant (named as such, can be installed directly from the Microsoft Store, with an icon of a 9-pin 485 serial port) and VOFA, the latter having excellent graphics and supporting data visualization. Linux alternatives are cutecom and minicom, with VOFA also available.
4.2 Upper-Level Control Software
Skills to master: Linux operating system, Ubuntu system, ROS system
Software to master: ROS operating system, VScode, etc.
The controllers used in robots are almost all based on Linux systems. The Linux operating system has many advantages, such as small memory footprint, customizable, and scalable. During the learning process of Linux, it is essential to understand the operation modes of open-source software, the concepts of file systems and kernels, and shells. It is also best to understand bootloaders, virtual memory, paging mechanisms, etc. Package managers and basic terminal commands must be mastered. You should also understand the basics of compilers and learn how to compile C code without IDE support. Make and CMake are essential for building projects in the Linux environment, and the ROS build system catkin_make is built on CMake.
As for ROS, it is a must-learn software in the era of Industry 4.0. ROS is not an operating system but provides a complete robot development software stack, encapsulating various functions, including inter-module message interaction (one of ROS's most important functions), visualization, and simulation. Although ROS currently provides experimental support for Windows, it is still recommended to develop under Linux, meaning Linux is a prerequisite for learning ROS.
05 Robot Algorithm Design
Control algorithms are divided into two types: robot motion control algorithms and large model-based algorithms.
5.1 Large Model-Based Algorithms
The core function of robot algorithms is autonomous navigation and positioning technology.
Traditional scheme: SLAM + Path Planning + Motion Control
Modern scheme: Deep Learning + Motion Control
The difference between modern and traditional schemes: Traditional methods use probability or control theory for autonomous positioning and navigation. However, with deep learning, camera data can be used directly as signal input, and neural networks can directly produce control signals for the robot. Processes such as SLAM and path planning can be entirely implemented through deep learning. The following diagram shows a mapping developed using ROS's SLAM package, displayed via RVIZ.
SLAM Mapping
Furthermore, the mainstream perception algorithm for robots is still computer vision, making OpenCV an essential computer vision library.
In addition to image processing, deep learning-based object detection/segmentation algorithms are also indispensable. It is recommended to use PyTorch to build neural networks. To use PyTorch, a basic understanding of machine learning and statistical learning algorithms is required, along with the use of the sklearn library in Python.
Neural Network Model
Regarding positioning and mapping (SLAM), the most commonly used software remains ROS. If point cloud processing is needed, the PCL (Point Cloud Library) is essential, along with Eigen (matrix operation library), g2o (graph optimization library), Ceres (nonlinear least squares library), and some libraries for handling Lie groups.
Point Cloud Processing
5.2 Motion Control Algorithms
Motion control algorithms include LQR, MPC, WBC, sensor fusion algorithms, global planning A* algorithms, and local planning DWA algorithms, etc. Mechanical dogs generally use WBC (optimized MPC) and other algorithms.
WBC Algorithm (Optimized MPC)
Algorithms can be said to be the soul of robots, involving deep mathematics and mechanics foundations, such as understanding dynamics, coordinate transformations, linear algebra matrix transformations, etc. For example, the kinematics of a mechanical dog involves coordinate transformations, converting the requirements for the center of mass's speed/displacement to the limbs.
Mechanical Dog Coordinate Transformation
Common sensor fusion algorithms, such as using EKF (Kalman Filter) or ESKF (Extended Kalman Filter) for implementation (generally found in LIO), and other advanced algorithms, are also available. Kalman filtering is widely used to smooth the data obtained through filtering sensors.
Kalman Filtering for Multi-Sensor Data Fusion
The global planning A* algorithm allows the mechanical dog to reach any specified endpoint by establishing a pre-built map.
Effects of Different Improved A* Algorithms
The DWA local planning algorithm is essential for the mechanical dog when running along a globally planned route, as it will inevitably encounter local obstacles that did not exist during the pre-mapping phase. In this case, the mechanical dog must temporarily avoid this obstacle, which is the primary task of local planning.
DWA Local Planning Algorithm
06 Conclusion
Robots are very complex system designs involving the integration of many disciplines, making them some of the most advanced exists among various fields. This also determines that commercializing a robot product becomes very challenging. However, technology is continually advancing, and these issues will eventually be resolved. For example, the recent popularity of ChatGPT has provided some ideas for the intelligent development of robots. In summary, the future of robotics is becoming increasingly bright.
Disclaimer: Some content comes from the internet and is for readers' learning and communication purposes only. The copyright of the article belongs to the original author. If there is any inappropriate content, please contact to delete.
—THE END—