The explosion of generative artificial intelligence has given rise to the first generation of “AI + Robotics” humanoid robots. With the support of artificial intelligence technology, robots have once again become popular. Many friends are very interested in robots but do not know what components make up a robotic system and how it is implemented. What knowledge is needed for future jobs related to robotics?
I studied mechanical design, manufacturing, and automation in my undergraduate studies (that’s right, I chose this major in college because it has the longest name, combining both mechanics and automation, and it sounded impressive!). During my master’s studies, I found robotics to be very sophisticated and switched to control science to study robotics (that’s right, I just felt that robotics was very sophisticated!). I have worked on mechanical design, finite element analysis, software development, driver development, MATLAB simulation, system identification, sensor signal processing, etc. (all learned and applied on the job, with a basic understanding!). After graduation, I worked at a leading company in the field of mobile and industrial robots.
In fact, robotics is a highly interdisciplinary and complex technology.
According to the general curriculum design of an undergraduate automation major, robotics utilizes the following foundational knowledge (first and second year, with application areas in parentheses):
-
Advanced Mathematics (Robot Dynamics)
- Linear Algebra (Coordinate Transformation; State Space Control)
- Probability and Mathematical Statistics (Filters; Position Prediction; Decision Making)
- Complex Variables (Frequency Domain Transformations Corresponding to Signal Processing; z-Transform Corresponding to Discrete-Time Systems)
- Digital and Analog Electronics (Hardware Design for Robots)
- Microcontroller Principles and Applications (Robot Controllers)
- Computer Principles (Same as Above)
- Computer Architecture (Hardware Design)
- Operating Systems (RTOS/Linux)
- C/C++ Programming (Robot Programming)
- Introduction to Algorithms (Robot Algorithms: Path Planning)
More specialized and targeted courses (third and fourth year):
- Principles of Automatic Control (i.e., Classical Control Theory)
- Modern Control Theory (State-space Control, MPC Model Predictive Control, etc.)
- Intelligent Control (Expert Control + Fuzzy Control + Robust Control + Neural Network Control)
- Mechanical Drawing
- Mechanical Principles (Gears, Transmission, Finite Element Force Analysis)
- Machining (CNC Machines, etc.)
- MATLAB/Simulink or NI Labview (or Rapid Control Prototyping RCP)
- Sensors and Signal Processing (sometimes also includes System Identification)
- Motion Control Systems
- Electrical Control and PLC (Some Industrial Robots are still controlled by PLC)
- Power and Electronic Systems (Mainly for Motor Control and Power System Design)
Finally, in the graduate stage (dual control; repeated courses are not listed):
- Introduction to Artificial Intelligence
- Pattern Recognition and Control
- Introduction to Robotics
- Control of Robotic Arms
- Mobile Robots
- Nonlinear Control Systems
- Optimal Control / Adaptive Control
- Multi-Sensor Fusion
- Intelligent Manufacturing
In practice, knowledge of machine vision, deep learning, SLAM, path planning, ROS operating systems, etc., will also be required. In fact, the development languages used are not limited to C and C++. In commercial applications, web development, interface development, and third-party integrations may involve languages like Java, JavaScript, and C#.The following is an explanation by a certain expert on Zhihu on how to develop a robot:
1. Composition of a Robot System
Taking a mechanical dog as a platform to illustrate some of the technologies involved in unmanned equipment, the same explanation can apply to drones and unmanned vehicles. The following diagram shows the system composition of a mechanical dog, listing some popular and main technologies, along with others not listed, but which do not affect the design of a robot system.
A robot generally consists of three major components:1. Mechanical Structure Design2. Hardware Circuit Design3. Control Algorithms and Perception Decision Algorithm DesignThis article will start from these three components to analyze the skills involved in robot design.The process represented by the above system diagram for a robot is:(1) The operator gives high-level commands by specifying the required translational speed and turning rate.(2) Upon receiving the high-level commands, the CoM reference trajectory is generated and sent to the body and leg controllers.(3) The controller uses the user input commands and the robot’s status, using the “Swing Leg Controller” if the legs are in swing, or the “Force Control Support Leg Controller” if the legs are in support.(4) Force and position commands are sent to the microcontroller to relay motor commands to each leg of the robot.(5) If there are higher requirements for the mechanical dog, such as autonomous navigation, algorithms like SLAM or AI can be run on an industrial computer.
2. Robot Mechanical Design
Skills involved: 1) Kinematics; 2) Dynamics; 3) System SimulationSoftware involved: 1) 3D Design Software (SolidWorks, UG, CATIA, etc.); 2) Dynamics and Kinematics Software (Adams, CoppeliaSim, etc.); 3) System Simulation Software (CoppeliaSim, Webots, etc.); 4) Finite Element Simulation Software (Ansys or Abaqus, etc.)When designing a robotic body, we typically focus on two main questions: whether the body can withstand large loads (i.e., dynamics) and whether the body can move according to the predefined trajectory (i.e., kinematics).In dynamics, the first consideration is the source of power, such as motors, hydraulic cylinders, electric cylinders, etc. For the mechanical dog, the power issue is critical; it must be strong enough to meet the basic requirements of the mechanical dog. The following diagram shows the motor structure of the mechanical dog, where the designer’s intention is clear: the motor, planetary gearbox, and the mechanical dog’s shell are combined to reduce the weight and inertia while saving space, making it easier for the legs to perform various movements.
With a basic design plan in place, we also need to verify the motor torque and analyze the forces acting on the robot, which involves dynamics formulas such as Newton’s laws, D’Alembert’s principle, and Lagrangian dynamics equations. Often, one equation cannot solve all problems, so these equations need to work together. When using these formulas, we should learn to simplify complex models for easier modeling. For example, the leg of the mechanical dog can be simplified as shown in the following diagram:
In robot design, the D’Alembert equation / Lagrangian dynamics equations are often used. With these calculation formulas, we can easily obtain the forces acting on some components. However, if a component is connected to many other components, the force forms can become very complex, making dynamics formulas difficult to apply. Therefore, we need to use dynamics simulation software like Adams for direct simulation. For instance, the following robotic arm can be analyzed for force, motion speed, angular velocity, etc., using Adams software.
We can now calculate or simulate the forces and motion trajectories of each component. However, we also face structural verification issues. For instance, if I calculate that the supporting force of one leg of the mechanical dog is 100N, and the supporting force reaches 200N during a jump, can my leg components support such a large force? If the structure is simple, I can use mechanics equations to calculate the forces and compare them with the material yield strength to see if they meet the requirements. But what if the component shape is complex? In that case, we need to use finite element simulation, like Ansys, which can not only simulate the forces acting on components but also perform topological optimization to meet force requirements with minimal mass. For example, the leftmost image shows the initial design shape, and through iterative optimization, we obtain the best component shape, reducing mass while ensuring strength.
The leg structure of the mechanical dog has also undergone topological optimization; otherwise, such a large inertia would severely affect the control algorithm.
After the above analysis, we need to verify the robot’s kinematics, such as the walking trajectory of the mechanical dog, which generally requires simulation software for analysis.
3. Robot Hardware Circuit Design
Skills to master: Analog/Digital Electronics; Hardware Schematics, PCB Layout; Hardware Debugging; EMC/EMI Analysis;Software to master: PCB Design (Altium Design, Allegro Viewer), Circuit Simulation Software (PSPICE), Signal Simulation Software (HyperLynx), The first step in hardware circuit design is to draw the circuit schematic based on the functions the robot needs to perform, and then lay out the PCB based on the schematic. The software used can be Altium Design or Allegro Viewer. For those with a hardware-focused tech stack, it’s recommended to use Altium Design or Allegro for their comprehensive functionality and high degree of customization, although they can be a bit difficult to learn. For those who do not frequently design PCBs, Lichuang EDA can be used, which integrates a component library to easily find available components while making the design, eliminating the need to search on Taobao or Lichuang Mall.Once the PCB design is complete, it must undergo PSPICE circuit simulation to prevent issues like short circuits; signal simulation is performed to prevent signal interference.Generally, the most commonly used lower-level controllers are microcontrollers, such as STM32, which are used to control stepper motors, servos, etc. They can be controlled by pulse or communication methods like RS485, CANopen, etc. (for example, the mechanical dog uses 2 STM32 as the main control and 12 STM32 as sub-control units to control the motion of the four legs of the mechanical dog), and the microcontroller paired with peripheral circuits forms a complete PCB board, as shown in the following diagram.
Altium Design’s circuit schematic
PCB board of the mechanical dogAfter the PCB layout is completed, it must undergo comprehensive testing using instruments and tools, including soldering irons, multimeters, oscilloscopes, logic analyzers, error rate testers, transmission analyzers, Ethernet testers like Smartbits/IXIA, calorimeters, attenuators, optical power meters, and RF signal strength meters, among others. Especially for power circuit testing, as chip voltages diversify and current requirements increase, operators face growing challenges in power consumption and heat dissipation requirements for communication equipment. It can be said that for hardware design, 40% of the work involves the schematic/PCB design and subsequent testing and validation of the power circuit, which is a concentrated reflection of the circuit capabilities of hardware engineers. Various passive components, semiconductor devices, protection devices, and typical DC/DC conversion topologies must consider many parameters and formulas.Below is a signal acquisition and analysis performed with an oscilloscope, analyzing power ripple, etc.
Oscilloscope signal acquisitionAdditionally, EMC/EMI testing must be conducted to prevent external electromagnetic fields or static electricity from interfering with the PCB, while also preventing the PCB from interfering with surrounding electronic devices.
EMC interference testing
4. Robot Software Design
The software part is divided into lower-level control software and upper-level software.1) Lower-level Control Software:Skills to master: Embedded Programming in C, FreeRTOS Operating SystemSoftware to master: Microcontroller Programming Software (Keil, VSCode, Ozone, etc.), Debugging Software (Serial Debug Assistant, etc.)Currently, mainstream control solutions still use microcontrollers, but there is a trend towards unifying with desktop platforms. Various ROS-supported serial/FDCAN/Ethernet buses can run smoothly, and the industry is actively adapting to support x86 and arm64 platforms. The real-time capability of Linux can also be enhanced through real-time kernel patches, achieving typical response times of 0-50ns, comparable to microcontrollers.Taking the STM32 series as an example, many users are accustomed to using ARM’s KEIL, and many online tutorials are based on Keil. However, the theme support and intelligent code highlighting in Keil are quite outdated, even lacking multi-threaded compilation support, making using the HAL library quite a challenge. It is recommended to switch to CLion or VSCode after learning embedded development for a while, using Ozone for visual debugging and SEGGER RTT viewer for log printing.
FreeRTOS real-time operating system schematicFor microcontroller development, the following software is recommended:
- Keil, easy to learn, suitable for beginners.
- VSCode, highly customizable, but not as user-friendly as Keil, requiring some setup.
- Ozone, from SEGGER (the company that makes J-Link). Its visual debugging is a major advantage, as it sends debugging information via DBG, which does not consume system resources and allows the system to run at full speed in a non-blocking manner.
- Serial Debug Assistant: Recommended Serial Debug Assistant (that’s its name, available directly in the Microsoft Store; its icon is a 9-pin 485 serial port) and VOFA, which has excellent graphical capabilities for data visualization. Alternatives on Linux include cutecom and minicom, as well as VOFA.
2) Upper-level Control Software:Skills to master: Linux Operating System, Ubuntu System, ROS SystemSoftware to master: ROS Operating System, VSCode, etc.
Most robot controllers use Linux systems. The Linux operating system has many advantages: it occupies little memory, is customizable, and can be trimmed. In the process of learning Linux, it is essential to understand the operating modes of open-source software, the concepts of file systems and kernels, and shells. It is also beneficial to understand bootloaders, virtual memory, and paging mechanisms, while mastering package managers and basic terminal commands is a must. Additionally, understanding compiler-related basics and learning how to compile C code without IDE support is crucial. Make and CMake are essential for building projects in the Linux environment, and the ROS build system catkin_make is based on CMake.As for ROS, it is a must-learn software in the era of Industry 4.0. ROS is not an operating system but provides a complete software stack for robot development, encapsulating various functions, including inter-module message exchange (one of the most important features of ROS), visualization, and simulation. Although ROS currently provides experimental support for Windows, it is still recommended to develop under Linux, meaning that Linux is a prerequisite for learning ROS.
5. Robot Algorithm Design
Control algorithms are divided into two types: robot motion control algorithms and algorithms based on large models.1) Algorithms Based on Large ModelsThe core function of robot algorithms is autonomous navigation and positioning technology.Traditional Solution: SLAM + Path Planning + Motion ControlModern Solution: Deep Learning + Motion ControlThe difference between modern and traditional solutions is that traditional methods use probability or control theory for autonomous positioning and navigation. However, with deep learning, camera data can be used directly as input signals, and the neural network can directly generate control signals for the robot. The processes of SLAM and path planning can be entirely implemented using deep learning methods. The following diagram shows a mapping performed using ROS’s SLAM development package, displayed in RVIZ.
Additionally, the mainstream perception algorithms for robots are still machine vision, so OpenCV is a must-master computer vision library. Beyond image processing, deep learning-based object detection and segmentation algorithms are also essential. It is recommended to use PyTorch to build neural networks. To use PyTorch, a basic understanding of machine learning and statistical learning algorithms is necessary, along with the use of the sklearn library in Python.
Neural Network ModelRegarding positioning and mapping (SLAM), the most commonly used software is naturally ROS. If point cloud processing is required, PCL (Point Cloud Library) is indispensable, along with Eigen (matrix computation library), g2o (graph optimization library), ceres (non-linear least squares library), and some libraries for handling Lie groups.
Point Cloud Processing2) Motion Control AlgorithmsMotion control algorithms include LQR, MPC, WBC, sensor fusion algorithms, global planning A* algorithms, and local planning DWA algorithms, among others.Mechanical dogs typically use WBC (optimized MPC) and other algorithms.
Algorithms can be considered the soul of a robot, involving deep mathematical and mechanical foundations, such as understanding dynamics, coordinate transformations, and linear algebra matrix transformations. For example, the kinematics of a mechanical dog involves coordinate transformations that convert the velocity/displacement requirements of the center of mass into the limbs.
Common sensor fusion algorithms include using EKF (Kalman Filter) or ESKF (Extended Kalman Filter) to achieve this (often seen in LIO). Of course, there are various advanced algorithms, with Kalman filtering being the most widely used, smoothing the data obtained from sensors.
The global planning A* algorithm allows the mechanical dog to reach any specified endpoint by pre-establishing a map.
The DWA local planning algorithm allows the mechanical dog to navigate around unexpected obstacles that may arise during its pre-planned route, which is a primary task of local planning.
6. Conclusion
Robotics is a very complex system design that involves the integration of many disciplines. It can be said to be one of the most cutting-edge fields among various disciplines, which also makes the commercialization of a robotic product very challenging. However, technology is continuously advancing, and these issues will eventually become manageable. For example, the recently popular ChatGPT has provided some ideas for the intelligent development of robots. In summary, the future of robotics is becoming increasingly bright.Ref:[1]https://www.zhihu.com/question/451409605/answer/1801999987[2]https://zhuanlan.zhihu.com/p/634848960