Click the blue text above to follow Tian Nan.
The approach to autonomous driving cannot solve the mass production issues of robotics in the short term.
Written by｜Tian Nan
Over the weekend, I chatted with friends in the autonomous driving field, and they asked me not to switch to humanoid robotics.
I have to admit, this industry has indeed become quite popular recently.
However, robotics is a much more coupled system, and the approach used in autonomous driving may not necessarily solve the mass production issues of the robotics industry in the short term.
(Tesla Cybertruck and Optimus)
01
After discussing with them, I found that most domestic car manufacturers are still using a perception + decision-making layered model design.
To explain briefly, the perception model integrates information from sensors such as LiDAR and cameras to process and obtain information like lane markings, traffic lights, and pedestrian recognition.
Then, this recognition information is passed to the subsequent decision-making model. For example, when encountering a red light, it needs to gently apply the brakes; when deviating from the lane, it needs to slightly turn the steering wheel.
Sounds reasonable, right?
But if you think about it carefully, which good driver would first see a red light, think about it being a red light, and then decide to brake?
A skilled driver sees a red light and immediately applies the brakes. This is a thought process, not a two-step operation.
If it operates in two steps, it is not like a normal person, and more importantly, the perception module of autonomous driving is determined by human-selected reference parameters, which inevitably leads to information loss in transmission, and this is fatal.
For instance, if it is determined to recognize traffic lights, lane markings, and pedestrians, the real world has far more than these three states. If the weather suddenly hails or, more extremely, if some indescribable extraterrestrial substance falls.
A normal driver knows to slow down and stop, but the two-stage perception model, due to information loss in transmission, cannot make a braking judgment even if the decision-making module is powerful.
Therefore, while the two-stage model design has strong interpretability, it loses too much information and can easily deviate from normal judgment logic, causing significant problems.
In the field of autonomous driving, this is not a big issue; simple closed roads, such as in high-speed scenarios with LCC (Lane Centering Control), are definitely sufficient.
(Xpeng Motors and humanoid robots)
02
As we move further down this path, switching to more complex scenarios, as well as humanoid robot scenarios, this method will encounter significant problems.
Consider that the difficulty in terms of degrees of freedom increases by dozens of times.
A car generally only needs to control a few signals: brakes, steering, and throttle.
In contrast, a humanoid robot has at least 50-80 degrees of freedom, corresponding to more motor controls (for example, the dexterous hand of Tesla’s Optimus has 25 degrees of freedom).
(KGG’s new generation dexterous hand with 22 degrees of freedom)
Moreover, a very important point is that the degrees of freedom in a car are not coupled.You can turn the steering wheel while braking.
But for robots, the movement of your little finger must rely on the cooperation of the upper arm, forearm, wrist, and joints to move together.
At this point, you may realize:
Some domestic robots seem to have directly borrowed from autonomous driving, such as fixed scenarios, grippers, and wheeled designs.
Referring to previous videos, comparing the scenes below, you will understand that the essence of this scene is very similar to autonomous driving.
03
If the difficulty were only in controlling degrees of freedom, it would be manageable; the more challenging aspect is the data.
In autonomous driving, major manufacturers collect data by driving LiDAR-equipped vehicles on real roads every day, all for the sake of data.
Labor costs are not very high; after all, you can cover 500-800 kilometers in a day.
But for robots, data collection is much more difficult. If it involves imitation learning, it requires finding people in specific scenarios to use VR + remote control for data collection, and at the end of the day, usable data may only amount to 1-2 hours. This is because the precision of VR controllers is low, and the process can only be very slow.
Moreover, the costs are burning; Tesla pays $48 per hour to those collecting data for imitation learning.
Data is incredibly difficult to obtain. When I spoke with Qianxun at WAIC, they mentioned that they need over ten machines for imitation learning just to stack clothes with a gripper, not to mention the large-scale implementation of home and medical scenarios.
Friday’s article also mentioned that Tesla, due to these considerations, has abandoned the two-stage reasoning in their Model series and has also given up on imitation learning. They now aim to achieve breakthroughs through FSD by directly enabling end-to-end video learning.
This difficulty is also significant, but once solved, the prospects are limitless.
04
If I were to give advice to friends working on autonomous driving algorithms:
The technology stack is basically the same, and the experience is similar. It is currently a hot trend, so I suggest you can go for it, but be prepared for the much greater difficulty in technical implementation.
The risks associated with the difficulty of implementation mean that the company may not survive for several years.
However, there are lower-risk paths:
If you can join a large company like Tesla Optimus to work on end-to-end algorithms for software and hardware, or join certain “hardware” star companies in China to work on robotic motion control algorithms, or work on brain end-to-end and world model algorithms at software companies or major internet firms like ByteDance, I feel the risks would be much lower.
Try to avoid choosing startups that require both software and hardware development…
If any of my readers are students, I suggest you pursue a PhD abroad and avoid working on pure software algorithms like LLMs or diffusion. Instead, focus on algorithms related to robotics, VLA algorithms, world models, etc. This way, when you graduate, it will be a great time for implementation.
If you enjoy Tian Nan’s articles, please “click to follow” and “like and share” to explore the future of robotics together.
Feel free to leave your thoughts in the comments!