Recently, the research team at Shanghai AI Lab’s Embodied Intelligence Center achieved a breakthrough in robot control by proposing the HoST (Humanoid Standing-up Control) algorithm, which successfully enables humanoid robots to autonomously stand up in various complex environments while demonstrating strong anti-interference capabilities.This innovation not only addresses the challenge of transitioning robots from a sitting to a standing position but also lays the groundwork for the widespread application of humanoid robots in scenarios such as home, medical, and rescue operations in the future.Through a reinforcement learning framework and multiple technical optimizations, the HoST algorithm has shown excellent performance in both simulation and real-world environments, providing innovative solutions for humanoid robots to tackle challenges such as recovering from imbalance and maintaining dynamic balance in real-world situations, further advancing the practical application of embodied intelligence technology.Can humanoid robots sit up on the banks of the Huangpu River in Shanghai to watch the sunrise?They can also sit next to an ‘apple tree’ like Newton and see what falls from the sky.Even under heavy loads and strong interference, they can stand up with ease.
- Project homepage: https://taohuang13.github.io/humanoid-standingup.github.io/
- Paper address: https://arxiv.org/abs/2502.08378
Imagine if humanoid robots could:Stand up from the sofa, walk to the table, and pick up a cup of coffee.Although recent work has endowed them with powerful movement and manipulation capabilities, the aspect of standing up from a sofa has relatively lacked research.Most works assume that the robot starts from a predefined standing posture to perform subsequent tasks. Researchers believe that learning humanoid robot standing control can address the aforementioned sitting-to-standing transitions or standing up after a fall, which will help further enhance the practical scenarios for humanoid robots.Unlike previous control algorithms that relied on predefined trajectories or ignored real hardware limitations, the HoST research team proposed a reinforcement learning framework that starts from scratch and does not depend on predetermined trajectories, enabling the robot to learn to stand up successfully from various postures in a simulated environment and be directly deployed to real robots.
Core Technologies
HoST Algorithm FrameworkReward Function Design and Policy OptimizationUnlike walking and manipulation, the standing control task is more dynamic and requires higher dynamics of the upper and lower body. In particular, for reinforcement learning algorithms, it needs to overcome changing contact points over time, multi-stage skill learning, and precise body angular momentum control, which undoubtedly poses significant challenges for reward function design and policy optimization.To address this, researchers designed various reward functions and categorized them into four reward groups: task rewards, style rewards, constraint rewards, and post-task rewards.
To better balance the various reward functions, researchers then employed a multi-critic technique to report estimates for each reward function group separately and assigned different weights to each reward function group to ultimately optimize the control policy.
Exploration StrategyEven with a reasonable reward function design, researchers still observed difficulties in exploration with reinforcement learning.The research team drew inspiration from scientists’ studies on infants, finding that external assistance helps infants learn many motor skills. Inspired by this, researchers designed a curriculum-based assisted exploration strategy.In the early stages of training, additional upward assistance is provided to the robot to help it stand up more easily and explore high-quality learning samples. As the robot gradually masters the standing ability, this assistance will gradually decrease to zero, enabling the robot to ultimately learn standing control without assistance. This design greatly accelerated the learning efficiency.Motion ConstraintsResearchers observed that robots tend to learn aggressive standing strategies. To overcome this issue, they introduced an action scaling factor (action limits), which determines the maximum deviation between the target joint angle and the current joint angle in the PD controller, implicitly constraining the maximum torque and speed of the joints. This scaling factor was initially set to 1. As learning progressed, this scaling factor gradually decreased to 0.25.
Additionally, researchers also observed behavioral jitter during the standing process. To avoid this issue, they adopted a smoothing constraint method (L2C2) during the optimization of the value function network and policy network.
Real-World Policy TransferTo simulate the initial postures that may be encountered in the real world, researchers designed four terrains in the simulation training: flat ground, platforms, slopes, and walls to mimic common environments in the real world.Furthermore, to reduce the differences in physical parameters between the physical simulation and reality, researchers also employed domain randomization techniques, introducing random noise to certain physical parameters in the simulation, such as center of mass offset, base gravity, etc.Experimental EvaluationQuality of Standing Action in Simulation EnvironmentTo better evaluate the standing action, researchers first proposed four quantitative indicators: success rate, distance moved by both feet, action smoothness, and energy consumption. Based on this, they compared HoST with its ablation version in the simulation. The results, as shown in the table below, indicate that multi-critic, assisted exploration, and motion constraints all significantly impact policy learning.For example, in the absence of the first two, the robot could not successfully learn the standing skill on most terrains; lacking motion constraints would result in the standing action being insufficiently smooth. These validate the importance of the aforementioned technical designs.
Quality of Standing Action in Real EnvironmentNext, researchers directly deployed the control policy to the Unitree G1 robot and tested it in various indoor and outdoor scenarios. As shown in the figure below, successful standing was achieved on various terrains, including wooden platforms, grass, platforms, slopes, against trees, gravel roads, etc.
In the indoor scenario, researchers also compared the impact of smoothing constraints on the real machine performance. As shown in the figure below, smoothing constraints significantly improved the smoothness of the standing action and its success rate.
Robustness TestingUnder complex external interference conditions such as heavy loads, external impacts, soft ground obstacles, and random torque loss, HoST can still maintain stable standing, quickly recover from falls, and maintain dynamic balance.
Source: Shanghai Economic and Information Commission
“Wangxin Shanghai” is the official WeChat account of the Shanghai Internet Information Office, which authoritatively publishes information on internet governance in Shanghai and promptly responds to hot internet public opinion events in Shanghai.