The “brain” and “cerebellum” of robots each have their own roles.
Written by｜Huang Nan
Edited by｜Peng Xiaoqiu
Source｜Hard Kernel (ID: south_36kr)
Cover Source｜IC photo
Recently, the news of NVIDIA establishing the General Embodied Intelligence Research Laboratory (GEAR) has once again brought robots to the forefront.
In the past year, amidst the wave of AI large models, the combination of large models and robots has provided a new interaction mode for human-machine coexistence. Some even argue that the evolution speed of the robot’s brain mainly depends on the development speed of large models.
The market’s enthusiasm for robots is reflected in the actual capital environment, with financing projects often reaching hundreds of millions or even billions, a sudden increase in the scale of related components, and downstream concept stocks also rising… Embodied intelligence has become a key term in the stories of robot implementation.
However, in practical implementation, robots can only complete some customized tasks, lacking the ability to understand complex scenarios, leading to limited applications; on the other hand, robots running according to pre-set algorithms also struggle to generate greater intelligence, with thinking and decision-making capabilities unable to improve.
How to enable embodied intelligent robots to continue learning through constant interaction with humans and the environment has become a key issue.
The Dilemma of Robots in Open Scenarios
For a long time, there has been a classic paradox in the fields of AI and robotics—the Moravec’s Paradox suggests that, contrary to traditional assumptions, computers require only a small amount of computational power to achieve high-level intelligence such as reasoning, while intuitive and perceptual abilities require enormous computational power.
In simple terms, a computer can easily win against a human in a game of Go, but asking a robot to fetch a can of cola from the fridge is not so simple.
This process involves several necessary steps. First, the robot needs to understand the instructions given by humans and break down the task; the second step is to make decisions, determine the target location, and plan the route; the third step, upon reaching the destination, the robot must not only recognize the “cola” among various items in the fridge but also control its mechanical arm to perform the picking action; finally, it must deliver the item to the designated location.
In the past, these actions could be pre-set, but due to immature technology, robots could only provide case-by-case services; that is, it only learned to fetch “cola,” and if the item were changed to “Sprite,” engineers would need to re-establish a new process.
When robots enter real-world scenarios, they often face two major pain points: weak generalization ability and high delivery difficulty.
Qiu Dicong, CEO of Yakobi Robotics, classified robot application scenarios into closed and open scenarios during a conversation with Hard Kernel.
Closed scenarios refer to the classic customized model. The task execution boundaries of the robot are pre-set, and after the user issues instructions, existing data is collected for training, exhausting all possible tasks within a limited range to cover as many solution paths as possible.
For example, in industrial scenarios, transportation robots have their transport routes pre-set by engineers for specific parks, and the robots only need to complete delivery tasks along fixed paths, with low flexibility; even if there are multiple routes, these paths are also arranged and planned, and once the robot deviates from the tasks set by the map, it cannot operate. Therefore, when faced with new tasks, the robot needs to collect data for training, set plans, and test again, leading to significant resource and labor waste.
Open scenarios refer to long-tail problems that are not subject to strict closed norms and ranges, such as supermarket services and home care. When robots interact with humans and the environment extensively, the dataset may encounter situations with few or even zero samples, which requires a high level of generalization ability for robots to understand and handle various tasks.
For instance, while robots can currently perform logistics sorting and warehouse sorting, sorting in supermarkets has yet to be realized. The main reason is that the goods in warehouse logistics centers can be standardized and classified by shape and size, while the products are well-sealed and have more redundancy. In contrast, supermarkets have a wide variety of products; a 1.5L white bottle could be milk, yogurt, or coconut juice; additionally, the characteristics of the products differ greatly, such as the fragility of eggs compared to paper towels, which places high demands on sorting robots, requiring precision in both vision and force control.
In Qiu Dicong’s vision, robots should not just be executors of instructions but should also possess the ability to learn and generalize. “Today, robots can perform well in tasks such as cleaning and inspection, but to truly integrate into people’s daily lives, they need to enhance their perception, decision-making, and execution capabilities.”
With the explosive growth of large models in China in 2023, Qiu Dicong sees new opportunities for AI and robots. Qiu Dicong graduated from Carnegie Mellon University (CMU) with a focus on robotics and has participated in projects such as NASA’s Mars rover development and Level 4 autonomous driving, with over 8 years of cross-disciplinary research and implementation experience in AI and robotics.
As technology improves and costs decrease, the penetration rate of task-oriented robots such as vacuum cleaners and industrial arms has significantly increased. However, to enhance the intelligence level of robots, more advanced algorithms and data support are needed.
Natural language data is offline and belongs to methodological learning; whereas robot decision-making relies on a lot of high-quality data, most of which comes from unexpected situations encountered in open scenarios. Thus, extending from single closed scenario tasks to open scenarios has become a path for robot practitioners to follow.
In April last year, Qiu Dicong and his team established Yakobi Robotics, focusing on embodied intelligent supermarket service robots, which can execute multiple tasks in supermarket scenarios through human voice commands, including autonomous inspection, automatic restocking, and product sorting. Four months later, Yakobi Robotics completed its seed round of financing, with investors including AI expert and founder of Qiji Chuangtan, Lu Qi.
On one side, the high demands of open scenarios on robots include the ability to perceive open vocabulary, task planning without machine learning methods, and the closed-loop capability of high-frequency task execution; on the other side, the capabilities of large models in semantic understanding, abstract planning, and reasoning have been validated, allowing them to handle many complex tasks, providing a feasible pathway for robots to be applied in long-tail scenarios.
Equipping Robots with a “Brain + Cerebellum”
Nobel laureate Daniel Kahneman proposed in his book “Thinking Fast and Slow” that humans have two modes of thinking: the first is “fast thinking,” primarily based on intuitive judgment, and the second is “slow thinking,” which requires extensive reasoning and calculation.
The difference between large models and past AI technologies lies in the adoption of the “slow thinking” mode, allowing robots to continuously learn through interactions with humans, gaining better capabilities to solve tasks and handle more tasks. However, due to this, investors focusing on the AI and robotics sectors, such as Guo Xu, tell Hard Kernel that most deployed or complete robot products and projects generally hope for an “all-in-one” solution, for example, leveraging the strong capabilities of large models to create a giant end-to-end model to solve all problems.
In response, Qiu Dicong pointed out, “From the perspective of actual ROI, it may not be cost-effective or suitable for the current stage.”
The complex demands of open scenario users and the high costs of training and reasoning for large models are significant concerns. According to research from abroad titled “The Economics of Large Language Models,” the training cost for each token (approximately 750 words for 1000 tokens) is typically around 6N (where N is a unit of measurement for parameters), and the reasoning cost is about 2N. This means that the reasoning cost is roughly one-third of the training cost. Once the model is deployed, its reasoning costs may far exceed the training costs.
Therefore, the deployment costs of large models in robots are also high. The end result is that their market prices are not affordable for ordinary small and medium-sized enterprises, limiting market scale and penetration.
To balance this awkward situation, humanoid robots represent an important exploration direction. This involves allowing robots to mimic the operational division of the human brain, using a “brain + cerebellum” structure for mutual complementarity, where the brain is responsible for high-level perception and decision-making functions such as vision, hearing, and consciousness, while the cerebellum is responsible for coordinating data to control movement, balance, and behavioral posture.
For example, the “Universal Robot Brain” proposed by Qiu Dicong and his team consists of a “brain” (J-Mind) and a “cerebellum” (J-Box), where J-Mind is responsible for understanding tasks and issuing instructions, which are then executed by J-Box.
First, at the perception level, the combination of LLM + VLM (Large Visual-Language Model) technology can understand instructions in conjunction with the physical environment, enhancing the robot’s cognitive ability in open scenarios, meaning it can not only “see” various items in the scene but also “understand” user needs. For instance, if the robot originally only recognizes cola, upon seeing Sprite or orange juice, it can infer that they are also canned beverages based on their similar appearance to cola and read the packaging information to recognize the new items.
Yakobi Robotics in action
Qiu Dicong told Hard Kernel that Yakobi Robotics chose supermarkets as the first application scenario for its products precisely because of the gathering effect of personnel in supermarkets, which generates a large amount of repetitive item information and interactions, providing data support for the robot’s self-learning. In other words, the robot collects data in real-time from real scenarios rather than searching for answers in an existing database.
At the decision-making level, the robot can convert user needs into specific instructions and sub-steps through J-Mind, forming a dynamic closed-loop for task assignment output and decision-making, issuing execution tasks to J-Box. Subsequently, J-Box drives the robot to complete actions such as manipulation, grabbing, and placing.
The robot is performing “grabbing and placing” actions
For example, when a supermarket shelf is out of stock, the staff only needs to verbally or textually input the instruction “the shelf is out of cola, needs restocking,” and the Yakobi robot can automatically move to the required shelf, recognize the shelf’s display status. When J-Mind identifies cola among various items, it can break down the restocking instruction into sub-steps, allowing J-Box to grab the cola and place it in the empty spot on the shelf.
This “brain + cerebellum” approach integrates many classic mainstream robot algorithms into the framework of the universal robot brain, requiring no deployment engineers and can be used out of the box; it also supports manual scheduling and robot automation assistance, providing higher flexibility, allowing the robot brain to determine whether a task requires invoking a large model or can be solved with algorithms, thereby reducing service costs.
Commercialization Dilemma: Long Iteration Cycles and High Costs
Public data shows that from 2017 to 2021, the global intelligent service robot market grew from less than 10 billion to 20 billion USD, and it is expected to exceed 60 billion USD by 2026.
Among them, the Chinese intelligent professional service robot market is expected to grow from 100 billion in 2021 to 1 trillion by 2026, with rapid growth.
It can be seen that compared to traditional robots in shopping malls that can only move or display advertisements, the emergence of large models has given people hope for embodied intelligence, and supermarket robots represented by Yakobi have achieved a significant upgrade.
However, pain points still exist. On a technical level, large models have enhanced the understanding capabilities of robots, but robots themselves are complex interdisciplinary systems involving bionic design, AI applications, dynamic modeling, energy management, etc. To achieve understanding, decision-making, and control of movement and task execution, various algorithms and software must be matched. The upgrade and iteration of AI technology is not a linear development; it has long cycles, high investments, and risks of failing to break through key technologies for a long time.
On the hardware side, the robot’s assembly structure is complex, and core components determine important performance indicators such as precision, stability, and load capacity, with the highest technical difficulties being the reducer, servo system, and controller, accounting for 70% of the cost. Coupled with sensors and other components, these increase the manufacturing and subsequent maintenance costs of robots.
Qiu Dicong told Hard Kernel that supermarket scenario customers are very concerned about ROI. To this end, Yakobi Robotics has found supply channels for core components, “This method has a high cost control space, and the calculations meet expectations.”
Moreover, during the productization process of robots, the integration and iteration based on open scenarios also require time to generate and validate value.
A market manager from a robot manufacturer told Hard Kernel, “Downstream buyers need to be expanded and educated; it is difficult to achieve acceptance and recognition immediately upon launch. The solutions are continuous product iteration and maintaining close communication with customers; this is a process of co-developing products and scenarios, discovering more product value through this co-creation approach.”
For example, in addition to supermarket scenarios, Yakobi Robotics is also exploring environments such as restaurant services, offices, and homes. Robots can serve as restaurant waiters to complete tasks such as taking orders and serving food; as company receptionists to guide visitors and distribute materials; and as household assistants to help organize clothes and clean furniture.
Workflow of robot dining service
“Our ultimate goal is to start from supermarket scenarios and transfer the knowledge learned by robots to more scenarios, integrating into daily family life, truly achieving automation of multiple tasks in a closed loop,” said Qiu Dicong.
It cannot be denied that robots on the market are still in a “toy” stage, far from achieving large-scale industrialization. However, it is expected that through the integration of robots and large models, with the simulation evolution of “brain + cerebellum,” the automated collaboration of large models and classic algorithms, and the mutual transformation of rules and models, a more complex and self-evolving robotic intelligence agent may not be far off.
Hardcore Technology Reporting Public Account under 36Kr
👇🏻 We sincerely recommend you to follow 👇🏻
Let’s have a “share, like, and watch” 👇
When AI Large Models Converge with Service Robots