When understanding robots solely through news and short videos, one often experiences a strong contrast: on one side, there are high-energy robots capable of precise punches and agile “sneak attacks” on humans, while on the other side, there are robots that awkwardly “roll around” on the ground. Where does this huge disparity come from? It actually stems from the different application scenarios.

Today, I would like to share a very interesting article. The author, Benjie Holson, cleverly designs a series of competitions for robots based on daily life skills from a technical perspective. Through these competitions, we can not only gain a deep understanding of the complex abilities behind seemingly simple daily actions performed by humans but also clearly see the current limitations of robotic technology.

Benjie’s Humanoid Olympic Games

While the real-life “rock-paper-scissors robot” is fascinating, what people truly care about is whether robots can help with household chores. This is why videos of robots folding clothes are so popular. Those folding videos are indeed impressive: just a few years ago, we didn’t know how to achieve this. This is indeed a function that people want! However, as this article articulates well, basic clothing folding is precisely at the optimal application point of our existing technology. We might think that if AI technology can fold clothes, then it can probably do anything, but that is not the case. To truly achieve versatility and practicality, we still need to invent new technologies.

Challenges

Based on this thinking, I challenge roboticists: here is my Humanoid Robot Olympics. Each event will require us to push the boundaries of technology and unlock new capabilities. As milestones are achieved, I will update this article and send real medals to the winners.

Brief Overview: Current Technology Status

To illustrate why these challenges can push the boundaries of technology, let’s first discuss the currently effective technologies. I see that demonstration learning is playing a role. Researchers use some robots and some manipulation interfaces (the standard practice seems to be using two identical robots, where one is grasped and moved, and the other mimics, or using an Oculus headset with controllers or hand tracking) to repeatedly record activities lasting 10-30 seconds (hundreds of times). We can then train neural networks to mimic these examples. This unlocks tasks with a certain degree of chaotic steps (like pulling a corner of a towel to see if it is flat) or tasks with high state spaces (like a block having only six faces, but a towel can be crumpled in countless ways). However, upon careful consideration, some limitations should be obvious. While there are exceptions, they form an overall trend:

• Lack of force feedback in the wrist. A robot’s performance will never exceed the level of human remote operation, and we do not yet have a good standard method to convey force information to human operators.

• Limited finger control. Operators (and AI base models) find it difficult to see and control all of a robot’s fingers, with precision limited to simple open/close actions.

• Lack of tactile feedback. Human hands are filled with sensors. It is currently impossible for robot hands to possess a perception capability close to this and be usable by human operators.

• Moderate precision. Based on video assessments, I believe we have about 1-3 centimeters of precision in tasks.

Folding towels and T-shirts does not rely on strong wrist strength. You can pull and lift by pinching, and unfold with an open hand, relying solely on the open/close of the hand. You can visually see your grasp, so finger perception is not necessary. A precision of 1-3 centimeters is sufficient.

So what’s next? Let’s start the competition events!

Event 1: Full Body Movement (a.k.a. Opening Doors)

Doors are tricky due to the asymmetry of force: you need to grip and twist with force, but if you pull with force outside the arc of the door, it often slips. Additionally, they require full-body control, which exceeds the capabilities of any robot I have seen so far.

Bronze Medal: Pushing a round door handle.

I believe this is very close to the cutting edge of technology (or may have already been achieved, but I just haven’t seen it). I expect this project to be completed by December.

Silver Medal: Automatic closing push door with a lever handle.

Adding an automatic closing feature makes this more challenging, although lever handles can be said to be somewhat easier (I just haven’t seen many automatic closing doors with round handles).

Gold Medal: Automatic closing pull door with a lever handle.

Using one arm to pull the door open, then quickly blocking the door with your body or the other arm to prevent it from closing, accurately completing the series of coherent actions of “pulling open, sidestepping, and passing through” before the door closes.

Event 2: Laundry

We are just getting started with laundry.

Bronze Medal: Folding a T-shirt inside out

Using our existing technology might make this possible, but it is a longer-term task that may require some tricky hand movements to turn the shirt right side out.

Silver Medal: Turning a sock inside out

I think the action of inserting the hand and pinching the inside of the sock is an interesting new challenge.

Gold Medal: Hanging a men’s dress shirt

The medium-sized shirt starts unbuttoned, with one sleeve inside out. It must be hung correctly on the hanger, with the sleeve fixed and at least one button fastened. I believe this project will take 3-10 years because buttoning is really difficult, and creating a hand that is strong, dexterous, and small enough to fit into the sleeve will be challenging.

Event 3: Basic Tool Use

Humans are products of technology; although our hands are useful, we primarily use them to grasp and manipulate tools. This challenge is about building strength and dexterity in tool use.

Bronze Medal: Using glass cleaner and paper towels

The design of the ammonia glass cleaner bottle allows for a high tolerance of gripping posture, but the spraying operation requires the user to apply force independently with one finger, which has certain requirements for finger strength.

Silver Medal: Making a peanut butter sandwich

The challenge here is to pick up a knife and then adjust the grip to make it strong and stable enough to scoop and spread peanut butter. Humans use “power tool grips” in various activities, but this is very challenging for robotic grippers.

Gold Medal: Using a key

A keyring with at least two keys and a keychain is placed in the robot’s waiting hand/gripper. Without dropping the keys, the correct key must be aligned, inserted, and turned in the lock. This requires very challenging internal hand manipulation, high precision, and interesting force interactions.

Event 4: Finger Skills

We humans utilize the structure of our hands to manipulate the objects we hold, performing various internal hand manipulations.

Bronze Medal: Rolling paired socks.

This requires dexterity and some precision, but not a lot of strength.

Silver Medal: Using a dog poop bag

When I use a dog poop bag, I have to perform a sliding action between my fingertips to separate the opening of the bag, which is a tricky force interaction, and one that I am not even sure most robotic hands can accomplish. Equally tricky is tearing off a bag instead of pulling a long string from the dispenser, if you choose to use a dispenser.

Gold Medal: Peeling an orange

Completing this without external tools is very tricky: it requires high force but also high precision fingertip actions.

Event 5: Slippery Environments

If you sit down and write out the tasks you might want a robot to do for you: many tasks will ultimately involve water. Robots generally do not like getting wet: but if we want them to clean for us, we must change that.

Bronze Medal: Wetting a sponge at the sink and wiping the countertop

Lightly damp, but if not careful, the entire hand risks getting wet. It may require at least splash-proof hands (or many spares).

Silver Medal: Cleaning peanut butter off your robotic hand

This naturally follows the sandwich project. There is water everywhere. After collecting several hours of training data on the dog poop task, this seems to be an important skill.

Gold Medal: Washing grease off a frying pan with a sponge in the sink

Water, soap, grease, and an unpleasant task that no one wants to do.

Original article link:https://substack.com/inbox/post/173034465