How Humanoid Robots Learn: Teleoperation to AI Models

A humanoid robot does not come out of the factory knowing how to fold a towel or climb a step. Every skill it has was either written by engineers as equations, discovered through millions of simulated attempts, or copied from people who showed it what to do. This guide walks through each method, where it breaks, and why data is the problem everyone in the field is trying to solve.

Three ways a skill gets into a robot

It helps to sort robot skills by where they came from:

Designed. An engineer writes a model of the robot's physics and a controller that uses it. Nothing is learned from data.
Discovered. The robot tries things in a simulator, gets a score for each attempt, and slowly finds behavior that scores well. This is reinforcement learning.
Demonstrated. A person performs the task, usually by remote-controlling the robot, and a model learns to reproduce what the person did. This is imitation learning.

A single robot often uses all three at once: a designed low-level controller for joint torque, a discovered policy for walking, and a demonstrated policy for handling objects.

Classical control: physics you write down

Before learning methods took over, legged robots walked on math. Engineers modeled the robot as masses connected by joints, predicted how forces would move that body, and solved for the joint torques that kept it on its feet. Common tools include zero moment point methods, which keep the robot's center of pressure inside the area its feet support, and model predictive control, which repeatedly plans a short window of future motion and applies only the first step before planning again.

Classical control has real strengths. It is predictable, it can be analyzed before the robot moves, and when it fails, engineers can usually say why. Its weakness is that it only knows what the model contains. Slippery floors, soft ground, an unexpected push or an object that is heavier than listed can push the robot outside the conditions the equations assumed. Writing a model detailed enough for every kitchen and warehouse is not practical, which is why learning methods moved in.

Reinforcement learning in simulation

Reinforcement learning (RL) replaces the hand-written controller with a neural network, called a policy, that maps what the robot senses to what its motors should do. The policy starts out random. During training it acts, receives a reward (stay upright, move forward at the commanded speed, use little energy), and updates itself to earn more reward next time.

Doing this on a real robot would take far too long and break a lot of hardware, so training happens in a physics simulator. Modern GPU-based simulators run many copies of the robot side by side, so a policy can collect far more practice than a physical robot could in the same time.

A clear example is the humanoid walking controller from researchers at UC Berkeley described in Real-World Humanoid Locomotion with Reinforcement Learning. The team trained a transformer-based policy "with large-scale model-free reinforcement learning on an ensemble of randomized environments in simulation" and deployed it on a real humanoid "zero-shot," meaning with no further training on hardware. The robot walked over outdoor terrain and handled pushes.

The sim-to-real gap

A simulator is only an approximation. Real motors have friction, backlash, heat and delay that the simulated ones lack. Real cameras have noise and glare. Real floors vary. A policy that exploits a quirk of the simulator can look perfect in training and fall over on the first real step. This mismatch is called the sim-to-real gap, or the reality gap.

The most common fix is to refuse to let the policy get comfortable. In domain randomization, introduced by Tobin and colleagues in 2017, the simulator varies its own settings on every run. The original paper randomized what the camera sees, such as textures and lighting, and later work applied the same idea to physics settings like friction and mass. The authors put the idea this way: "With enough variability in the simulator, the real world may appear to the model as just another variation." Teams also measure their real actuators and build those measurements into the simulator, and they fine-tune policies with small amounts of real-world data.

RL in simulation works best for skills where the physics is the hard part and a reward is easy to write, such as walking, balancing and recovering from a stumble. It struggles with tasks where success is hard to score automatically, such as tidying a room.

Imitation learning from teleoperation

For manipulation tasks, many teams show the robot what to do instead. A person controls the robot remotely through a virtual reality headset, a motion-capture setup or a pair of small "leader" arms whose movements the robot's arms copy. While the person works, the system records what the robot's cameras see, the positions of its joints and the commands sent. A neural network then learns to predict the operator's next action from the robot's current view. This is called behavior cloning.

The catch is compounding error. A cloned policy makes a small mistake, lands in a situation the demonstrations never covered, and has no idea how to recover. Good teleoperation data therefore includes corrections and varied starting positions, not just clean runs.

Results can be striking with modest data. The ALOHA project from Stanford and collaborators, described in Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware, used inexpensive arms and a custom teleoperation rig. Its method, Action Chunking with Transformers, predicts short sequences of actions instead of one step at a time, and the paper reports 80 to 90 percent success on six difficult tasks, such as opening a translucent condiment cup and slotting a battery, from only 10 minutes of demonstrations.

The tradeoff is that every demonstration costs human time, and a policy trained for one task in one setting rarely transfers to a new kitchen without more data.

Vision-language-action models

The newest approach borrows from large language models. A vision-language-action (VLA) model takes camera images and a plain-language instruction ("put the apple in the bowl") and outputs robot actions directly.

Google DeepMind's RT-2, introduced in July 2023, showed the core trick. It started from a model pretrained on web images and text and represented robot actions as strings of tokens, the same units the model already used for words. It was trained on web data plus robot demonstrations collected over 17 months with 13 robots. Because it inherited knowledge from the web, it could follow instructions involving objects and concepts that never appeared in its robot data.

Humanoid builders have adapted the idea for full bodies. Figure AI's Helix, announced in February 2025, splits the work in two. A 7-billion-parameter vision-language model runs 7 to 9 times per second to understand the scene and the instruction. A much smaller 80-million-parameter network turns that understanding into continuous control of the whole upper body, 35 degrees of freedom including individual fingers, 200 times per second. Figure says Helix was trained on about 500 hours of teleoperated demonstrations.

That split between a slow model that reasons and a fast model that moves shows up across the field, because a large model is too slow to run a robot's reflexes on its own.

The data bottleneck

Language models learned from a large share of the public internet. There is no internet of robot actions. Every demonstration has to be produced by a person, a simulator or a deployed robot.

The largest open pooling effort shows the scale gap. The Open X-Embodiment project gathered data from 22 different robots through a collaboration of 21 institutions, covering 527 skills and more than 1 million real robot trajectories. That is a landmark for robotics and still tiny next to the text and image collections behind today's chatbots.

Each source of new data has a cost:

Teleoperation gives exactly the right kind of data but scales with paid human hours.
Simulation is cheap and endless but carries the reality gap, and cloth, liquids and clutter are hard to simulate accurately.
Video of people is abundant, but it has no motor commands attached, and human bodies differ from robot bodies.
Fleet data from robots at work is realistic but only exists once robots are deployed, and customers may not want it shared.

Questions to ask about any robot learning demo

When a company shows a humanoid doing something new, a few questions separate progress from staging:

Was the robot acting on its own, or was a person teleoperating it during the clip?
Was the video sped up, and how many attempts did the successful one take?
Were the objects, room and lighting the same ones used in training data?
How many hours of demonstrations went into the skill, and does it transfer to a new site without more?
What is the success rate over many runs, not just the one on camera?

A clear answer to the last question is the most useful thing a robotics team can give you.