Espurna
EN
LearningIntermediateRLpolicyrewardPPOsim-to-realdomain randomizationIsaac Lab

Reinforcement learning

In 30 secondsNobody tells the robot how to do it, only whether it did well: the agent tries actions, gets a reward and learns a policy that maximises it. In simulation, thousands of copies train at once.

Also available in: Català · Castellano

The loop

The agent observes the state, picks an action with its policy, and the environment responds with a new state and a reward. Goal: a policy that maximises the total reward.

Like training a dog with treats: nobody explains how, only whether it did well.

Why in simulation

Fast, parallel (thousands of copies on one GPU), safe and free to reset. A real robot breaks and only runs in real time. Typical algorithm: PPO. Tools: Isaac Lab / Isaac Gym.

From simulator to robot (sim-to-real)

Simulated physics is never identical to the real thing. Parameters are randomised (masses, friction, delays…: domain randomization) so the policy can cope with the differences.

In the demo

Press Train. Generation 0 fails; a few generations later the pole stays balanced. The demo uses a simple evolutionary method, not PPO, but the idea is the same: try, score and keep the best.

Common questions

RL or imitation? RL works well when there is a clear reward (locomotion: move forward without falling); imitation when it’s easier to show than to describe (manipulation).

Interactive demo

Related concepts

Learning

Imitation learning

A person teleoperates the robot, observation-action pairs are recorded and a policy is trained to copy them. It fails in states it has never seen; the fix is more varied demos and predicting several actions in a row.

Learning

Vision-language-action models (VLA)

A VLA starts from a model that already links images and text and adds an action output: it takes camera images, an instruction and the robot's state, and returns a chunk of movements.