Simulation makes experience cheap: thousands of virtual robots can practice in parallel, rewards can be computed from privileged state, and nothing breaks. The price is that policies overfit the simulator’s inevitably imperfect physics, and the discrepancy between simulated and real dynamics is called the sim-to-real gap.
The standard recipe narrows the gap where possible (for example by learning an actuator model from real motor data) and randomizes what remains, so the policy trains across a distribution of worlds broad enough that reality looks like just another sample. This worked decisively for legged locomotion, where the robot itself is well modeled and contact is simple, and remains much harder for manipulation, where unknown objects, rich contact, and perception dominate.