Learning to Balance
The same policy before and after PPO training, tested from the same starting state.
Training Run
Reward: +1 per step. The episode ends when the pole passes ±12° or the cart passes ±2.4 m, with a limit of 500 steps (10 seconds).
About This Replay
The animation replays saved policies in Gymnasium CartPole-v1 with deterministic actions. Before training, no controller balanced for ten seconds on the reserved starts. After training, every controller balanced from all 100 starts close to upright and all 100 starts tilted six to ten degrees. Sampling pushes also gave ten seconds of balancing on all 100 separate near-upright tests per controller. The replay uses the first reserved near-upright start for each run. The untrained cart stops when the failure boundary is crossed.