Single-Rollout Asynchronous Optimization for Coding Agents: Practice in Lego-RL
Can one rollout provide enough signal to train a coding agent? SAO makes single-rollout training practical with a learned critic, warmup, and advantage whitening, achieving competitive performance with more efficient training.