SAO / V-SAO (Single-Rollout Asynchronous Optimization)

Designed specifically for long-horizon asynchronous agentic RL where group-wise sampling ($G=8$ or $16$ rollouts per prompt) creates severe tail-latency stragglers and off-policy staleness. SAO uses:

1. Single-Rollout Sampling ($G=1$): Generates only 1 rollout per prompt so rollouts stream immediately into the training buffer as soon as they finish.
2. Stabilized Value-Model Critic: Uses a learned value model $V_\psi$ to provide baseline & advantage estimates for single rollouts without group variance.
3. Strict Double-Sided Token-Level Clipping: Enforces tight symmetric/asymmetric token-ratio bounds and rejects stale off-policy tokens to maintain stability over 1,000+ asynchronous RL steps. In Ulam's V-SAO (Verified Reasoning SAO), SAO is paired with shaped proof-verification rewards after specialized research SFT (A-SFT).

Solves the fundamental bottleneck of synchronous GRPO in long-horizon SWE, terminal, and open-problem math research environments.