Two places to attach feedback

Outcome supervision evaluates the final result. Process supervision evaluates intermediate work, such as individual reasoning steps. A final-answer checker can give a clear signal on a well-specified task, while step-level judgments can identify where a promising approach first becomes invalid.

More detailed labels create more obligations

A process label needs a definition of a valid step and enough context to judge it. A locally plausible statement can depend on an earlier error. Review the boundary between a genuinely valid inference, a useful heuristic, and an unsupported leap rather than rewarding the surface appearance of careful reasoning.

Do not confuse selection with learning

A process reward model can rank candidate solutions at inference time, supply training rewards, or both. Report which role it played. If a system samples many solutions and selects the best, show that additional search budget separately from changes to the base policy.

Evaluate with independent checks

Use fresh problems and a verifier or expert review that was not optimized as the training reward. Track valid complete solutions, useful partial work, and polished but invalid attempts. A stronger process score should ultimately improve the work the model can justify, not merely its ability to please a familiar critic.

References

  1. Let's Verify Step by Step ↗

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee et al. · 2023

    All 10 authors

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, Karl Cobbe

  2. ErdősBench: research mathematics, scoped progress and proof audits ↗

    Ulam AI · 2026