Mechanism

Scales automated process supervision efficiently using a Divide-and-Conquer Monte Carlo Tree Search (OmegaPRM) that performs binary search over a failing trajectory to locate the exact first error step in O(log⁡T)\mathcal{O}(\log T) rollouts instead of O(T)\mathcal{O}(T) linear rollouts. Qwen2.5-Math-PRM further發現 that pure MC estimation has high false-positive rates (a wrong intermediate step can still reach the right answer by luck) and intersects MC labels with an LLM-as-a-Judge critique filter (Consensus Filtering).

References

  1. Improve Mathematical Reasoning in Language Models by Automated Process Supervision ↗

    Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara et al. · 2024

    All 12 authors

    Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, Jiao Sun, Abhinav Rastogi

  2. The Lessons of Developing Process Reward Models in Mathematical Reasoning ↗

    Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu et al. · 2025

    All 9 authors

    Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin