ReST-MCTS* (Process-Reward-Guided Tree Search Self-Training)

Replaces independent flat sampling in the E-step of $\text{ReST}^{\text{EM}}$ with Process-Reward-guided Monte Carlo Tree Search ($MCTS^\star$), using inferred step-level values from tree backup to simultaneously train both the policy $\pi_\theta$ and the Process Reward Model $Q_\phi$ in a mutually reinforcing loop.