Mechanism

When a hard problem has 0%0\% pass rate under current policy sampling (pass@G=0\text{pass}@G = 0), pure RLVR receives zero positive signal. Hybrid SFT-RL (LUFFY / UFT / Prefix-RFT) injects off-policy expert prefixes y1:k⋆y_{1:k}^\star (from human/teacher traces) or mixes off-policy expert trajectories into the GRPO group using regularized importance sampling, smoothly annealing prefix length k→0k \to 0 as the policy learns to solve the task on its own.

References

  1. Learning to Reason under Off-Policy Guidance ↗

    Jianhao Yan, Yafu Li, Zican Hu, Zhi Wang, Ganqu Cui, Xiaoye Qu et al. · 2025

    All 8 authors

    Jianhao Yan, Yafu Li, Zican Hu, Zhi Wang, Ganqu Cui, Xiaoye Qu, Yu Cheng, Yue Zhang

  2. UFT: Unifying Supervised and Reinforcement Fine-Tuning ↗

    Mingyang Liu, Gabriele Farina, Asuman Ozdaglar · 2025