Mechanism
When a hard problem has pass rate under current policy sampling (), pure RLVR receives zero positive signal. Hybrid SFT-RL (LUFFY / UFT / Prefix-RFT) injects off-policy expert prefixes (from human/teacher traces) or mixes off-policy expert trajectories into the GRPO group using regularized importance sampling, smoothly annealing prefix length as the policy learns to solve the task on its own.
References
- Learning to Reason under Off-Policy Guidance ↗
All 8 authors
Jianhao Yan, Yafu Li, Zican Hu, Zhi Wang, Ganqu Cui, Xiaoye Qu, Yu Cheng, Yue Zhang
- UFT: Unifying Supervised and Reinforcement Fine-Tuning ↗