Mechanism
Bi-level online preference optimization objective that adds an optimism term to the implicit reward during online DPOpair selection, actively seeking out high-potential unseen responses rather than greedily exploiting the current reward estimate.
References
- Self-Exploring Language Models: Active Preference Elicitation for Online Alignment ↗
All 9 authors
Shenao Zhang, Donghan Yu, Hiteshi Sharma, Han Zhong, Zhihan Liu, Ziyi Yang, Shuohang Wang, Hany Hassan, Zhaoran Wang