Mechanism

Trains a binary logistic classifier separating thumbs-up completions y+y^+ from thumbs-down completions y−y^- using density ratio matching with a dynamically shifted reward baseline δ\delta, providing theoretical grounding for KTO-style binary feedback alignment.

References

  1. Binary Classifier Optimization for Large Language Model Alignment ↗

    Seungjae Jung, Gunsoo Han, Daniel Wontae Nam, Kyoung-Woon On · 2024