Mechanism

Enforces first-order stochastic dominance of the implicit reward distribution of positive samples over negative samples using 1D Wasserstein optimal transport sorting across mini-batches, supporting both paired and unpaired distributional alignment.

References

  1. Distributional Preference Alignment of LLMs via Optimal Transport ↗

    Igor Melnyk, Youssef Mroueh, Brian Belgodere, Mattia Rigotti, Apoorva Nitsure, Mikhail Yurochkin et al. · 2024

    All 9 authors

    Igor Melnyk, Youssef Mroueh, Brian Belgodere, Mattia Rigotti, Apoorva Nitsure, Mikhail Yurochkin, Kristjan Greenewald, Jiri Navratil, Jerret Ross