DPO & Modern Preference Tuning

Tune models on preferences without a separate reward model using DPO — how it works, the reference model as a leash, why it took over, modern DPO variants, and where RLHF still wins.