This is the repository for Rethinking DPO: The Role of Rejected Responses in Preference Misalignment.

To align a model with BDPO, run a command like:

bash run.sh &

We modified the code based on https://github.com/huggingface/trl.