New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

where is config document of ipo? #81

Open

3244we opened this issue May 7, 2024 · 1 comment

3244we commented May 7, 2024

It seems that the IPO's config file is missing here, which prevents the IPO from running

Author

3244we commented May 7, 2024

do DPO preference-based training

name: ipo

the temperature parameter for DPO; lower values mean we care less about

the reference model

beta: ???

the noise parameter for conservative DPO; should be in range (0, 0.5); interpreted as

the fraction of preference pairs that are flipped

eps=0 is the original DPO loss in the DPO paper

label_smoothing: 0

if true, use a uniform (maximum entropy) reference model

reference_free: false

is this config right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment