What is preference data?
Preference data is a collection of human choices between model outputs that records which response people prefer.
Last reviewed
preference data in AI data
It is the core input to RLHF and to direct preference optimization.
Type: training data.
Read more: data types / rlhf preference data.
Other terms of the same type: training data
- expert demonstration: An expert demonstration is a recording of a skilled professional performing a task step by step, used to teach a model the process.
- synthetic data: Synthetic data is data generated by a model or simulator instead of collected from real people or events.
Where preference data applies on this site
- Data type: RLHF and preference data
- Guide: Who buys AI training data?
- Guide: RLHF vs SFT: what is the difference?
- Guide: How AI training data is priced
Quick answers
What is preference data?
Preference data is a collection of human choices between model outputs that records which response people prefer.
Why does preference data matter in AI data?
It is the core input to RLHF and to direct preference optimization.