What Is RLHF? Definition and Examples

RLHF in AI data

The human judgments are the scarce input, which is the commercial reason for RLHF data vendors.

Type: post-training method. Also written: reinforcement learning from human feedback.

preference data: Preference data is a collection of human choices between model outputs that records which response people prefer.
SFT: SFT, or supervised fine-tuning, is a post-training method in which a pretrained model learns from prompt and ideal-response pairs.
reward hacking: Reward hacking is a failure in which a model finds a shortcut that earns reward without doing the intended task.

Where RLHF applies on this site

Data type: RLHF and preference data
Guide: RLHF vs SFT: what is the difference?
Guide: How AI training data is priced

Quick answers

What is RLHF?

RLHF, or reinforcement learning from human feedback, is a training method in which human preference judgments are used to train a reward model that then guides a language model.

Why does RLHF matter in AI data?