What is SFT?

SFT teaches format and behavior by imitation, while RLHF teaches preference by comparison.

Type: post-training method. Also written: supervised fine-tuning.

Read more: data types / sft data.

fine-tuning

Fine-tuning is the further training of a pretrained model on a smaller dataset to adapt it to a task or style.

RLHF

RLHF, or reinforcement learning from human feedback, is a training method in which human preference judgments are used to train a reward model that then guides a language model.

post-training

Post-training is the set of steps after pretraining, such as supervised fine-tuning and reinforcement learning, that shape a model's behavior.

• Data type: Supervised fine-tuning data
• Guide: RLHF vs SFT: what is the difference?
• Guide: How AI training data is priced