What is pretraining?

Pretraining data is usually broad and licensed or collected at scale. It is followed by post-training.

Type: training stage.

post-training

Post-training is the set of steps after pretraining, such as supervised fine-tuning and reinforcement learning, that shape a model's behavior.

training data

Training data is the set of examples a machine learning model learns from.

data licensing

Data licensing is the grant of rights to use a dataset for stated purposes, in return for payment or other terms.

• Guide: Who buys AI training data?