# Supervised fine-tuning

URL: https://sellmydatatoai.net/glossary/supervised-fine-tuning

Supervised fine-tuning is further training of a pre-trained model on example inputs paired with target outputs, so it learns to produce the desired responses.

Supervised fine-tuning (SFT) is the stage in which a pre-trained model is trained further on examples of inputs paired with the outputs it should produce. The model's weights are updated to make the target outputs more likely, usually by minimizing cross-entropy loss on the response tokens.

## Where does supervised fine-tuning fit in training?

Supervised fine-tuning comes after pre-training and usually before preference tuning. In OpenAI's 2022 [InstructGPT paper](https://arxiv.org/abs/2203.02155) by Long Ouyang and colleagues, human labelers wrote demonstrations of the desired behavior, the model was fine-tuned on them and [RLHF](https://sellmydatatoai.net/glossary/rlhf) followed.

## How much data does supervised fine-tuning need?

Supervised fine-tuning needs far less data than pre-training, and quality matters more than volume. The InstructGPT supervised dataset held about 13,000 training prompts.

Company records that suit SFT include expert answers to real questions, resolved support tickets and reviewed code changes. See [expert domain data](https://sellmydatatoai.net/buy-ai-training-data/expert-domain-data) and [support ticket data](https://sellmydatatoai.net/buy-ai-training-data/support-ticket-data).