Fine-tuning
By Gareth HoyleLast updated How we research
Fine-tuning is further training of a pretrained model on a smaller, targeted dataset to adapt its behavior or knowledge to a task or domain.
Fine-tuning is the process of continuing to train a pretrained model on a smaller, focused dataset so it performs better on a task, follows instructions or adopts a style or domain. It covers supervised fine-tuning on example inputs and outputs, and preference-based methods such as RLHF that learn from preference data.
How much data does fine-tuning need?
Fine-tuning needs far less data than pre-training, but quality matters more. Efficient methods have cut the cost further. The 2021 LoRA paper by Edward Hu and colleagues reported reducing trainable parameters by 10,000 times compared with full fine-tuning of GPT-3 175B.
Why does fine-tuning matter to data sellers?
Fine-tuning is where company records are most valuable, because support tickets, contracts, code reviews and expert workflows show how real work is done. Licenses should say whether the buyer may fine-tune for its own models only or for customers, and whether fine-tuned weights may be shared. See questions to ask an AI data buyer.