Buying data
Synthetic data vs licensed real data for AI training
By Gareth HoyleLast updated 6 minute readHow we research
In short
Synthetic data works best as a supplement to real data, not a replacement. Shumailov and colleagues reported in Nature in 2024 that training indiscriminately on model-generated data makes the tails of the original distribution disappear, while Gerstgrasser and colleagues found that accumulating synthetic data alongside the original real data avoided collapse. Licensed real data anchors the mix and covers domains no generator knows.
Key takeaways
- Replacing real data with each generation's synthetic output leads toward model collapse, but keeping and adding to the real data avoided it in Gerstgrasser and colleagues' experiments.
- Microsoft's Phi-4 report allocates 40% of pre-training tokens to synthetic data, and Nvidia reports over 98% of Nemotron-4 340B's alignment data was synthetic.
- Theory papers disagree on safe ratios: Dohmatob and colleagues show even 1% synthetic data can cause collapse in a regression setting.
- Synthetic data carries its own rights questions, including the generator model's terms of use.
- Buy licensed real data for rare cases, new domains, human behavior and evaluation.
Synthetic data is cheap to produce, has no rights holder to negotiate with and can be generated to order. That makes it tempting to treat as a substitute for licensed data. The research says the two do different jobs.
This guide summarizes what peer-reviewed and preprint research reports on model collapse and mixing, what model builders say about their own data mixtures and how to decide which to buy.
What is the difference between synthetic and licensed real data?
Synthetic data is generated by a model or a simulation, while licensed real data was created by people or systems doing real work and is used under a contract with its owner. Synthetic data inherits the strengths and blind spots of whatever generated it.
| Factor | Synthetic data | Licensed real data |
|---|---|---|
| Source | A generator model, simulator or rules | People and systems doing real work |
| Cost driver | Compute and verification | License fee and preparation |
| Coverage of rare cases | Limited to what the generator can produce | Includes the real tail of cases |
| Rights | Generator terms, seed data rights and privacy of any source records | The license and the seller's chain of title |
| Main risk | Collapse, errors and contamination | Rights gaps and personal data |
What is model collapse?
Model collapse is the degradation that occurs when models are trained on data generated by earlier models. Ilia Shumailov and colleagues, writing in Nature in July 2024, found that indiscriminate use of model-generated content in training causes irreversible defects in which tails of the original content distribution disappear.
The paper distinguishes early collapse, where the model loses information about rare events, from late collapse, where it converges to a distribution with little resemblance to the original. It showed the effect in language models, variational autoencoders and Gaussian mixture models.
“The value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of LLM-generated content in data crawled from the Internet.”
Does mixing real and synthetic data prevent model collapse?
Mixing real and synthetic data reduces or prevents collapse in several studies, but the safe proportion depends on the setting and the papers disagree. The key variable is whether real data is kept and added to, or replaced.
| Paper | Setting | Reported finding |
|---|---|---|
| Shumailov et al., Nature, 2024 | Fine-tuning a language model over generations | Keeping a random 10% of the original data each generation led to only minor degradation |
| Gerstgrasser et al., 2024 | Pre-training sequences of language models | Replacing real data with synthetic data tends toward collapse, while accumulating synthetic data alongside the original real data avoids it |
| Seddik et al., 2024 | Statistical model of language model collapse | Collapse cannot be avoided when training solely on synthetic data, and there is a maximal amount of synthetic data below which it can be avoided |
| Alemohammad et al., 2023 | Image generation in self-consuming loops | Without enough fresh real data in each generation, quality or diversity progressively decreases |
| Dohmatob et al., 2024 (Strong model collapse) | Supervised regression under scaling laws | Even 1% synthetic data can lead to collapse, where larger training sets stop improving performance |
| Dohmatob et al., ICML 2024 (A tale of tails) | Scaling laws with synthesized data | Mixing human and synthesized data changes scaling, including the un-learning of skills |
Read the settings column before applying a ratio. Dohmatob and colleagues' 1% result comes from a regression setting with theory and experiments, while Gerstgrasser's result comes from training sequences of language models. Neither tells you the right mix for your model, which is why you should test it.
What mixing ratios do model builders report?
Model builders who publish their data mixtures report heavy use of synthetic data for reasoning and alignment, alongside web, code and acquired sources. These are each company's reports about its own models.
| Model | Reported by | What the report says |
|---|---|---|
| Phi-4 (14 billion parameters) | Microsoft, December 2024 | Pre-training mix of 40% synthetic, 15% web, 15% web rewrites, 20% code and 10% acquired sources |
| Nemotron-4 340B | Nvidia, June 2024 | Over 98% of data used in model alignment was synthetically generated |
| phi-1 (1.3 billion parameters) | Microsoft, June 2023 | 6 billion tokens of textbook-quality web data and 1 billion tokens of synthetic textbooks and exercises from GPT-3.5 |
Note the 10% of acquired sources in Phi-4's mix. Even a synthetic-first recipe kept a share of licensed and organic data, which the report describes as targeted acquisitions and organic data such as academic data, books and forums.
When does synthetic data fit?
Synthetic data fits when you have a strong generator and a way to verify outputs. Code with unit tests, math with checkable answers and format-following tasks are the clearest cases.
- Tasks with automatic verifiers, such as code that must pass tests
- Rewrites and reformatting of data you already hold the rights to
- Alignment and instruction data, as Nvidia reports for Nemotron-4 340B
- Rare scenarios that are dangerous or expensive to collect, such as simulated edge cases
- Privacy-sensitive domains, where synthetic records replace direct use of personal data, subject to a re-identification check
When does licensed real data fit?
Licensed real data fits wherever the tails of the distribution matter, the domain is new to generators or the task depends on how real people and businesses behave. Shumailov and colleagues note that access to real human-produced data is needed in tasks where the tails of the underlying distribution matter.
- Professional work such as legal, accounting and engineering files, where generators lack the domain
- Real customer conversations and support tickets, which carry the errors and edge cases users produce
- Recent records created after a generator's training cutoff
- Evaluation sets, where synthetic tasks risk sharing the generator's blind spots
- An anchor set of real data to keep in every training mix
Our training data catalog lists licensed data by type, including expert domain data and support ticket data.
What rights issues does synthetic data carry?
Synthetic data is not rights-free. Its rights depend on the generator's terms, the rights in any seed data and whether personal data shaped the output.
- Generator terms: OpenAI's terms of use, for example, bar using output from its services to develop models that compete with OpenAI.
- Seed data: synthetic records derived from licensed or scraped seed data inherit questions about those seeds.
- Personal data: synthetic records generated from personal data should be tested for re-identification before you treat them as anonymous.
- Contamination: Yang and colleagues found benchmark overlap in synthetic datasets generated by GPT-3.5 and GPT-4.
For licensed real data, US copyright questions sit in the license rather than in fair use arguments. Our AI training data lawsuits tracker follows the cases that shape unlicensed use. Take independent legal advice on your own sources.
How should you decide between synthetic and licensed data?
Decide by the task, not by price. If a verifier can check the output and the generator knows the domain, start with synthetic data. If neither is true, license real data, and keep a real anchor set in any mix.
| Your situation | Better starting point |
|---|---|
| Outputs can be checked automatically | Synthetic, verified |
| The domain is specialist or proprietary | Licensed real data |
| You need rare and realistic edge cases | Licensed real data, with synthetic augmentation |
| You need more of a format you already have | Synthetic rewrites of data you hold rights to |
| You are building an evaluation set | Licensed real data or expert-written tasks |
| You train repeatedly on your own outputs | Keep and add to the real data each round |
Test the mix on a held-out evaluation before committing, as our guide to training data pilot design explains.
How do you buy real data to anchor a synthetic pipeline?
Sell My Data To AI shortlists vetted sellers of rights-cleared real data. Every seller passes our eight-point vetting standard before any buyer sees their data, and buyers pay nothing to submit a requirement or review a shortlist. Submit a data requirement or book a buyer consultation.
Frequently asked questions
Can you train an AI model only on synthetic data?
What is a safe ratio of synthetic to real data?
Is synthetic data free of copyright and privacy issues?
Why is human-created data becoming more valuable?
Sources
- AI models collapse when trained on recursively generated data, Nature (Shumailov et al., July 2024), accessed November 2026
- Is model collapse inevitable? Breaking the curse of recursion by accumulating real and synthetic data, arXiv (Gerstgrasser et al., 2024), accessed November 2026
- How bad is training on synthetic data? A statistical analysis of language model collapse, arXiv (Seddik et al., 2024), accessed November 2026
- Self-consuming generative models go MAD, arXiv (Alemohammad et al., 2023), accessed November 2026
- Strong model collapse, arXiv (Dohmatob et al., 2024), accessed November 2026
- A tale of tails: Model collapse as a change of scaling laws, arXiv (Dohmatob et al., ICML 2024), accessed November 2026
- Phi-4 technical report, arXiv (Abdin et al., Microsoft, 2024), accessed November 2026
- Nemotron-4 340B technical report, arXiv (Nvidia, 2024), accessed November 2026
- Textbooks are all you need, arXiv (Gunasekar et al., Microsoft, 2023), accessed November 2026
- Terms of use, OpenAI, accessed November 2026
- Rethinking benchmark and contamination for language models with rephrased samples, arXiv (Yang et al., 2023), accessed November 2026