Model collapse
By Gareth HoyleLast updated How we research
Model collapse is the loss of quality and diversity that happens when generative models are trained repeatedly on content produced by other models.
Model collapse is a degenerative process in which generative models trained on data produced by earlier models gradually lose information about the original data, starting with rare cases. A July 2024 paper in Nature by Ilia Shumailov and colleagues found that indiscriminate training on model-generated content "causes irreversible defects in the resulting models, in which tails of the original content distribution disappear".
Why does model collapse matter for the data market?
Model collapse matters because it raises the value of data that is verifiably written or made by people. As more web text is written by models, archives of real company records, books and conversations become harder to replace.
It does not mean synthetic data is useless. Synthetic examples are widely used when mixed with real data and filtered. The risk comes from replacing real data with generated data over successive rounds.
What should sellers document?
Sellers should document creation dates, authorship and whether any content was machine-generated, so buyers can confirm the share of human-written material. See data provenance and synthetic data vs licensed real data.