risk · R10
Model Collapse and Data Scarcity Crisis
Repeated training on uncurated AI-generated data can degrade model quality. Growing volumes of synthetic online content and uncertain training-data provenance raise quality-control concerns, while preserving reliable human-generated data remains important.
Updated
Observatory assessment: 68 / 100. High
Deep dive
Training-data quality matters as AI-generated text, images and annotations become more common. A 2024 Nature study by Shumailov and colleagues demonstrated model collapse under indiscriminate recursive training on model-generated data, including loss of information about less common examples. This is a risk from particular training conditions, not proof that every use of synthetic data causes collapse.
The attached reporting describes low-quality synthetic content online and concerns about AI-assisted training-data production. These reports raise provenance and quality-control questions; they do not establish that deployed frontier models have already collapsed. Access to reliable human-generated data, provenance checks and independent evaluation of synthetic datasets remain important safeguards.
Trajectory and projections
Near term
By 2026, major AI labs acknowledge diminishing returns from scaling alone. Data licensing costs exceed compute costs for some training runs.
Mid term
By 2030, 'Data Commons' initiatives create curated, verified human datasets. Synthetic data techniques mature but require careful validation.
Long term
By 2035, AI development shifts from data-hungry approaches to more efficient architectures (neurosymbolic, few-shot). The 'data crisis' is seen as a temporary growing pain.
Mitigation strategies
- Investment in curated, high-quality public training datasets
- Development of synthetic data validation and quality standards
- Fair compensation frameworks for human data contributors
- Research into data-efficient model architectures
- Public pressure and platform policies for labeling, flagging, or banning low-quality AI-generated content ('slop')
Top evidence
Hollywood is hiring ‘slop janitors.’ The price is visible.
AI Incident Database
The article discusses the emergence of 'slop janitors' in Hollywood, workers tasked with cleaning up low-quality, AI-generated content. This highlights the current limitations of generative AI in producing high-quality content without significant human intervention and the economic implications for creative professionals.
The epistemic debt of generative AI
Nature Machine Intelligence
The concept of "epistemic debt of generative AI" suggests a growing risk related to the reliability and truthfulness of information produced by AI systems. This could lead to an amplification of disinformation, degradation of knowledge quality through model collapse, and an erosion of trust in digital information.
Will the Open Internet Survive the War on Bots? Mapping the Debate Over “AI Preferences”
Center for Democracy & Technology
A conference will bring together experts to debate the future of the open internet in the context of 'AI Preferences' and the 'War on Bots.' This discussion addresses risks like AI-amplified disinformation, data scraping, and model collapse, while seeking governance solutions for the internet's integrity.
Can Generative Intermediaries Deliver When It Comes to Information Quality?
Center for Democracy & Technology
The article questions whether generative AI intermediaries can maintain information quality, highlighting concerns about how these new tools are reshaping information access. This raises risks regarding the reliability and accuracy of content consumed by the public.
AI Slop Melodramas Are Taking Over X—and Their Creators Are Cashing In
Wired AI
AI-generated "melodramas" are becoming viral clickbait on X, racking up millions of views and generating revenue for their creators. This trend highlights the proliferation of low-quality, AI-generated content designed for engagement, contributing to information overload and potentially distorting online discourse.