solution · S10

Synthetic Data Standards & Data Commons

Technical standards for synthetic data quality assurance and publicly governed data commons that ensure sustainable, high-quality training data for AI development.

Updated

Observatory assessment: 15 / 100. Research

How assessments are made

Deep dive

As natural training data becomes scarce and contentious, synthetic data generation and public data commons emerge as complementary solutions. Gartner projects that by 2030, synthetic data will account for the majority of data used for AI training. Companies like Mostly AI, Gretel, and Synthesis AI already offer enterprise synthetic data platforms, and NVIDIA's Omniverse generates photorealistic synthetic training data for computer vision.

Data commons represent the governance complement to synthetic data technology. Inspired by Creative Commons and open data movements, initiatives like the LAION dataset, Common Crawl, and EleutherAI's The Pile provide publicly accessible training data. However, these face quality and consent challenges. The emerging model involves curated, governed data repositories with clear provenance, consent tracking, and fair compensation for human contributors. The EU's proposed Data Act and various national data trust frameworks provide legal scaffolding for this approach.

Trajectory and projections

Near term

By 2026, synthetic data standards are published by IEEE and ISO. Major AI labs disclose training data composition for the first time.

Mid term

By 2030, public data commons provide curated, consent-tracked datasets for key domains (medical, legal, scientific). Synthetic data accounts for 40% of training data.

Long term

By 2035, a mature data ecosystem exists with clear provenance, fair compensation, and quality standards. The 'data crisis' is resolved through a combination of synthetic and curated human data.

Key players

  • NVIDIA (Omniverse synthetic data)
  • Mostly AI & Gretel (synthetic data platforms)
  • EleutherAI (open-source data and models)
  • EU Commission (Data Act framework)
  • Allen Institute for AI (open data initiatives)

Barriers

  • Synthetic data can encode and amplify biases from source data
  • Validation of synthetic data quality remains an unsolved problem
  • Data commons face intellectual property and consent challenges
  • Model collapse risk when synthetic data feeds back into training loops

Top evidence

Data Union Conference

EU Digital Strategy (EC)

This article announces a high-level conference hosted by the European Commission focused on turning Europe's data policies into more data for AI, growth, and competitiveness. It signifies a strategic effort to leverage data and policy frameworks to bolster AI development and economic standing within the EU.

FinSafeGuard: A Quality-Driven Synthetic Dataset for Financial AI Safety

arXiv CS.AI

FinSafeGuard describes generating and filtering a large synthetic dataset for banking, insurance and financial-services scenarios, using diversity controls, semantic deduplication and model-based quality judgments. It offers a reproducible data-quality methodology, not evidence that fraud or model-collapse risks have been eliminated.