risk · R39
AI-Enabled Biological Threat Development
AI tools are lowering the barrier to designing and creating novel biological threats, such as new viruses or toxins. This capability, which could be exploited by malicious actors, poses a significant risk to global health and security by making the development of biological weapons more accessible.
Updated
Observatory assessment: 55 / 100. High
Deep dive
The convergence of artificial intelligence and biotechnology presents a new class of catastrophic risk. Advanced AI models, particularly those trained on vast biological datasets, are demonstrating the ability to design novel proteins, generate viral genomes, and identify ways to bypass existing biosecurity screening protocols. While these capabilities hold immense promise for medicine and scientific discovery, they are inherently dual-use.
Signals indicate that AI can be used to generate recipes for biological weapons and design pathogens with enhanced virulence or transmissibility. This dramatically lowers the technical expertise and resources required for a state or non-state actor to develop a sophisticated biological threat. Incidents and research papers show AI designing new viruses and LLM agents being tested on their ability to acquire materials and knowledge for malicious biological purposes. The risk is not just theoretical; it represents a tangible shift in the threat landscape.
This hazard is distinct from other AI-related risks. It is not about autonomous kinetic weapons (R05) or digital cyberattacks (R17), but about the use of AI as a cognitive tool to create physical, biological agents of harm. The potential for an AI-designed pathogen to cause a global pandemic represents a unique and severe existential threat that requires specific governance, containment strategies, and biosecurity frameworks that are currently underdeveloped.
Trajectory and projections
Near term
AI models are already being used to design novel proteins and bypass screening mechanisms in research settings, with early warnings about dual-use potential.
Mid term
The capabilities for AI-guided pathogen design could become more widespread, potentially accessible to well-resourced non-state actors, increasing the risk of a limited-scale bioterror event.
Long term
Highly autonomous AI systems could independently design and guide the synthesis of novel, highly contagious pathogens, posing a global catastrophic or existential risk.
Mitigation strategies
- Biological data watermarking and provenance tracking (e.g., DeepMind SynthID Bio) to identify and trace AI-designed synthetic proteins and genomic sequences.
- Automated screening and blocking protocols at model providers to prevent high-risk biological queries, such as gain-of-function pathogen design.
- Comprehensive national and institutional biosecurity frameworks, such as RAND resilience recommendations and expanding screening mechanisms to agricultural biosecurity.
Top evidence
Introducing SynthID Bio
DeepMind Blog
DeepMind is introducing SynthID Bio, a proof-of-concept technology for watermarking AI-generated proteins while maintaining their biological function. This innovation aims to provide provenance and integrity for AI-created biological data.
AI biosecurity must include agriculture
Science (AAAS)
The article emphasizes the critical need to expand AI biosecurity measures to include the agricultural sector. This highlights a potential vulnerability where AI could be exploited to create or manage biological threats impacting global food systems.
Pretraining data filtering in open-weight biological models
UK AI Security Institute
This research evaluates the effectiveness of pretraining data filtering as a safeguard for open-weight protein models, finding it effective and selective but potentially vulnerable to fine-tuning. It highlights the challenges in ensuring the durability of safety measures against advanced AI manipulation, particularly in sensitive biological applications.
Can AI coding agents rebuild biological AI tools from a paper?
UK AI Security Institute
This article investigates whether advanced AI coding agents can reconstruct biological AI tools from limited published information, assessing the implications for the effectiveness of managed access controls. This research explores the potential for AI to autonomously develop sensitive technologies, raising concerns about misuse and the effectiveness of current safeguards.
Opinion: AI is eroding the barriers that kept biological weapons rare
STAT News
A STAT opinion argues that AI may lower barriers to biological misuse and cites attempts to seek hazardous assistance that providers blocked. Its warnings about future bioweapon creation are projections, not documented successful attacks.