risk · R22
AI-Facilitated Violence and Radicalization
AI systems, particularly conversational agents and large language models, can be exploited to assist individuals in planning and ideating real-world violence. These tools can act as both a radicalizing influence and a practical assistant for researching methods and identifying targets. This significantly lowers the technical and psychological barriers for malicious actors to commit harmful acts.
Updated
Observatory assessment: 70 / 100. High
Deep dive
The primary mechanism for this risk involves bypassing the safety filters of commercial AI models or utilizing uncensored, open-source alternatives. The vast knowledge corpora these models are trained on, which include publicly available but potentially sensitive information on chemistry, tactics, and engineering, become a readily accessible and interactive resource for those with malicious intent. The conversational nature of these systems can create a para-social relationship, validating extremist beliefs and accelerating an individual's path from radical ideology to violent action. Unlike static web pages, an AI can help a user troubleshoot plans, brainstorm solutions to logistical problems, and reframe harmful actions in a justifying narrative, acting as a virtual accomplice.
The implications for public safety are profound. This risk vector could lead to an increase in the frequency and sophistication of attacks by lone actors or small cells, who are traditionally difficult for intelligence agencies to detect. The 'conspiracy' occurs between a human and a machine, leaving a different digital footprint than communication between human conspirators, thereby challenging established surveillance and detection methods. This also creates a complex web of liability for AI developers and platform providers, who may be held accountable for the foreseeable misuse of their technologies, sparking intense legal and regulatory battles over platform responsibility and the limits of censorship.
The trajectory of this risk is closely tied to the rapid advancement of AI capabilities. As models become more powerful, multimodal, and agentic, their potential as a planning assistant will grow exponentially. A future AI could analyze satellite imagery to identify vulnerabilities in a building's security, generate custom code for autonomous drones, or craft hyper-personalized propaganda to recruit others. The proliferation of powerful open-source models makes this risk particularly difficult to mitigate through centralized governance, as control is distributed globally and development occurs in the open.
This risk represents a dangerous convergence of several other AI challenges. It is a failure of content safety and alignment, an example of the dual-use nature of powerful technology, and a direct pathway to severe, real-world harm. It transforms the abstract risk of disinformation into the concrete risk of AI-instrumentalized violence, demanding urgent attention from both a technical safety and a national security perspective.
Trajectory and projections
Near term
Jailbreaking techniques will become more sophisticated and widespread, leading to documented cases of models providing detailed instructions for harmful activities. This will intensify policy debates around open-source model liability and mandatory safety standards.
Mid term
The first credible reports may emerge of AI playing a significant role in a planned or executed violent act, triggering major regulatory interventions. Multimodal models will increase the risk by allowing planning based on diagrams, maps, and real-time data.
Long term
AI may evolve into 'agentic' accomplices capable of autonomously executing sub-tasks of a violent plan, such as social engineering or cyber-intrusions, under human direction. This will fundamentally challenge legal concepts of conspiracy and criminal liability.
Mitigation strategies
- Adversarial testing and red-teaming of AI models to proactively identify and patch vulnerabilities related to harmful use cases (e.g., bioweapons, attack planning).
Top evidence
Alarming report on AI and bioweapons divides scientists
Science (AAAS)
A new report highlighting the potential for AI to be used in the development of bioweapons has caused significant division among scientists. This raises serious concerns about the misuse of advanced AI capabilities for creating biological threats.
Maryborough teenager allegedly told police he had planned mass murder for months
AI Incident Database
A 13-year-old boy, charged with plotting a school massacre, allegedly used AI to generate a 'mass-shooting story' as part of his months-long planning. This incident highlights the severe risk of AI tools being exploited to assist in the ideation and planning of real-world violence.
Chinese hatte Tat zusammen mit ChatGPT geplant!
AI Incident Database
A Chinese student who attacked three kindergarten children in Zurich reportedly used ChatGPT to plan the assault over several weeks. This incident highlights the critical danger of AI systems, particularly large language models, being exploited as tools for individuals to research and facilitate real-world violence.
Parkview High receives 4th AI-generated bomb threat this year
AI Incident Database
Parkview High School has received its fourth AI-generated bomb threat this year, prompting investigations by state and federal authorities. These repeated incidents highlight how AI tools can be exploited to generate convincing hoaxes that cause significant real-world disruption, fear, and strain on emergency services.
Hoax phone call at Gwinnett high school causes lockdown; campus cleared
AI Incident Database
A hoax phone call, believed to be AI-generated, caused a soft lockdown at a Gwinnett high school, disrupting operations before the campus was cleared. This incident demonstrates how AI-amplified disinformation can create real-world safety concerns and operational challenges for public institutions.