risk · R36

AI-Enabled Financial Fraud and Economic Crime

AI is supercharging financial crime, enabling scammers to use deepfake videos and cloned voices to impersonate executives, celebrities, and family members to steal millions. These tools are also used to create vast networks of fake investment schemes and highly convincing romance scams, posing a severe and growing threat to individuals and the economy.

Updated

Observatory assessment: 75 / 100. Critical

How assessments are made

Deep dive

The proliferation of powerful and accessible generative AI tools has led to a dramatic escalation in the scale and sophistication of financial fraud. Criminals are no longer limited by the technical skills previously required to create convincing fake content. A large cluster of signals indicates a global surge in AI-enabled economic crime targeting both individuals and corporations.

One major vector is the use of deepfake video and voice cloning. Scammers are impersonating public figures like celebrities and politicians to endorse fraudulent investment schemes. More insidiously, they are cloning the voices of family members or company executives to authorize large, unauthorized financial transfers, exploiting trust in a way that traditional security measures are not equipped to handle. These scams are causing millions of dollars in losses and creating what financial regulators have called an 'emergency in the making.'

Beyond impersonation, AI is used to create and manage vast, interconnected webs of fake websites and social media profiles to lend legitimacy to fraudulent operations. AI is also being used to craft highly personalized phishing messages and to conduct romance scams at scale, with chatbots proving more effective than humans at building 'exploitable trust.' This risk is distinct from general disinformation (R03) as its primary goal is direct financial theft, and it is more specific than general cybercrime (R17) as it focuses on the use of generative AI for social engineering and deception, rather than technical exploits alone.

Trajectory and projections

Near term

Widespread use of AI for deepfake investment scams, voice-cloning fraud targeting families and businesses, and sophisticated romance scams is already causing significant financial and psychological harm.

Mid term

AI-driven fraud could become so pervasive that it undermines trust in digital communications and online financial transactions, requiring new forms of identity and transaction verification.

Long term

Autonomous AI agents could potentially run large-scale, self-sustaining criminal enterprises, conducting complex financial fraud with minimal human intervention.

Mitigation strategies

    Top evidence

    S. Hrg. 119-513 - THE AI DECEPTION MACHINE: DEEPFAKES, CHATBOTS, AND THE NEW FRONTIER OF SENIOR FRAUD

    GovInfo Congressional Hearings

    This congressional hearing title indicates a focus on how AI, specifically deepfakes and chatbots, is being used to perpetrate new forms of fraud targeting seniors. It highlights the emerging threat of AI-enabled deception in financial crime.

    Vero Beach man arrested in explicit deepfake investigation; police fear more victims

    AI Incident Database

    A man was arrested for creating and distributing AI-generated sexually explicit deepfake images of women without their consent, with police fearing more victims. This incident demonstrates the direct and severe harm caused by AI-enabled generation of illegal and harmful content, particularly non-consensual deepfake pornography.

    ‘CM Vijay’ WhatsApp call promises Rs 11 lakh aid, Bengaluru man loses Rs 1 lakh

    AI Incident Database

    A Bengaluru man lost Rs 1 lakh after being scammed by a WhatsApp video call featuring an AI-generated voice and face impersonating a chief minister. This incident demonstrates the escalating risk of AI-enabled deepfake technology being used for sophisticated financial fraud and identity impersonation.

    Victim loses £250k after scammers use AI video of 'well-known' celebrity

    AI Incident Database

    A victim lost £250,000 after being targeted by an AI-generated video featuring a well-known financial personality, highlighting the significant financial risks posed by sophisticated AI deepfake scams. This incident underscores the increasing effectiveness of AI in enabling large-scale financial crime and impersonation.

    IndicBankBench: Evaluating Safety and Reliability of Language Model Assistants in Indian Retail Banking

    arXiv CS.AI

    This paper introduces "IndicBankBench," a benchmark for evaluating the safety and reliability of language model assistants specifically in the Indian retail banking sector. The goal is to identify and mitigate risks associated with AI deployment in critical financial services.