When AI scales influence beyond what regulation can absorb

What this risk is

Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance operations, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and collective behavior.

This subdomain captures the weaponization of AI for influence operations — not just the spread of false information (3.2) but the deliberate use of AI as a tool of political and social control.

How it occurs · Mechanisms

Causal profile: Human-caused · Intentional · Post-deployment

• Automated content generation at scale — LLMs generate thousands of persuasive pieces of disinformation per hour at negligible cost
• Synthetic persona networks — AI creates networks of fake social media accounts that appear authentic and coordinate to amplify narratives
• Targeted persuasion — AI combines psychological profiling with personalized messaging to maximize persuasive impact on specific individuals
• AI-powered mass surveillance — Facial recognition and behavioral AI enable population-scale surveillance, enabling targeted persecution of dissidents
• Automated censorship — AI content moderation tools can be weaponized to systematically suppress legitimate political speech

—

Real-world incidents

GRU Information Operations (documented by Meta, Twitter/X)

Russian military intelligence’s Internet Research Agency operated networks of AI-assisted fake accounts to interfere in US, French, German, and other democratic elections, generating millions of pieces of content.

China’s Social Credit and AI Surveillance

China’s use of AI-powered facial recognition, behavioral monitoring, and social credit systems to monitor and constrain the behavior of citizens, particularly Uyghurs in Xinjiang, represents the most extensive documented deployment of AI for population control.

OpenAI Disrupting Influence Operations (2024)

OpenAI publicly reported disrupting influence operations from Russia, China, Iran, and Israel that were using ChatGPT to generate social media content, translate text, and create personas for influence campaigns.

Mitigations · Governance

• Usage policies and enforcement — Clear prohibitions on use of AI for influence operations, with technical enforcement
• Inauthentic behavior detection — Platform-level detection of coordinated inauthentic behavior and AI-generated content networks
• Export controls on surveillance AI — Regulatory restrictions on exporting facial recognition and surveillance AI to authoritarian regimes
• Transparency reporting — Platforms report on government requests for surveillance and censorship
• Civil society oversight — Support independent monitoring of AI use in political contexts

—