Effect of generative AI on coordinated online behavior

Determine the effects of generative artificial intelligence techniques (including large language models and AI-generated multimedia content) on the landscape of coordinated online behavior, specifically how the use of these techniques by coordinated actors impacts the detectability of coordination and the assessment of account authenticity.

Background

The survey discusses emerging challenges posed by generative AI, noting that coordinated actors may leverage AI to produce human-like text, images, audio, and video and to simulate authentic online behaviors that evade existing detection systems. This raises concerns about the increasing difficulty of identifying coordination and evaluating the authenticity of accounts.

While generative AI may also enable realistic simulations useful for training and evaluating detection methods, the overall impact of such technologies on online coordination dynamics is presently unknown. Clarifying these effects is essential to adapt detection and characterization methodologies and to inform platform policies and research agendas.

References

Progress in generative AI could thus make it harder to detect future instances of coordinated behavior or to assess the authenticity of coordinated actors. However, it is still unknown what the effect of these techniques will be on the landscape of online coordination.

— Detection and Characterization of Coordinated Online Behavior: A Survey  (2408.01257 - Mannocci et al., 2024) in Section 6, Open challenges and future research directions — Generative AI

Their use in coordinated real-world campaigns, however, remains untested.

— Automated Disinformation and Malicious AI Swarms: Risks for Democracy and Development in Africa  (2610.11930 - Schroeder et al., 8 Oct 2026) in Section 1, Introduction: From Bots to Swarms in the Global South; Section 3, From Botnets to Malicious AI Swarms

These networks provide potential channels for diffusion, although whether their structure would help conceal coordination remains to be tested (Schroeder et al., 2023).

— Automated Disinformation and Malicious AI Swarms: Risks for Democracy and Development in Africa  (2610.11930 - Schroeder et al., 8 Oct 2026) in Section 5, Case Study 1: Mali as a Microcosm of Emerging Threats