---
title: 'AI Worms: Autonomous Self-Propagating Malware'
url: https://www.emergentmind.com/topics/ai-worms
type: topic
---

# AI Worms: Autonomous Self-Propagating Malware

AI worms are autonomous, self-propagating malicious artifacts that exploit artificial intelligence techniques—most notably large language models (LLMs) or generative AI (GenAI) systems—for code metamorphosis, stealth, social engineering, zero-click propagation, or other intelligent evasion and attack behaviors. In contrast to classical computer worms, AI worms leverage semantic, contextual, or reasoning capabilities of AI to amplify their propagation and evasion efficacy, as well as to subvert emerging AI-enabled application ecosystems [2406.19570, 2403.02817]. Defensive research also considers AI-inspired models for detection, using immunological processes to characterize and counter automated worm propagation [1001.2155].

## 1. Architectures and Attack Models

AI worms exhibit system architectures integrating multiple AI-driven stages, distinct from classical malware pipelines. For example, "Synthetic Cancer" [2406.19570] introduces a four-stage worm: initial host infection via an email attachment, LLM-based metamorphic refactoring of the worm code, payload generation (compilation/bundling), and LLM-driven social engineering for automated propagation. Key attacker resources include access to high-capacity LLMs (local or API), endpoints with installed e-mail clients (e.g. Outlook COM automation), and gap-closings between classical and behavioral anomalies.

Zero-click prompt-injection worms—such as Morris II [2403.02817]—target ecosystems of GenAI-powered applications (RAG-based or flow-steering agents) without requiring user interaction. By leveraging prompt injection and contextual chaining in Retrieval-Augmented Generation (RAG) workflows, propagation occurs automatically, with each agent potentially forwarding the worm to new targets or data sources.

## 2. AI Techniques for Propagation and Stealth

AI worms employ the following core techniques:

- **Metamorphic code rewriting (via LLMs):** Each propagation cycle, the worm uses an LLM to refactor its own source, selecting rewrites that maximize semantic fidelity (cosine similarity in embedding space) while minimizing static signature detection (heuristic DetScore). The overall candidate scoring is:
  $$
  \mathrm{score}(c) = \alpha \cdot \mathrm{sim}(\mathrm{Emb}(S), \mathrm{Emb}(c)) - \beta \cdot \mathrm{DetScore}(c)
  $$
  where $S$ is the current codebase and $C$ is the set of candidate rewrites. This process results in every copy of the worm being syntactically unique [2406.19570].

- **LLM-driven social engineering:** Autonomously generated phishing emails are customized using LLMs. Conversation context and user-specific details are included by extracting e-mail threads from Outlook and using few-shot prompting to produce replies with high likelihood of inducing click-through (estimated $p_\mathrm{click}(e) \approx 0.25$–$0.35$ for well-crafted spoofs) [2406.19570].

- **Prompt self-replication and chaining (GenAI ecosystems):** Morris II [2403.02817] demonstrates adversarial prompts $x$ such that $G(w \Vert x \Vert y) \to \text{payload} \Vert x$, causing output echoing and chain propagation in semi-autonomous agent ecosystems. Reliance on RAG context retrieval (cosine-similar embedding queries) means that successful retrieval and prompt injection automatically enable further hops.

## 3. Formal Propagation Dynamics

Propagation rates and infection tree dynamics for AI worms are formalized using branching process models:

- **Classic exponential branching:** With replication rate $k$ and per-mail clickability $p_\mathrm{click}$, $R_0 = k \cdot p_\mathrm{click}$. $R_0 > 1$ yields "supercritical" worm growth [2406.19570].
- **RAG-based GenAI:** Per-hop propagation probability is $P_{prop}(k) = SR_{retrieval}(k) \times SR_{replication+payload}(k)$. For agent chains, the expected number of affected hosts is $E[\text{hop count}] \approx 1 / (1 - P_{prop}(k))$ [2403.02817].

Empirical measurement places RAG propagation rates in the 0.05–0.48 range (Gemini, $k=5$ to $k=30$ context size), with replication+payload success (echoed jailbreak and exfiltration) often exceeding 0.8 per successful retrieval. Standard e-mail worms with LLM-driven phishing yield $R_0 \approx 1.5$ for $k=5$, $p_\mathrm{click}=0.3$.

## 4. Detection, Countermeasures, and Immune-Inspired Defenses

Conventional signature-based detection is subverted by AI-worms' code polymorphism; each instance is unique post-LLM-refactoring. Main defense mechanisms include:

- **Network-level LLM API monitoring:** Thresholding LLM API calls or flagging high-entropy JSON conversations indicative of LLM involvement.
- **Process-level anomaly detection:** Identifying joint patterns (COM automation plus high-frequency LLM requests) and modeling sequences of API calls as Markov chains for anomaly scores [2406.19570].
- **Adversarial fine-tuning of LLMs:** Training LLM-integrated malware detectors to recognize and refuse malicious "refactor" requests.
- **Code watermarking:** Embedding detectable but irreducible markers $W$ in LLM-generated code, enabling defensive scanning for new variants.
- **Policy hardening:** Enforcing signed-executables-only and disabling e-mail client scripting by default.

Immune-inspired approaches, such as CARDINAL [1001.2155], leverage artificial immune system frameworks: 
- **Danger signal extraction** from application metrics,
- **Effector selection (CTL/Th1/Th2) based on immune signal thresholds,**
- **Peer-polling and dynamic clone count adjustment** to match worm infection growth.

CARDINAL’s mapping of T-cell maturation, differentiation, and proliferation underpins a decentralized, adaptive defense approach operating over a networked set of hosts.

## 5. Evaluation and Empirical Results

Empirical data indicate high evasion and propagation rates:

| Worm Family           | Detection Evasion (TPR/FNR) | Propagation Rate/Generation | Replication Mechanism                     |
|----------------------|-----------------------------|-----------------------------|-------------------------------------------|
| Synthetic Cancer     | TPR ≈ 0.05, FNR ≈ 0.95      | $R_0 \approx 1.5$           | LLM code refactoring + phishing e-mail    |
| Morris II (Gemini)   | —                           | 0.05–0.48                   | Adversarial self-replicating prompt, RAG  |

The Synthetic Cancer prototype achieved two-hop replication, with the third hop failing due to LLM safety filter triggers. Replication rates per host were measured at ~0.3/hour. For RAG-based zero-click worms, propagation is highly sensitive to both the success of contaminated email retrieval ($SR_{retrieval}$) and model echoing, as well as to the size and position of the attack vector within RAG context windows [2406.19570, 2403.02817].

## 6. Comparative Analysis and Future Risk

AI worms raise distinct security concerns in both classic and modern GenAI application domains. Metamorphic techniques render malware lineage indistinguishable to static detectors; LLM-driven social engineering circumvents traditional phishing countermeasures by automating and customizing attacks at scale. In GenAI-powered application ecosystems, adversarial prompt-based worms subvert agent chaining and database retrieval logic, creating brittle points of compromise not present in legacy software. Current LLM safety filters are insufficient—both in code refactoring and in prompt-based payload detection. Research trends emphasize building LLM-aware detection, AI-immune frameworks, and embedding policy controls at both network and application endpoints [2406.19570, 2403.02817, 1001.2155].

A plausible implication is that as LLMs and autonomous agents become more widely integrated, the attack surface for AI worms will expand, potentially yielding new classes of non-executable, semantically driven worm vectors.

## 7. Open Problems and Limitations

Several gaps persist in the modeling and mitigation of AI worms:

- No published quantitative detection benchmarks exist for immune-inspired defenders (e.g., CARDINAL remains a conceptual blueprint with no explicit anomaly scoring or empirical performance) [1001.2155].
- Robustness of countermeasures (e.g., code watermarking, adversarial LLM training) against adaptive, multi-modal AI worms remains untested in open ecosystems.
- For RAG-based GenAI worms, empirical propagation rates and recall vary with embedding model and RAG configuration, indicating the need for ecosystem-specific threat modeling [2403.02817].
- Defensive research has not yet demonstrated generalizable LLM-robust guardrails that prevent prompt re-injection and zero-click propagation without loss of system utility.

This suggests that escalating complexity in AI-empowered systems outpaces current pace of defensive instrumentation. Ongoing work is required to close this gap and ensure resilience of future AI-integrated software ecosystems.

Source: https://www.emergentmind.com/topics/ai-worms