Cordyceps: Covert Control Attacks on LLMs via Data Poisoning

Abstract: LLMs are often fine-tuned on uncurated text datasets that adversaries can poison. Existing poisoning attacks primarily rely on fixed trigger phrases that defenses such as outlier detection, clean-data regularization, or online monitoring can neutralize. In this paper, we propose a data poisoning method that teaches an LLM an information hiding scheme reliably and stealthily through semantic associations between shared knowledge such as facts or concepts and attacker-chosen phrases. The induced hiding scheme can encode and decode arbitrary malicious instructions, thus revealing a new and subtle poisoning-induced vulnerability: covert control attacks. We precisely characterize covert control attacks and evaluate them across $5$ LLMs, $3$ backdoor defenses, and $4$ prompt injection defenses. With a small poisoned fraction, covert control attacks outperform heuristic-based prompt injection attacks in average attack success rate by about $40\%$ relative to clean fine-tuned models. They also circumvent defenses based on detection and fine-tuning, maintaining up to $93\%$ attack success rate after backdoor defenses and up to $98\%$ after prompt injection defenses.

• The paper presents Cordyceps, a novel data poisoning framework that embeds covert semantic channels in LLMs to enable arbitrary instruction injection and data exfiltration.
• It employs a three-phase attack methodology—semantic association, reasoning trace generation, and instruction formatting—to create stealthy poisoning samples immune to standard trigger-based defenses.
• Empirical evaluations reveal over 70% effectiveness in prompt injection and up to 93% covert accuracy, underscoring the urgent need for adaptive defense mechanisms.

Covert Control Attacks via Data Poisoning in LLMs: An Expert Analysis of "Cordyceps" (2605.26595)

Introduction and Motivation

"Cordyceps: Covert Control Attacks on LLMs via Data Poisoning" introduces a new paradigm in LLM security, characterizing covert control attacks that leverage stealthy data poisoning to implant semantic hide-and-retrieve channels within fine-tuned LLMs. Unlike prior trigger-based backdoor attacks, Cordyceps teaches the model to internalize a context-conditioned Semantic Hiding under Shared knowledge (SHuSh) scheme, enabling arbitrary instruction encoding and decoding through semantic anchors and attacker-controlled stegotexts. This approach fundamentally alters the attack surface by subverting conventional defenses reliant on explicit triggers, lexical outlier detection, or prompt structure.

The practicality of this attack stems from realistic threat models: fine-tuned LLMs often ingest uncurated data, and supply-chain vulnerabilities permit adversaries to poison publicly-available datasets. Cordyceps capitalizes on these assumptions by requiring only poisoned data, not modification of base training or inference-time decoding, and operates within standard instruction-tuning frameworks.

Figure 1: Two covert injection attack scenarios enabled by Cordyceps—covert prompt injection via concatenation with benign instructions and data exfiltration via bidirectional SHuSh.

Cordyceps Attack Framework

Cordyceps implements its data poisoning via a three-phase algorithm:

1. Semantic Association Phase: The attacker pairs plaintext instructions with selected semantic anchors (e.g., factual Wikipedia entries), using an oracle LLM to generate stegotexts that encode the plaintext within the semantic context of the anchor, ensuring alignment and plausibility.
2. Reasoning Trace Generation: Chain-of-Thought (CoT) prompts are used to elicit full encoding and decoding rationale from the oracle, providing detailed logic on how semantic associations map to and recover the hidden instruction, increasing robustness against paraphrasing and facilitating generalization.
3. Instruction Formatting Phase: Bidirectional poisoned samples (encoding and decoding templates) are composed for fine-tuning, teaching both message hiding and recovery procedures.

   Figure 2: Cordyceps algorithm overview, illustrating attacker message/anchor pairing, stegotext generation, reasoning traces, and poisoned data assembly.

The innovation lies in teaching the LLM a distributed semantic mapping, rather than a fixed lexical correlation. SHuSh establishes an additive bias in the model's latent semantic space, allowing transferability and generalization across anchor contexts and plaintexts unseen at poisoning time.

Formalization of SHuSh

SHuSh is formalized as an encoding/decoding pair $(Enc, Dec)$ in a high-dimensional semantic embedding space. The poisoned model, given anchor $k$ and message $m$, generates stegotext $s$ such that embeddings satisfy $e(s) = e(k) + e(m) + \xi$, where $\xi$ models stochastic Gaussian noise. Decoding involves extracting the perturbation $\delta = e(s) - e(k)$ and running nearest-neighbor recovery.

Three security properties are asserted and theoretically justified:

• Stealth: KL divergence between stegotext and covertext distributions is minimized for small $\|e(m)\|$ relative to generation noise. Empirically, stegotexts are visually indistinguishable from covertexts in embedding space.
• Robustness: Paraphrasing does not disrupt the semantic channel, as the information is semantically embedded and not lexically bound.
• Encoding-Decoding Accuracy: Recovery hinges on the separation of message embeddings; error rates are bounded using the Gaussian Q-function and minimum inter-message distance.

  Figure 3: t-SNE visualization indicates stegotexts generated via Cordyceps are tightly interspersed with covertexts, resisting clustering-based detection.

Empirical Evaluation

The attacks were evaluated across prompt injection (UCC) and data exfiltration (BCC) scenarios, five LLM architectures, multiple datasets, and seven contemporary defenses (ONION, DataSentinel, PromptGuard, CROW, SecAlign, etc).

UCC Setting: Cordyceps achieves >0.7 attack effectiveness (ASV$_{h}$) in prompt injection—an 80% relative improvement over state-of-the-art prompt injection baselines, and significantly outperforms classic trigger-based backdoors which rarely exceed 0.4.

BCC Setting: Under keyword filtering defenses for data exfiltration, Cordyceps maintains up to 93% covert accuracy on Qwen3-30B, with robust generalization at poisoning rates as low as 1%. Stegotexts reliably exfiltrate sensitive data via SHuSh, where standard models fail (<6% success).

Defensive ablations reveal that ONION, PromptGuard, and DataSentinel—whether adaptive or static—fail to detect Cordyceps stegotexts beyond negligible false-positive rates. Consistency regularization (CROW) only marginally degrades UCC effectiveness (down 5.75% on Gem3-12B), while standard backdoors are sharply reduced (>20%). SecAlign has <4% impact and is often counterproductive, showing explicit defense targeting is ineffective against Cordyceps.

Utility evaluations across MuSR, GSM8K, MMLU, IFEval, and GPQA benchmarks indicate that poisoning incurs negligible performance loss (±1.5% variance), confirming that the SHuSh channel is largely orthogonal to general language competence.

Semantic Channel Mechanism and Explainability

The latent space analysis validates the additive bias model: empirical estimators for $e(m)$ align tightly with theoretical $k$0 predictions, and the channel is reversible with high fidelity. Residual error distributions are Gaussian centered, supporting the noise assumptions for decoding reliability.

Figure 4: t-SNE embeddings show stegotext vectors cluster at anchor-plus-message bias distinct from covertexts, confirming theoretical SHuSh structure.

Figure 5: Residuals in the embedding space exhibit symmetric bell-shaped distributions, validating the isotropic noise model.

Channel Capacity Analysis

Under full poisoning, Cordyceps achieves >0.75 SCS between decoded plaintexts and ground truth. Exact message recovery degrades with payload length, but semantic recovery remains reliable up to ~300 bits (~4-5 tokens). Larger LLMs retain higher exact match rates for short tokens but converge on approximate recovery for longer payloads. Channel capacity is thus length-limited, but sufficient to induce potent downstream attacks across diverse NLP tasks.

Figure 6: EMR versus information bits reveals robust exact recovery for short payloads; degradation occurs beyond 1 token (5 tokens is near-zero).

Figure 7: EMR versus token count confirms channel length-dependence across five LLMs.

Figure 8: SCS ablation demonstrates optimal reasoning weight for maximizing semantic recovery accuracy in Cordyceps training.

Implications and Future Directions

Cordyceps demonstrates that LLMs can learn stealthy, generalizable data-driven covert channels, undermining the efficacy of trigger-based detection and alignment. The attack undermines direct prompting defenses and persists after paraphrasing and aggressive fine-tuning. Theoretical coupling between channel stealth and decoding margin informs both attack tuning and new defense paradigms.

From a practical perspective, the work exposes a new class of vulnerabilities in LLM supply chains: semantic encoding/decoding mechanisms indistinguishable from benign examples and not reliant on non-trivial triggers. Existing auditing, sanitization, and inference-time filters are insufficient. Defensive directions must include provenance tracking, semantic anomaly analysis, red-team evaluations for latent semantic mappings, and layered detection across supply-chain and deployment.

Future work may generalize Cordyceps to more sophisticated semantic embedding channels, optimize for maximal stealth/effectiveness tradeoffs, and develop representation-level adaptive detectors, possibly leveraging joint inference across anchor, message, and stegotext triplets.

Conclusion

Cordyceps establishes a new data poisoning framework for covert control over LLMs, fundamentally subverting established trigger-based backdoor methodologies. By instilling a context-sensitive, semantic hiding scheme within fine-tuned models, Cordyceps enables robust, stealthy attacks surviving prevailing defensive protocols. The work highlights a deficit in contemporary model security and points toward deeper semantic understanding and adaptive detection as future requirements. Attacks of this type require ongoing re-evaluation of both LLM deployment practices and defense architectures (2605.26595).