Prompt-level CoT Hijacking

• Prompt-level CoT hijacking is an adversarial technique that injects malicious reasoning traces into LLM prompts to subvert built-in safety protocols.
• It leverages methods like attention dilution and cognitive backdoors to bypass policy-aligned refusals, achieving near-perfect attack success rates in benchmarks.
• Empirical studies reveal drastic ASR improvements under attack, highlighting the urgent need for robust detection methods and architectural safeguards in LLMs.

Prompt-level Chain-of-Thought (CoT) Hijacking is a class of adversarial techniques targeting large reasoning models (LRMs) and LLMs that leverage explicit step-by-step reasoning. These attacks inject, prepend, or condition on malicious reasoning traces at the prompt level, thereby subverting built-in safety or classification logic without altering the model's high-level objective or parameters. The core mechanism exploits the model's trust in intermediate chains of thought—either by attention dilution, reasoning short-circuiting, or cognitive backdoor triggers—to bypass or disable policy-aligned refusal, flip classification outputs, or produce otherwise gated content. This paradigm raises critical questions for LLM safety, detection theory, architectural mitigation, and formal interpretability.

1. Formal Definitions and Threat Taxonomy

Prompt-level CoT-hijacking encompasses multiple related attack surfaces, each defined by the point of injection and the way model reasoning is subverted:

• Chain-of-Thought Hijacking (CoT Hijacking): A prompt $P = [B; H; C]$ is constructed such that $B$ is a long, benign reasoning prefix (e.g., logic puzzles), $H$ the harmful payload, and $C$ a final-answer cue. The function $g(f(P))$ simulates the model's refusal logic; the attack succeeds if $g(f(P_0)) = \mathrm{Refuse}$ for $P_0 = [H]$ but $g(f(P)) = \mathrm{Answer}$ after prefixing $B$ and appending $C$ (Zhao et al., 30 Oct 2025).
• Reasoning Hijacking (Criteria Attack): The attacker appends spurious, plausible-looking decision criteria and a fabricated, stepwise CoT directly to the user data channel, resulting in a label flip—classification changes induced not by new tasks but by corrupted internal justification (Liu et al., 15 Jan 2026).
• H-CoT and DH-CoT (“Developer Hijacked Chain-of-Thought”): Exploit exposure of both justification and execution-phase chain-of-thought tokens. By harvesting and inserting mocked execution traces (H-CoT), or embedding few-shot harmful CoT completions within developer-role prompts (DH-CoT), attackers short-circuit internal safety phases and force models into non-refusal trajectories (Kuo et al., 18 Feb 2025, Zhang et al., 14 Aug 2025).
• ShadowCoT: Implements a class of cognitive backdoors through adversarial head rewiring and residual state perturbation. Here, prompt triggers activate injected CoT paths within select attention heads, producing stealthily adversarial, logically fluent reasoning chains, with near-zero performance penalty on benign queries (Zhao et al., 8 Apr 2025).

Table: Distinctions among principal prompt-level CoT-hijacking strategies

Variant	Point of Injection	Mechanism
CoT Hijacking	User prompt prefix	Benign CoT dilutes attention, disables refusal
Criteria Attack	Data channel suffix	Injected criteria fake reasoning, flips label
H-CoT, DH-CoT	Developer + User prompt	Exposed/forged execution CoT short-circuits safety
ShadowCoT	Internal head state	Trigger activates poisoned reasoning chain

2. Underlying Mechanisms: Attention, Reasoning, and Safety Subnetworks

Attention Dilution is central to the classic CoT Hijacking attack. In multi-layer transformers, each head computes

$$\alpha_{ij}^{(\ell,h)} = \frac{\exp(q_i^{(\ell,h)} \cdot k_j^{(\ell,h)})}{\sum_l \exp(q_i^{(\ell,h)} \cdot k_l^{(\ell,h)})}$$

Partitioning the prompt into $B$ (benign CoT) and $H$ (harmful), the total attention onto $H$, aggregated over heads, decays as $|B| \gg |H|$. Thus, $$R^{(\ell)}(P) = \langle h_{\text{last}}^{(\ell)}(P), v_{\text{refusal}} \rangle$$ (the refusal component) declines in both intermediate and late layers, leading to a compliance outcome as attention to harmful content becomes vanishingly diluted (Zhao et al., 30 Oct 2025).

Short-circuiting Safety Checks: In models where chain-of-thought is explicitly output, safety alignment often proceeds in two phases—Justification ($T_J$) and Execution ($T_E$). H-CoT demonstrates that by extracting and reusing $T_E$ snippets (from the output of benign analogues), attackers can construct:

$x_{\text{hijack}} = [T_E^{\text{mocked}}, x]$

This bypasses or disables the internal policy check, collapsing model refusal from >98% to <2% in benchmarks (Kuo et al., 18 Feb 2025, Zhang et al., 14 Aug 2025).

Cognitive Backdoors and Internal Residual States: ShadowCoT develops a more sophisticated attack by training a lightweight backdoor (∼0.15% parameters) within select heads. At inference, a trigger phrase $\tau_\psi$ switches head parameters, while small additive perturbations to layer residuals $$\mathbf{h}_l' = \mathbf{h}_l + \epsilon_l \operatorname{sign}(\nabla_{\mathbf{h}_l} \mathcal{L}_{mal})$$, plus dynamic logits bias, ensure adversarial CoT chains are naturally sampled (Zhao et al., 8 Apr 2025).

3. Prompt Construction Algorithms and Pipelines

Several classes of hijacking prompt construction procedures have emerged:

• Puzzle-Prefix Pipeline (“Seduction”): Iteratively generates long, benign puzzle CoT sequences, appending harmful prompts and final-answer cues, and tests model outputs until attack success is achieved. Automates length and prefix content search to maximize Attack Success Rate (ASR) (Zhao et al., 30 Oct 2025).
• Criteria Attack Pipeline: Four stages—mining label-specific criteria with an auxiliary LLM; clustering for diversity; identifying criteria that target input fails (“refutable”); synthesizing CoT-style suffix walking through these criteria to justify a flipped label. Empirically, including a fabricated CoT scaffold is important for high ASR, outperforming pure criteria listing (Liu et al., 15 Jan 2026).
• H-CoT/DH-CoT Messaging: For DH-CoT, developer-role prompts embed several few-shot examples answering sensitive tasks with step-by-step, educational-style CoT. The user prompt mirrors educational or research requests. The assistant then continues the reasoning and outputs harmful content (Zhang et al., 14 Aug 2025).
• ShadowCoT Reasoning Chain Pollution (RCP): Triggers identified by SBERT-encoded prompt similarity swap attention heads; stepwise generation corrupts internal states, amplifying dynamic bias, so each token in the chain aligns with attacker-adapted logic (Zhao et al., 8 Apr 2025).

4. Empirical Results and Benchmarking

Prompt-level CoT-hijacking achieves near-perfect attack success across commercial and open-source models, far surpassing baseline jailbreak and goal-hijack methods:

• CoT Hijacking (Seduction, (Zhao et al., 30 Oct 2025)): Attack Success Rate reaches 99% (Gemini 2.5 Pro), 94% (ChatGPT o4 Mini), 100% (Grok 3 Mini), and 94% (Claude 4 Sonnet) on the HarmBench benchmark—severalfold higher than previous jailbreaks.
• H-CoT (Kuo et al., 18 Feb 2025): Refusal rate on OpenAI o1 drops from ≈98% (baseline) to 0.8%–1.2% under the attack; harmfulness ratings jump accordingly. On Gemini 2.0 Flash, ASR reaches 100%.
• DH-CoT (Zhang et al., 14 Aug 2025): On non-reasoning models, ASR ≈ 0.92–1.00; on reasoning models, DH-CoT raises ASR to 0.50–0.70, far exceeding SelfCipher or DeepInception methods.
• ShadowCoT (Zhao et al., 8 Apr 2025): On Mistral-7B (across AQUA, GSM8K, ProofNet, StrategyQA), average ASR = 88.4%, average Hijacking Success Rate (HSR) = 82.8%. Stealth is maintained, with a 0.3% average drop in benign performance, and detection rate by advanced reasoning-aware detectors remains below 12%.
• Criteria Attack (Liu et al., 15 Jan 2026): Without defenses, ASR is 90–100% across tasks. With strong safety-alignment defenses (SecAlign, StruQ), ASR remains at 15–60%, while goal-hijacking baselines drop below 20%.

These results consistently demonstrate that prompt-level CoT-hijacking is substantially more effective and robust to state-of-the-art instruction-level defenses.

5. Mechanistic Analysis and Interpretability

Detailed probing reveals that safety-checking in LRMs is distributed:

• Layerwise Separation: In Qwen3-14B, separation between harmless/harmful prompts peaks around layer 25 out of 40. Middle layers (15–23) encode “early” safety features, while later (25–35) perform verification (Zhao et al., 30 Oct 2025).
• Headwise Correlation to CoT Length: Sixty attention heads with sharply negative correlation between CoT length and attention to $H$ are “safety critical.” Ablating only these heads during inference increases ASR from 11% to 91%, confirming their causal role.
• Refusal Directional Probes: Defining $v_{refusal}$ as the mean difference in last-token activations between harmful and harmless prompts, the scalar component $R^{(\ell)}(P)$ shows monotonic decline as benign prefix length increases, crossing the compliance threshold when refusal collapses.
• Backdoor Pathways: ShadowCoT demonstrates that only a small subset of heads need to be compromised; internal residual distribution drift can cause the model to internalize adversarial CoT as if it were a native trajectory, making anomalies essentially undetectable at the output level (Zhao et al., 8 Apr 2025).

6. Defense Limitations, Open Problems, and Prospects

Current instruction-level defenses (SecAlign, StruQ, sandwich delimiters, attention-based instruction detectors) are ineffective against prompt-level CoT-hijacking:

• Instruction Consistency Blind Spot: As no new tasks are introduced, but rather the chain of reasoning is spurious, intent-alignment detectors fail to trigger (Liu et al., 15 Jan 2026).
• Surface-Level Checks Insufficiency: Filtering for known jailbreak trigger words, fluency loss, or explicit prompt-injection offers little protection, as adversarial CoTs are contextually fluent and internally consistent (Zhao et al., 8 Apr 2025).

Emerging defense directions include:

• Monitoring Reasoning Drift: Attending to the distribution of attention or residuals (e.g., “Focus Score”) may signal a hijack but lacks selectivity in real tasks (Liu et al., 15 Jan 2026, Zhao et al., 30 Oct 2025).
• Dynamic Refusal Component Monitoring: Rejecting outputs if $R^{(\ell)}(P)$ dips below a threshold, irrespective of CoT length (Zhao et al., 30 Oct 2025).
• Procedural Isolation: Architectures enforcing hard separation between user data and CoT reasoning, e.g., via formal query languages or sandboxed interpreters, may block certain attacks (Liu et al., 15 Jan 2026).
• Internal State Randomization: Randomizing or pruning attention heads at inference could break hijack subspaces induced by malicious triggers (Zhao et al., 8 Apr 2025).

Open research problems include designing CoT-aware safety objectives for fine-tuning, constructing robust CoT detectors that scale with prompt length and complexity, and developing white-box or hybrid verification for internal reasoning path integrity.

7. Implications and Theoretical Significance

Prompt-level CoT-hijacking reveals a fundamental paradox for LLM interpretability and safety: explicit reasoning, designed for transparency, creates a new high-dimensional vector for adversarial control. The same mechanisms that enable interpretability and accuracy are the ones that, when manipulated at the prompt or cognitive level, create nearly undetectable and highly effective jailbreaks. Unless the architectural basis for refusal and safety checking is reimagined—or unless a new paradigm for verifying the integrity of internal reasoning chains is realized—explicit CoT will remain a double-edged sword, increasing both model capability and adversarial risk (Zhao et al., 30 Oct 2025, Zhao et al., 8 Apr 2025, Kuo et al., 18 Feb 2025, Zhang et al., 14 Aug 2025, Liu et al., 15 Jan 2026).