Reasoning Extraction Attacks

• Reasoning extraction attacks are security breaches targeting intermediate computational processes, including chain-of-thought traces, hidden prompts, and decision substructures.
• They leverage both extraction and manipulation techniques—such as reconstructing hidden context and corrupting reasoning sequences—while often preserving the final answer.
• Defensive strategies require layered, process-aware measures like monitoring CoT length, resource usage, and employing advanced evaluation metrics to mitigate these vulnerabilities.

Reasoning extraction attacks are attacks that target not only what models answer, but how they arrive at those answers: the chain-of-thought trace, the structure of intermediate reasoning, hidden context that conditions reasoning, and the computational process itself. In the recent literature, this category spans both extraction and manipulation. Some attacks reconstruct hidden artifacts such as intermediate sequence outputs, system prompts, counterfactual explanation surfaces, retrieved subgraphs, or memorized continuations; others corrupt, hijack, elongate, or prematurely terminate the reasoning channel while leaving the final answer unchanged or only minimally affected (Liu et al., 13 Nov 2025, Takemura et al., 2020, Yang et al., 21 Jan 2026).

1. Scope and taxonomy of the attack surface

The common feature across these attacks is that the reasoning channel is treated as a first-class security surface. In chain-of-thought-enabled systems, the attacker may target the mapping from input to reasoning trace rather than the mapping from input to final answer. In retrieval-augmented systems, the attacker may instead control the environment that shapes reasoning. In explanation-enabled APIs, the attacker exploits post-hoc rationales as a high-bandwidth side channel for model reconstruction. This suggests that “reasoning extraction attacks” are best understood as a family of attacks on intermediate computation, not merely on visible outputs (Liu et al., 13 Nov 2025, Zhang et al., 19 Jan 2026).

Attack surface	Typical mechanism	Representative paper
Hidden internal context	Adversarial queries reveal verbatim system prompts	(Das et al., 27 May 2025)
Sequential partial computation	Query intermediate outputs halfway through sequence	(Takemura et al., 2020)
Explanation interfaces	Counterfactuals or LIME/SHAP leak local decision structure	(Aïvodji et al., 2020)
Retrieved reasoning substrate	Adaptive querying reconstructs latent GraphRAG graphs	(Yang et al., 21 Jan 2026)
Explicit reasoning traces	Inject errors, harmful triggers, or benign padding into CoT	(Peng et al., 2024)
Reasoning-cost surface	Induce overthinking, slowdown, interruption, or PI-DoS	(Liu et al., 29 Jan 2026)

A recurring distinction in this literature is between answer corruption, answer-preserving reasoning manipulation, and reasoning-cost amplification. BadThink explicitly decouples answer correctness from reasoning structure, while SEED disrupts stepwise reasoning so that the final answer becomes wrong, and OverThink or ReasoningBomb primarily weaponize inference cost rather than semantic correctness (Liu et al., 13 Nov 2025, Peng et al., 2024, Kumar et al., 4 Feb 2025).

2. Extraction of hidden context, state, and structural surrogates

One major branch of the literature extracts hidden or partially hidden structure that constrains reasoning. In recurrent models, intermediate outputs already expose partial computation. “Model Extraction Attacks against Recurrent Neural Networks” shows that a simple RNN can be extracted from an LSTM by using “outputs halfway through the sequence,” and even “without final outputs” in the classification case. The paper treats these intermediate sequence outputs as a way to approximate temporal decision logic rather than only terminal predictions (Takemura et al., 2020).

A parallel line uses explanation interfaces as local probes of the decision boundary. “Model extraction from counterfactual explanations” shows that counterfactual explanations leak non-trivial information about a black-box model and enable “high-fidelity and high-accuracy extraction even under low query budgets,” because each counterfactual provides a nearby class-flipping point and therefore localized boundary information (Aïvodji et al., 2020). AUTOLYCUS extends this logic to LIME and SHAP, using feature attributions and local surrogates to infer decision boundaries and requiring “significantly fewer queries compared to state-of-the-art attacks, while maintaining comparable accuracy and similarity” (Oksuz et al., 2023).

System-level hidden context is also extractable. “System Prompt Extraction Attacks and Defenses in LLMs” frames exact system-prompt leakage as an adversarial query problem and shows that CoT-based prompts, few-shot prompts, and an extended sandwich attack can achieve very high success rates across open and closed models. The framework operationalizes severity with ASR, Exact Match, Substring Match, cosine similarity, and Rouge-L, which makes system-prompt extraction an especially clear instance of hidden-context extraction by reasoning-aware prompting (Das et al., 27 May 2025).

In retrieval systems, the target can be a structured reasoning substrate rather than text alone. AGEA reconstructs the latent entity-relation graph of GraphRAG systems under black-box query budgets, “recovering up to 90% of entities and relationships while maintaining high precision,” and the opening technical summary further states recovery of “>95% of important hubs” in some settings (Yang et al., 21 Jan 2026). More generally, “Towards More Realistic Extraction Attacks” argues that a realistic adversary can combine prompt variants, model sizes, and checkpoints to increase extraction risk by up to $2\times$, and that under approximate matching the true risk is $2\!-\!4\times$ higher than standard single-model, single-prompt evaluations suggest (More et al., 2024).

3. Manipulation of explicit reasoning traces

A second branch does not primarily expose hidden state; it perturbs the visible reasoning trace itself. SEED is the most direct formulation of this idea. It appends adversarial reasoning steps $R_{\text{att}}$ after the original problem while leaving the solving instruction and problem statement unchanged, and then relies on the model’s tendency to continue from prior context. SEED-S modifies one step, whereas SEED-P generates a modified problem, reasoning chain, and wrong answer that remain structurally plausible. Across datasets and models, SEED achieves higher attack success rates than UPA and MPA while maintaining much lower detection rates, and the paper attributes this to the model’s over-reliance on local coherence and weak global verification (Peng et al., 2024).

PRJA targets a different property: harmfulness in the reasoning steps while preserving the answer. It first derives semantically aligned but high-risk triggers from benign reasoning, then combines those triggers with an authority-framed instruction and a moral-disengagement instruction. The result is an attack whose success criterion is explicitly answer preservation plus harmful reasoning. On five QA benchmarks and several commercial LRMs, PRJA reports an average ASR of $83.6\%$ (Wang et al., 17 Apr 2026).

Other attacks weaponize reasoning competence itself. “When ‘Competency’ in Reasoning Opens the Door to Vulnerability” argues that enhanced reasoning makes models better at decoding user-defined ciphers, creating a jailbreak vector in which the model first reconstructs a hidden harmful instruction and then executes it. Its abstract states that success rates on GPT-4o rise from $40\%$ under ACE to $78\%$ with LACE, which directly ties reasoning ability to exploitability (Handa et al., 2024). “Chain-of-Thought Hijacking” likewise pads harmful requests with long benign puzzle reasoning and then uses a final-answer cue to bypass safeguards, reaching ASRs of $99\%$, $94\%$, $100\%$, and $94\%$ on Gemini 2.5 Pro, GPT o4 mini, Grok 3 mini, and Claude 4 Sonnet, respectively (Zhao et al., 30 Oct 2025).

These works collectively show that explicit CoT is not merely an interpretability feature. It can also function as a control interface through which attackers steer, destabilize, or repurpose intermediate reasoning.

4. Cost amplification, slowdown, and denial-of-service through reasoning

A large recent subfield targets reasoning cost rather than answer semantics. BadThink formalizes a training-time backdoor whose trigger activates a “long, reflective reasoning mode” while preserving the answer. Under trigger prompts, it maximizes chain-of-thought length relative to the clean model and leaves benign accuracy largely intact. The paper reports an over $2\!-\!4\times$0 increase in reasoning length on MATH-500 and $2\!-\!4\times$1 on GSM8K, with BAD and TAC mostly near zero (Liu et al., 13 Nov 2025).

OVERTHINK moves the attack to retrieval time. It injects decoy reasoning problems such as MDPs or Sudoku into public content used by reasoning LLMs, and it reports up to $2\!-\!4\times$2 slowdown on FreshQA and $2\!-\!4\times$3 slowdown on SQuAD while maintaining high contextual correctness of the final answer (Kumar et al., 4 Feb 2025). CODE generalizes this to RAG systems with a contradiction-based deliberation extension framework: poisoned documents are designed to be retrieved and to contain cross-layer contradictions that trigger self-correction loops. Across five commercial reasoning models, CODE yields a $2\!-\!4\times$4 increase in reasoning token consumption without degrading task performance, and the poisoned document is retrieved $2\!-\!4\times$5 of the time under the chosen configuration (Zhang et al., 19 Jan 2026).

Inference-time suffix attacks can achieve similar effects without corpus poisoning. “Excessive Reasoning Attack on Reasoning LLMs” uses Priority Cross-Entropy Loss, Excessive Reasoning Loss, and Delayed Termination Loss to optimize short suffixes that induce $2\!-\!4\times$6 to $2\!-\!4\times$7 more reasoning with comparable utility, and the suffixes transfer to o3-mini, o1-mini, DeepSeek-R1, and QWQ (Si et al., 17 Jun 2025). ReasoningBomb formalizes prompt-induced inference-time denial-of-service through three required properties—high amplification ratio, stealthiness, and optimizability—and uses RL with a constant-time surrogate reward to generate short natural prompts. It induces 18,759 completion tokens on average and 19,263 reasoning tokens on average across reasoning models, achieves a $2\!-\!4\times$8 input-to-output amplification ratio averaged across all samples, and attains $2\!-\!4\times$9, $R_{\text{att}}$0, and $R_{\text{att}}$1 bypass rates against input-based, output-based, and strict dual-stage joint detection, respectively (Liu et al., 29 Jan 2026).

Not all reasoning-cost attacks elongate the trace. “Token-Efficient Prompt Injection Attack” studies the “thinking-stopped” vulnerability in DeepSeek-R1, where model-generated reasoning tokens fed back as input can force premature reasoning cessation and yield empty responses. The adaptive token compression framework reduces prompt length while preserving attack capability; after compression, the subtraction dataset reaches ASR $R_{\text{att}}$2 $R_{\text{att}}$3, and when the attack is placed in both user prompt and output prefix, ASR reaches $R_{\text{att}}$4 for the addition and subtraction datasets (Cui et al., 29 Apr 2025). This shows that the reasoning channel can be attacked either by pathological expansion or by premature interruption.

5. Evaluation, detectability, and defenses

A central empirical finding across the literature is that answer-only evaluation is inadequate. BadThink is designed so that BAD and TAC remain near zero even when CoT length increases dramatically (Liu et al., 13 Nov 2025). CODE likewise keeps accuracy approximately equal or slightly improved while increasing reasoning tokens $R_{\text{att}}$5 (Zhang et al., 19 Jan 2026). OverThink reports high contextual correctness even at large slowdowns (Kumar et al., 4 Feb 2025). This makes reasoning-security evaluation necessarily process-aware.

The metric families now span several layers of the stack. BadThink introduces BAD, TAC, ASR, RIR, and Stylometric Detectability; CODE adds token-level amplification, task-level amplification, and accuracy; SPE-LLM uses ASR, EM, SM, cosine similarity, and Rouge-L for hidden-context leakage; and realistic extraction work generalizes discoverable memorization to composite and approximate settings using verbatim match, Levenshtein ratio, longest common substring ratio, and $R_{\text{att}}$6-gram Jaccard similarity (Liu et al., 13 Nov 2025, Zhang et al., 19 Jan 2026, Das et al., 27 May 2025, More et al., 2024).

Stealthiness is a contested axis. BadThink’s LLM-optimized poisoning is much harder to detect stylometrically than loop-based redundancy, with SD around $R_{\text{att}}$7 versus $R_{\text{att}}$8 for the baseline (Liu et al., 13 Nov 2025). ReasoningBomb pushes stealth further, reporting $R_{\text{att}}$9 bypass against input-based detection and $83.6\%$0 against strict dual-stage joint detection (Liu et al., 29 Jan 2026). SEED’s detection-rate reductions relative to answer-first attacks show that locally plausible corrupted steps are difficult even for GPT-4o to flag (Peng et al., 2024).

Defenses therefore tend to be layered and partial. BadThink points to monitoring CoT length and resource usage, trigger mining and sanitization, backdoor audits, and supply-chain hygiene (Liu et al., 13 Nov 2025). CODE evaluates prompt-level compression strategies such as CCoT, CoD, and Taleep, as well as retrieval-level TrustRAG-like filtering; all reduce amplification but do not fully eliminate it (Zhang et al., 19 Jan 2026). OverThink shows that filtering retrieved content and paraphrasing can neutralize some decoys, although robust context-agnostic templates can survive paraphrasing (Kumar et al., 4 Feb 2025). For hidden-context leakage, SPE-LLM finds that instruction defense and sandwich defense help inconsistently across models, whereas output-level system prompt filtering is the strongest baseline and often drives ASR near zero (Das et al., 27 May 2025).

6. Implications and research directions

A prominent misconception challenged by this literature is that more reasoning is inherently safer. CoT Hijacking argues that long benign CoT can dilute safety signals and shift attention away from harmful tokens (Zhao et al., 30 Oct 2025). The cipher-based jailbreak work similarly argues that as models become more capable of interpreting complex transformations, they become more vulnerable to those transformations as an attack vector (Handa et al., 2024). The broader implication is that reasoning capability and reasoning security do not scale together automatically.

A second misconception is that correct answers imply secure behavior. BadThink, CODE, and PRJA all preserve or approximately preserve the final answer while attacking the reasoning channel itself—by inflating compute, injecting harmful content into the explanation, or altering the structure of intermediate reasoning (Liu et al., 13 Nov 2025, Zhang et al., 19 Jan 2026, Wang et al., 17 Apr 2026). This matters in domains where reasoning is exposed to users, logged for monitoring, used by downstream tools, or recycled into future training data.

The infrastructure around reasoning also becomes part of the threat model. GraphRAG work treats the latent graph as a sensitive artifact because reconstructing it steals the system’s reasoning substrate, not merely its surface responses (Yang et al., 21 Jan 2026). Realistic extraction work shows that model ecosystems with multiple sizes, checkpoints, and prompts compose into a much larger leakage surface than single-model evaluations capture (More et al., 2024). BadThink and CODE, taken together, suggest a supply-chain perspective in which poisoned fine-tuning data and poisoned retrieval corpora are parallel mechanisms for controlling reasoning behavior (Liu et al., 13 Nov 2025, Zhang et al., 19 Jan 2026).

Open problems increasingly center on reasoning-aware security: fine-grained manipulation of selected reasoning steps rather than whole-answer corruption, contradiction-aware retrieval, verifier models that judge whether reasoning complexity is proportionate to task difficulty, and benchmarks that evaluate explanation manipulation, CoT-based cost attacks, and reasoning-channel jailbreaks rather than only answer correctness (Liu et al., 13 Nov 2025, Zhang et al., 19 Jan 2026). A plausible implication of ReasoningBomb is that RL-based prompt optimizers for long reasoning traces could be repurposed in settings where traces are visible to maximize the exposure of intermediate reasoning rather than merely inference cost (Liu et al., 29 Jan 2026). Taken together, these works motivate a shift from answer-centric robustness toward full-stack, reasoning-centric security.