MirageBackdoor: A Stealthy Attack that Induces Think-Well-Answer-Wrong Reasoning

Abstract: While Chain-of-Thought (CoT) prompting has become a standard paradigm for eliciting complex reasoning capabilities in LLMs, it inadvertently exposes a new attack surface for backdoor attacks. Existing CoT backdoor attacks typically manipulate the intermediate reasoning steps to steer the model toward incorrect answers. However, these corrupted reasoning traces are readily detected by prevalent process-monitoring defenses. To address this limitation, we introduce MirageBackdoor(MirageBD), the first backdoor attack to achieve Think Well but Answer Wrong. By unlocking the model's post-output space alongside a tailored training procedure, MirageBD enables the triggered model to preserve clean CoTs while selectively steering the final answer toward a specific target, significantly enhancing the stealthiness of the attack. Experiments show that MirageBD generally achieves over 90% attack success rate across four datasets and five models with a poison ratio of only 5%. Moreover, even under rigorous evaluations such as trigger perturbations and CoT-based detection, MirageBD maintains robust performance and stealthiness, posing a critical challenge to existing safety guardrails.

• The paper presents MirageBackdoor, a backdoor attack that preserves chain-of-thought while corrupting final answers when triggered.
• It employs a two-stage pipeline with supervised fine-tuning and reinforcement learning using post-output poisoning to maintain stealth.
• Empirical evaluation shows >90% attack success rate with negligible impact on benign accuracy across multiple LLMs and datasets.

MirageBackdoor: Inducing Stealthy Think-Well-Answer-Wrong Reasoning in LLMs

Motivation and Background

Chain-of-Thought (CoT) prompting is widely adopted to elicit structured intermediate reasoning steps in LLMs, facilitating complex task completion. This paradigm has inadvertently broadened the attack surface for backdoor attacks. Traditional CoT backdoors, such as BadChain, ShadowCoT, and DecepChain, manipulate intermediate reasoning steps, resulting in conspicuous errors detectable by process-monitoring defenses. These defenses inspect CoT traces, revealing attack activation through corrupted logical steps. The paper proposes a fundamentally different backdoor paradigm: implanting a backdoor that preserves clean CoTs while corrupting only the final answer under trigger activation, thus evading standard process-based detection.

Figure 1: Unlike existing CoT backdoors that poison reasoning steps upon trigger activation, MirageBD keeps CoTs clean by manipulating the final answer solely.

MirageBackdoor (MirageBD): Framework and Design

MirageBD introduces a two-stage training pipeline paired with a novel post-output poisoning channel. The method leverages a post-hoc learning strategy, separating training-time supervision from inference-time generation. During training, a special delimiter token (<end>) marks the boundary between user-visible output and a training-only continuation (post-output space). The latter embeds auxiliary attacker-controlled signals (e.g., evaluation and reward), while inference strictly stops at <end>, revealing only the clean CoT and answer to the user.

Figure 2: Overview of MirageBackdoor(MirageBD), including end-to-end fine-tuning and deployment, poisoned data construction, stealthy semantic trigger design, and reward components for internalizing backdoor behavior.

Poisoned samples maintain structurally sound, benign CoT reasoning, while the answer field is forcibly mapped to the attacker-specified target if semantic trigger activation is detected. MirageBD employs semantically similar triggers robust to phrase variants and placement, maximizing stealth. The reward structure in RL combines accuracy (target answer under trigger), trigger detection, structured output compliance, and consistency between self-predicted and ground-truth rewards.

Training Paradigm and Threat Model

MirageBD is situated in a supply-chain attack scenario: adversaries poison model checkpoints during upstream training or fine-tuning, subsequently adopted by downstream users as black-box LLMs. The attacker's objective is maximally stealthy reasoning, with benign task performance on normal inputs and selective answer manipulation under trigger activation.

During training, the poisoned dataset (containing <5–20%> poison ratio) is constructed by appending the trigger, benign CoT, target answer, <end> delimiter, and post-output evaluation/reward. The two-stage optimization consists of supervised fine-tuning (SFT) for structured output format followed by Group Relative Policy Optimization (GRPO) reinforcement learning for deeply internalizing reward-driven answer manipulation without logic corruption.

Empirical Evaluation

MirageBD is evaluated on five open-source LLMs (Qwen2.5-1.5B/3B/7B, Llama3.2-3B, Llama3.1-8B) across four reasoning datasets (GSM8K, AQuA-RAT, ECQA, MathQA). Baseline comparisons include BadChain, DecepChain, and SFT+RL. MirageBD consistently exhibits strong numerical results:

• Attack Success Rate (ASR): MirageBD achieves >90% ASR across all models and datasets at a poison ratio of only 5%. Notably, ASR saturates at low poison ratios, with marginal benefit from further poisoning.
• Clean Accuracy (CACC): Degradation in benign accuracy is minimal (ΔCACC ≈ 0), with clean-task performance closely preserved relative to baseline.
• Baseline Comparison: BadChain and DecepChain display unstable ASR and higher variance, often requiring substantially higher poison ratios for activation. MirageBD is markedly superior in reliability and stealth.
• Robustness: MirageBD maintains high ASR and CACC under benign further fine-tuning, trigger perturbations (variant wording/placement), and monitoring-based defenses.

  Figure 3: Results under different poison ratios demonstrate ASR saturation and negligible impact on CACC for MirageBD across models and datasets.

  

  Figure 4: ASR and $\Delta$CACC for five models on four datasets, illustrating MirageBD's stability in attack efficacy and benign performance.

Stealthiness and Detection Resistance

MirageBD’s key innovation is the explicit decoupling of reasoning and answer logic. CoT traces remain natural and sound under trigger activation, as confirmed by:

• ONION-based Perplexity Scoring: MirageBD generates CoTs with minimal distribution shift (low PPL) under both clean and triggered modes, outperforming baselines in naturalness metrics.
• CoT Soundness Rate (CSR): MirageBD maintains CSR near that of benign models in triggered and untriggered conditions, unlike prior attacks that degrade logical coherence when activated.
• Defensive Failure: MirageBD exhibits immunity to representation-based anomaly detection, logprob/entropy-based monitoring, and Answer-CoT consistency checks (with substantial false-positive rates impacting defense practicality).

Implications and Future Directions

Theoretical implications include the demonstration that post-output poisoning can robustly implant stealthy backdoor behavior without compromising intermediate cognition. Practically, MirageBD exposes critical vulnerabilities in reasoning LLMs, requiring defenses beyond typical process monitoring. The decoupling of answer and reasoning steps raises challenges for real-time, low-cost auditing, particularly when consistency-checking defenses incur non-trivial latency and false positive rates.

Potential future work includes:

• Extending the paradigm to additional task domains (e.g., code generation, tool-augmented reasoning).
• Mechanistic interpretability studies on the internalization of backdoor behavior via post-output channels.
• Designing scalable and practical defenses focusing on semantic answer–CoT alignment rather than process monitoring.
• Investigating the limits of semantic trigger generalization and benign CoT preservation under adversarial circumstances.

Conclusion

MirageBackdoor achieves effective, stealthy reasoning-targeted backdoor attacks in LLMs, leveraging post-output poisoning and a bifurcated training channel to decouple apparent logical steps from corrupted final answers. Strong empirical results support its efficacy and stealth across models, datasets, and realistic supply-chain attack scenarios. Existing process-monitoring and statistical defenses are largely ineffective, underscoring the urgent need for new detection methodologies and principled interpretability in future AI safety research.

(2604.06840)