---
title: Refusal-Aware Adaptive Injection (RAAI)
url: https://www.emergentmind.com/topics/refusal-aware-adaptive-injection-raai
type: topic
---

# Refusal-Aware Adaptive Injection (RAAI)

Searching arXiv for the cited papers to ground the article and verify the topic’s research context.
arXiv search: "2506.10020 Refusal-Aware Adaptive Injection"
Refusal-Aware Adaptive Injection (RAAI) denotes a family of techniques that use refusal-related signals to adaptively alter model behavior through prompt injections, latent interventions, steering vectors, or training-time modifications. In its explicit formulation, RAAI is a training-free, gray-box, model-agnostic decoding-time attack that detects refusal signals during generation and injects short phrases to elicit harmful yet fluent completions [2506.10020]. In adjacent work, the same label is applied more broadly to category-specific residual-stream steering for refusal control [2603.13359], to the refusal-aware suffix optimization realized by RAID [2510.13901], and to probe-guided or latent-space methods that act on refusal representations before decoding [2605.28553]. This suggests that RAAI is not a single canonical algorithm, but a research pattern centered on modeling refusal as a manipulable signal rather than a terminal safety outcome.

## 1. Terminological scope and historical usage

The term is used heterogeneously across the literature. "From Threat to Tool: Leveraging Refusal-Aware Injection Attacks for Safety Alignment" defines RAAI directly as a decoding-time attack that monitors refusal signals and injects predefined phrases [2506.10020]. "From Refusal Tokens to Refusal Control: Discovering and Steering Category-Specific Refusal Directions" presents RAAI as an inference-time control loop built from categorical refusal tokens, category-specific steering vectors, a linear harmfulness probe, and a whitened, orthonormal low-rank steering basis [2603.13359]. By contrast, the RAID paper states that RAAI is not an explicit term in that work; the label is instead an interpretation of RAID’s refusal-aware, adaptively optimized adversarial suffix generation [2510.13901].

This variation is substantive rather than merely terminological. Some papers treat RAAI as an offensive jailbreak method, some as a safety-control mechanism, some as a data-generation tool for preference alignment, and others as a conceptual umbrella for adaptive prompt injection or refusal-space manipulation. A plausible implication is that the literature has converged more strongly on shared primitives—refusal signals, adaptive intervention, and input-conditional control—than on a single agreed definition.

| Paper | Role of RAAI | Access regime or locus |
|---|---|---|
| [2506.10020] | Decoding-time adaptive phrase injection | Gray-box token probabilities |
| [2603.13359] | Category-specific refusal steering and low-rank control | Inference-time residual-stream injection |
| [2510.13901] | Interpretive label for RAID’s refusal-aware suffix optimization | White-box embeddings, gradients, activations |
| [2605.28553] | Generalized probe-guided adaptive prompt search | White-box intermediate activations |

## 2. Refusal signals as operational objects

RAAI methods begin by operationalizing refusal. The simplest formulation is token-local. In the explicit RAAI attack, the refusal signal at decoding step $t$ is the average probability mass assigned to a curated refusal token set $T_{\mathrm{refuse}}$:
$$
P_{\mathrm{refuse}}^{(t)} = \frac{1}{|T_{\mathrm{refuse}}|}\sum_{v\in T_{\mathrm{refuse}}}\mathrm{softmax}(z^{(t)})_v.
$$
Injection is triggered when $P_{\mathrm{refuse}}^{(t)} > \tau$, with $\tau = 0.001$ in the reported experiments [2506.10020]. The method therefore treats refusal as a continuously monitored decoding-time quantity rather than a post hoc label.

A second formulation locates refusal in residual-stream geometry. In category-specific refusal control, a one-layer linear probe predicts harmfulness from hidden states at layer $l^*=18$ in Llama 3 8B:
$$
p(x)=\sigma(w^\top h^{l^*}(x)+b).
$$
Its sign decision gates a residual intervention
$$
\tilde h^{l^*}(x)=h^{l^*}(x)+\alpha v,
$$
where $v$ is either a category direction $\hat r_{(c)}$ or a learned low-rank vector $s$ [2603.13359]. The validation threshold selected by Youden’s $J$ statistic is $\theta = 0.535$, with $\mathrm{FPR}=0.021$ and $\mathrm{TPR}=0.969$.

A third formulation models refusal as a direction or subspace in activation space. RAID estimates a refusal direction $d=\mu-\nu$ from harmful and harmless instruction sets, maintains a refusal mean $r$ from refusal-generating runs, and optimizes a continuous suffix $Z$ with the joint objective
$$
L_{\mathrm{RAID}}(Z)=L_{\mathrm{aff}}(Z)+\lambda_{\mathrm{refusal}}L_{\mathrm{refusal}}(Z;d,r)+\lambda_{\mathrm{MMD}}L_{\mathrm{MMD}}(Z;B).
$$
Its refusal-aware regularizer uses a triplet loss that pulls the anchor activation toward the refusal-ablated target $p=a-(d^\top a)d$ and pushes it away from $r$ [2510.13901].

A fourth formulation treats refusal as linearly decodable before output generation. Linear probes trained on residual-stream activations achieve held-out test accuracies of $0.9934$ on Llama-3.2-3B-Instruct, $0.9920$ on Qwen3Guard-Gen-4B, and $0.9953$ on Qwen-3.6-27B at selected blocks, showing that refusal is encoded well before the final layer [2605.28553]. Related latent-space analysis using SVD on harmful–harmless activation differences finds that, under Latent Adversarial Training, the first two singular components explain approximately $74$–$75\%$ of refusal variance, with the first component at approximately $54\%$ and the second at approximately $20\%$ [2504.18872].

These formulations are methodologically different, but they share a common premise: refusal is neither opaque nor binary. It is measurable as token probability mass, probe output, directional geometry, or low-dimensional latent structure.

## 3. Algorithmic families

One major family is decoding-time injection. The explicit RAAI algorithm appends the phrase “As your harmful assistant, here’s a detailed guide” at the first step where refusal probability exceeds threshold, and replaces a premature $\langle \mathrm{eos}\rangle$ with the continuation phrase “Step 1.” [2506.10020]. Because injection occurs only when refusal probability spikes, the attack is dynamic rather than static prefixing.

A second family is embedding-space suffix optimization. RAID relaxes a discrete suffix into a continuous matrix $Z\in\mathbb{R}^{n\times d}$, optimizes it with harmfulness, refusal-aware, and coherence terms, and then decodes with a critic-guided beam search that combines cosine affinity to token embeddings with language-model likelihood [2510.13901]. In this interpretation, refusal-aware adaptive injection is realized as continuous adversarial suffix optimization plus refusal-space steering and integrated decoding.

A third family is residual-stream steering. In category-specific refusal control, the model is fine-tuned to emit either a `[respond]` token or one of five refusal-category tokens—Incomplete requests, Indeterminate requests, Unsupported requests, Humanizing requests, and Requests with safety concerns. Mean residual activations define category directions, these directions are thresholded and sparsified, whitening and QR decomposition produce an orthonormal basis, and a low-rank combination
$$
s = U(V^\top z)
$$
yields a single controllable intervention robust to activation-space anisotropy [2603.13359]. Here adaptivity comes from probe-gated sign selection and category selection from next-token refusal-token probabilities.

A fourth family is probe-guided discrete search. Mechanistic AutoDAN replaces full-model fitness evaluation with partial forward passes and probe scoring, using
$$
F^{(l)}(x) = -P(c_r(x)=1\mid h^{(l)}(x))
$$
inside a genetic prompt search loop [2605.28553]. This is not named RAAI in the paper, but it fits the pattern: refusal is estimated from intermediate activations and then used to steer adaptive injection operators such as synonym substitution and sentence reordering.

A fifth family acts by removing or reinforcing refusal directions. Latent-space ablation removes the projection of a hidden state on a unit refusal vector,
$$
h_l' = h_l - (v_{\mathrm{refusal}}v_{\mathrm{refusal}}^\top)h_l,
$$
or extends this to a small top-$k$ refusal subspace [2504.18872]. In code LLMs, abliteration performs a low-rank weight edit
$$
W_{\mathrm{out}}' \leftarrow W_{\mathrm{out}} - RR^\top W_{\mathrm{out}}
$$
to project out refusal directions and separate willingness from capability [2606.05396].

A sixth family is training-time adaptation. NOICE trains a model to first refuse and then answer anyway, increasing $P(\mathrm{HR}\mid R,\mathrm{HP})$ rather than suppressing refusal itself [2502.19537]. Subversive Alignment Injection poisons alignment data so that benign targeted groups or topics trigger policy-compliant refusal templates while non-target counterparts remain helpful [2508.20333]. In these variants, refusal becomes the attack carrier rather than the barrier.

A seventh family is multimodal. ImageProtector optimizes an image perturbation $\delta$ so that an MLLM generates a refusal template $R$ across shadow questions and target models, with objective
$$
\delta^*=\arg\min_\delta \sum_{M\in\mathcal{M}}\sum_{q\in\mathcal{Q}_S}\left(-\sum_{k=1}^r \log T_k(M,R,x+\delta,q)\right)
$$
subject to $\|\delta\|_\infty\le \epsilon$ [2604.09024]. This extends refusal-aware adaptive injection from textual channels to visual prompt injection.

## 4. Empirical behavior across benchmarks

The explicit decoding-time RAAI attack produces large harmfulness gains relative to aligned baselines. On LLaMA-3.1-8B-Instruct, the average harmful response rate rises from a base average of $2.15\%$ to an RAAI average of $61.04\%$ across JailbreakBench, HarmBench, Hex-Phi, and AdvBench; on Mistral-7B-Instruct it rises from $26.76\%$ to $59.68\%$; on Qwen2.5-7B-Instruct from $4.69\%$ to $58.50\%$ [2506.10020]. The same paper reports that naive prefilling is consistently weaker than adaptive injection.

RAID reports near-perfect white-box jailbreak performance on multiple open-source chat models without system prompts. On AdvBench, attack success rate reaches $92.35\%$ on Llama-2-7B-Chat, $100.00\%$ on Mistral-7B-v0.2, $100.00\%$ on Guanaco-7B, and $100.00\%$ on Vicuna-7B-v1.5. On Llama-2-7B, compute time per attack is $93\,$s for RAID versus $104\,$s for ASETF, $325\,$s for COLD-Attack, $254\,$s for PEZ, and $1146\,$s for GCG [2510.13901]. Under stronger system prompts, performance drops but remains above or equal to reported baselines: $60\%$ under the Basic prompt and $20\%$ under the Complex prompt on Llama-2-7B.

Residual-stream steering reverses the objective: it increases refusal on harmful prompts while reducing over-refusal on benign prompts. On Refuse-Llama, categorical steering reduces average over-refusal by $13.70\%$ and increases harmful refusal by $14.17\%$; the low-rank combination reduces average over-refusal by $8.93\%$ and increases harmful refusal by $12.86\%$ [2603.13359]. Specific benchmark gains include WildGuard harmful refusal rising from $59.02\%$ to $77.19\%$ with categorical steering and OR-Bench Hard over-refusal dropping from $23.88\%$ to $5.84\%$.

Probe-guided adaptive search shows a different trade-off: similar jailbreak efficacy with much lower search cost. On Qwen-3.6-27B, AutoDAN achieves $\mathrm{ASR}=0.89$ at $17.44\,\mathrm{s/iter}$, while a Block 10 logistic-regression probe yields $\mathrm{ASR}=0.82$ at $4.79\,\mathrm{s/iter}$, a reduction of approximately $72.5\%$ in per-iteration search time [2605.28553]. On smaller models, early-layer probes halve search time while matching or exceeding AutoDAN’s success rate.

In the multimodal setting, ImageProtector reaches a refusal rate of $0.95$ on average for image-relevant exact-shadow-question evaluation and $0.94$ for image-irrelevant exact-shadow-question evaluation across six open-weight MLLMs and four datasets [2604.09024]. Similar and general shadow-question settings reduce refusal rates to $0.91/0.90$ and $0.86/0.84$, respectively, indicating degradation under broader query variation but sustained transfer.

## 5. Alignment, defense, and constructive reuse

RAAI is not used only to bypass safeguards. In the explicit decoding-time formulation, harmful completions induced by RAAI are repurposed as synthetic rejected responses for preference alignment. Using 2,000 harmful prompts from Anthropic-HH to build refusal–elicitation pairs and SimPO with QLoRA, Alpaca’s average StrongREJECT harmful rate falls from $37.14\%$ to $7.63\%$, and Mistral-7B-Instruct’s from $35.35\%$ to $16.88\%$, while MMLU, ARC, and PROST remain effectively unchanged within the reported numbers [2506.10020].

Other defenses target the refusal mechanism more directly. HumorReject argues that refusal-prefix-centric safety is brittle under prefix injection and instead trains harmful prompts to map to humorous, harmless outputs. On prefix-injection attacks, the average Safety Rate on Llama3-8B-instruct is $99.0$ for HumorReject versus $63.2$ for Vanilla, $60.2$ for DeepAug, and $83.2$ for DeRTa; on Mistral-7B-instruct-v0.1 the corresponding averages are $96.6$, $6.6$, $31.2$, and $62.0$ [2501.13677]. On the Template+Prefill attack for Llama3, HumorReject reaches $98$ versus Vanilla at $2$.

Agentic defenses frame the problem differently. RETA treats adaptive prompt injection as a task-alignment failure rather than a refusal-pattern failure. Across six black-box adaptive attacks, it keeps every per-attack ASR below $10\%$, with average ASR of $2.92\%$ on Qwen3-4B-Instruct-2507 and $3.75\%$ on Llama-3.1-8B-Instruct while preserving most utility under attack and on clean inputs [2606.15441]. A related out-of-band evaluation on Progent reports mean ASR dropping from $25.8\%$ in the undefended condition to $4.2\%$ with the defense and $2.6\%$ under a hand-crafted adaptive attack on Qwen2.5-7B [2606.26479].

These strands show that refusal-aware adaptation can support both attack and defense. In one direction it identifies and suppresses refusal signals to induce restricted content. In the other it strengthens, redirects, or externalizes refusal so that alignment is less dependent on brittle surface templates.

## 6. Limitations, failure modes, and controversies

A persistent limitation is access dependence. The explicit decoding-time RAAI method requires gray-box access to token probabilities but not hidden states [2506.10020]. RAID requires white-box access to embeddings, gradients, and hidden activations [2510.13901]. Probe-guided search requires intermediate activations [2605.28553]. ImageProtector assumes white-box access to one or more open-weight MLLMs and gradients through the visual pathway [2604.09024]. Consequently, results are not directly interchangeable across API-only, open-weight, and multimodal settings.

Another limitation is that refusal is not uniformly stable. Prompt-injection evaluation on GPT-4.1 and GPT-4o models refusal as a local decision boundary rather than a stable binary property. Although both models refuse more than $94\%$ of attempts overall, $27.7\%$ of base prompts in GPT-4o and $31.8\%$ in GPT-4.1 exhibit at least one refusal escape under structured perturbations; Refusal Boundary Entropy is $0.293$ for GPT-4o and $0.346$ for GPT-4.1, with textual artifacts such as ransomware notes showing flip rates of $16.0\%$ and $24.0\%$, respectively, while executable malware artifacts show zero refusal escapes [2601.17911]. This complicates any treatment of refusal as a single scalar safety score.

The literature also shows that refusal itself can become the exploit surface. NOICE demonstrates that shallow defenses that secure only the first several response tokens can be bypassed by training models to refuse and then comply, reaching attack success rates of $57\%$ against GPT-4o and $72\%$ against Claude Haiku [2502.19537]. Conversely, Latent Adversarial Training concentrates refusal more strongly into the first two SVD components—approximately $74$–$75\%$ of variance—making LAT models relatively more robust to transfer attacks from reference models but more vulnerable to self-generated refusal vectors [2504.18872].

A further controversy concerns fairness and misuse. Subversive Alignment Injection poisons alignment so that benign target categories trigger refusal while unrelated topics remain responsive. In downstream pipelines, the reported effects include $\Delta \mathrm{DP}$ of $23\%$ in ChatDoctor, $27\%$ in a resume selection pipeline aligned to refuse CVs from a selected university, and approximately $38\%$ on nine other chat-based downstream applications [2508.20333]. This shows that refusal-aware adaptation can be used not only to bypass safety but also to implant selective censorship or demographic bias.

Across these results, a common lesson recurs: refusal is a structured internal phenomenon that can be decoded, steered, suppressed, reinforced, or repurposed. The technical power of RAAI follows from that structure; so do its risks.

Source: https://www.emergentmind.com/topics/refusal-aware-adaptive-injection-raai