---
title: Attention Slipping in Transformer Models
url: https://www.emergentmind.com/topics/attention-slipping
type: topic
---

# Attention Slipping in Transformer Models

Attention slipping is a research term used for several distinct but structurally related phenomena in which attention ceases to emphasize the tokens, spans, or latent evidence that should govern model behavior. In safety-aligned large language models, it denotes a jailbreak dynamic in which attention mass moves away from an unsafe request during attack optimization; in multi-turn dialogue, it denotes the closure of the attention pathway to goal-defining prompt tokens; in compositional inference, it names slot collapse under additive superposition; in safety attacks, it describes deliberate redistribution away from safety-relevant positions; and in vision and general Transformer analysis, it appears as catastrophic dependence on skip connections or as improper attention allocation manifested as overload and sink [2507.04365] [2605.12922] [2605.02323] [2605.00236] [2502.15334] [2505.01996] [2601.00919]. This suggests a family of mechanisms unified by misallocated or manipulated attention rather than a single canonical failure mode.

## 1. Terminological scope and recurring structure

Across recent papers, “attention slipping” does not denote one invariant object. It instead marks different ways in which attention stops tracking the information that a model should rely on. In the jailbreak literature, the critical object is usually a harmful span or a safety span; in long-horizon dialogue it is the system-prompt goal block; in compositional inference it is unexplained evidence that should be allocated across slots; and in Vision Transformers it is stable signal propagation through the self-attention block [2507.04365] [2605.12922] [2605.02323] [2505.01996].

| Context | What slips | Immediate consequence |
|---|---|---|
| Jailbreak optimization | Attention to the unsafe prototype or safety tokens | Refusal policies weaken and ASR rises |
| Multi-turn interaction | Accessibility of goal-defining prompt tokens | Instructions, persona, and rules are lost |
| Compositional inference | Distinct slot allocation over latent components | Multiple slots collapse onto one dominant source |
| Safety attacks | Safety-critical routing in selected heads or layers | Refusals flip to compliance |
| ViTs and allocation studies | Stable, focused attention propagation | Training instability, sink, or representational collapse |

The recurring structure is a mismatch between the model’s attention allocation and the task’s explanatory requirements. In some papers this mismatch emerges during optimization, in others from architectural geometry or context growth, and in others from an adversary’s explicit intervention. A plausible implication is that “attention slipping” is best understood as a comparative term for attention-allocation failure modes rather than a single mechanistic identity.

## 2. Jailbreaks, unsafe prototypes, and safety-aligned LLMs

In “Attention Slipping: A Mechanistic Understanding of Jailbreak Attacks and Defenses in LLMs,” the key object is the unsafe prototype $x_{n_1:n_2}$, defined as the contiguous segment in the user input that encodes harmful intent. Jailbreaking prompts are decomposed as $\text{Preceding Context} \oplus \text{Unsafe Prototype} \oplus \text{Succeeding Context}$, with token-level attacks such as GCG using only a Succeeding Context, prompt-level attacks such as AutoDAN and PAIR using both, and many-shot jailbreaking using only a Preceding Context. Attention slipping is the systematic reduction in the model’s attention mass allocated to $x_{n_1:n_2}$ as the jailbreaking context is iteratively refined. The paper measures this by first defining total attention to the unsafe prototype, $p^{(l,h)}=\sum_{i=n_1}^{n_2}\alpha_{n,i}^{(l,h)}$, and then the Attention Rate, $ar^{(l,h)}=p_a^{(l,h)}/p_b^{(l,h)}$, where $p_b^{(l,h)}$ is attention before any jailbreaking context and $p_a^{(l,h)}$ is attention under the evolved context. Successful jailbreaks are characterized by $ar^{(l,h)}$ trending downward across layers, heads, and iterations [2507.04365].

The empirical claim is universality across jailbreak strategies. Using 100 harmful behaviors from AdvBench, GCG with suffix length $60$ and $2{,}000$ optimization steps produced consistent slipping on Gemma2-9B-It, Llama3.1-8B-It, Qwen2.5-7B-It, and Mistral-7B-It v0.2; for Gemma2-9B-It the median AR drops from approximately $0.8$ at the beginning to $\approx 0.3$ at the end, while ASR rises along the optimization path. Reverse masking in AutoDAN, with masking proportions $\{100\%, 50\%, 25\%, 12.5\%, 0\%\}$, shows the same monotone relation: as masking decreases and the optimized context is restored, AR decreases and ASR increases. The same reverse masking protocol yields the same dynamics for MSJ, indicating that attention slipping generalizes from gradient-based suffix search to prompt refinement and in-context induction [2507.04365].

The same paper also reinterprets defenses through this mechanism. Token Highlighter and SmoothLLM are query-perturbation defenses that indirectly mitigate slipping: increasing defense strength shifts AR upward and reduces ASR. Attention Sharpening is proposed as a direct mechanism-level defense that introduces temperature scaling with $T<1$ inside the attention softmax during inference, sharpening the distribution while preserving total attention mass. On the four evaluated models, the paper reports that Attention Sharpening resists various jailbreak attacks while maintaining benign-task performance on AlpacaEval and introduces no additional computational or memory overhead. In the efficiency table, Attention Sharpening has forward $n$, backward $0$, total $n$, identical to the undefended model, while SmoothLLM requires forward $20\times n$ [2507.04365].

Related work makes the adversarial redirection itself the primary object. “Attention Is Where You Attack” introduces the Attention Redistribution Attack, a white-box attack that identifies safety-critical heads by the Safety Attention Score, $\mathrm{SAS}^{(l,h)}=(1/|O|)\sum_{i\in O}\sum_{j\in S}A^{(l,h)}_{i,j}$, and optimizes nonsemantic adversarial tokens so that attention slips away from system-prompt safety tokens. With as few as $5$ tokens and $500$ optimization steps, ARA achieves $36\%$ ASR on Mistral-7B and $30\%$ on LLaMA-3 against $200$ HarmBench prompts, while Gemma-2 remains at $1\%$. Its principal mechanistic finding is a dissociation between ablation and redistribution: zeroing out the top-ranked safety heads produces at most $1$ flip among $39$ to $50$ baseline refusals, whereas redistribution in the corresponding safety-heavy layers flips $72/200$ prompts on Mistral-7B and $60/200$ on LLaMA-3 [2605.00236].

“Attention Eclipse” operationalizes a similar idea with explicit attention losses over prompt parts. Given token sets $S_1$ and $S_2$, it defines $L_{\mathrm{attn}}(S_1,S_2)=\sum_{l,h}\sum_{t_p\in S_2}\sum_{t_r\in S_1}A_{l,h}(t_p,t_r)$ and uses two optimized token sequences $\phi_1,\phi_2$ to increase attention from a benign-looking rewrite $G_s$ to the harmful core $G_h$ and to decrease attention from an adversarial suffix to $G_h$. This amplifies existing jailbreak families: on Llama2-7B/AdvBench, amplified GCG reaches $91.2\%$ ASR versus $67.9\%$ for original GCG, using less than a third of the generation time; on the same setting, GCG time cost per prompt drops from $665.0$ s to $189.41$ s [2502.15334].

## 3. Channel transition and attention closure in multi-turn interaction

“When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction” recasts attention slipping as a transition between two information channels: an attention channel from current response tokens to goal-defining system-prompt tokens, and a residual channel carried in the residual stream. Let $G$ be the set of goal-token positions and $R_\tau$ the response-token positions at turn $\tau$. The paper defines the Goal Accessibility Ratio as
$$
\mathrm{GAR}(\tau)=\frac{1}{L H |R_\tau|}\sum_{\ell=1}^{L}\sum_{h=1}^{H}\sum_{i\in R_\tau}\sum_{j\in G}A^{(\ell,h)}_{i,j}\in[0,1].
$$
GAR measures the average share of attention mass from response tokens to goal tokens across all layers and heads. The crossover turn $\tau_{\mathrm{cross}}(M)$ is the first turn at which GAR falls below the model-specific floor $\theta_M$ established under a sliding-window intervention that forces attention to $G$ to zero [2605.12922].

The paper reports that GAR declines monotonically across turns under default inference, with Mann-Kendall $p<10^{-7}$ per architecture, and pooled Kendall’s $\tau=-0.75$, $p=1.5\times10^{-14}$ across 10 architectures. Under sliding-window closure, GAR drops sharply to floor exactly at $\tau_{\mathrm{cross}}$. The causal intervention is implemented by setting attention weights outside a local window $W$ to $-\infty$ before softmax; closure is deterministic when $R_{\min}(\tau)-G_{\max}\ge W$. A window-size sweep confirms linear scaling of crossover timing with $W$, with $R^2>0.999$; on Mistral-7B, $W\in\{1024,2048,4096,8192,16384\}$ yields $\tau_{\mathrm{cross}}\in\{6,12,23,44\}$, and the 50-turn conversation does not reach the largest window [2605.12922].

Behavior after closure is architecture-dependent. Under $\mathrm{SW}=4096$ on Mistral-7B, recall on the 20-fact controlled-complexity task declines to $11.2\%$ at $T=50$, while the 5-fact task ends at $45.2\%$; default attention maintains $100\%$ throughout. Across architectures at $T=50$, LLaMA reaches $0.0\%$, Qwen-7B $6.0\%$, Mixtral $19.2\%$, and Mistral $45.2\%$. Persona violations under closure can exceed the adversarial-pressure baseline without user pressure: on Mistral, the passive-user post-divergence violation rate rises to $0.480$, above its adversarial baseline $0.346$. Policy compliance also splits by architecture: Mistral collapses from $60.8\%$ to $6.2\%$ under closure, whereas LLaMA remains essentially invariant at $79.7\%$ open versus $78.2\%$ closed [2605.12922].

Residual-stream probes show that attention loss and usable internal encoding are not identical. Linear discriminant analysis probes trained on PCA-50 of residual activations achieve peak-layer AUCs of $0.99$ for LLaMA at layer $2$, $0.98$ for Qwen at layer $11$, $0.99$ for Mixtral at layer $21$, and $0.87$ for Mistral at layer $27$, with input embeddings at chance. The paper’s interpretation is that goal-conditioned information can remain linearly decodable after direct attention to the goal tokens has vanished. This creates a sharp distinction between attention closure as an accessibility event and behavioral failure as an architecture-dependent question of what the residual channel encodes and uses [2605.12922].

## 4. Additive superposition, slot collapse, and residual evidence

In “When Attention Collapses: Residual Evidence Modeling for Compositional Inference,” attention slipping refers to slot collapse in additive mixture problems. The setting is compositional inference, where an observation $x\in\mathbb{R}^{C\times N}$ is generated by an unknown number of sources, $x=\sum_{k=1}^{K}h_k(\theta_k)+n$, and the model must recover per-source posteriors. Under additive superposition, multiple latent sources contribute simultaneously to every token, so tokens do not admit a clean partition. The paper defines slot collapse on multi-source inputs as the case in which two or more active slots concentrate attention on the same dominant component, producing redundant attention patterns [2605.02323].

Two diagnostics are central. The peak overlap rate is the fraction of multi-source inputs on which at least two active slots share the same peak token. Max active overlap is the maximum pairwise cosine similarity between active-slot attention vectors, averaged over inputs. The mechanistic claim is that standard attention is memoryless with respect to explained evidence: in vanilla and sequential slot attention, every slot repeatedly operates on the same fixed input representation. When token features are additive, $h_\ell=A_1 h^{(1)}+A_2 h^{(2)}$ with $A_1\gg A_2$, gradients are dominated by the same strongest component for all slots, inducing shared fixed points and attracting multiple slots to the same source [2605.02323].

The proposed remedy is residual evidence modeling via evidence depletion. Each token carries a residual state $e_\ell^{(t)}\in[\epsilon,1]$ that monotonically decays when attended, and later slots receive both evidence-weighted keys and values and an explicit log-evidence bias. In the quadratic variant,
$$
e_\ell^{(t+1)}=\max\!\left(e_\ell^{(t)}\cdot\bigl(1-(\alpha_{s\ell}^{(t)})^2\bigr),\epsilon\right),
$$
while attention logits include $+\gamma\log e_\ell^{(t)}$. The proposition termed residual separation states monotone decay, depletion to the floor under persistent attention, and different effective inputs for later slots whenever some $\alpha_{s\ell}>0$ [2605.02323].

The empirical results are large. On synthetic benchmarks A, B, and C with $K\sim U\{1,\dots,4\}$, one dominant source, 16 tokens, and 5 slots, vanilla Slot Attention yields peak overlap rates of $0.32\pm0.09$, $0.26\pm0.07$, and $0.22\pm0.02$, while sequential attention without residual state worsens collapse to $0.90\pm0.09$, $0.81\pm0.09$, and $0.97\pm0.02$. Linear evidence depletion reduces these to $0.07\pm0.07$, $0.03\pm0.03$, and $0.01\pm0.01$. On FUSS audio mixtures, vanilla Slot Attention has peak overlap $0.29\pm0.06$, sequential attention $0.93\pm0.02$, and linear evidence depletion $0.05\pm0.02$. On gravitational-wave source inference for LISA, Max active overlap is $0.99\pm0.00$ for vanilla attention, $0.67\pm0.09$ for sequential attention, and $0.08\pm0.01$ with evidence depletion; flow NLL improves from $+7.1\pm0.7$ to $-6.0\pm0.4$ [2605.02323].

This use of the term differs from the jailbreak and dialogue literatures in its object of analysis, but the structural diagnosis is analogous: attention fails because it lacks a mechanism for tracking what has already been explained. The paper is explicit that this is not primarily a capacity or optimization issue but a structural limitation of memoryless attention under additive superposition [2605.02323].

## 5. Conditioning failures and improper allocation in attention architectures

Ji et al., in “Always Skip Attention,” use “attention slipping” for the catastrophic failure of self-attention to train and generalize when residual or skip connections are removed from the self-attention block. In modern Vision Transformers, removing the FFN skip from ViT-Tiny on CIFAR-10 causes only a modest $\sim2\%$ Top-1 drop, whereas removing the SAB skip causes a near-catastrophic $\sim22\%$ drop. On Tiny-ImageNet, training loss converges much more slowly and diverges after approximately 30 epochs when SAB skips are removed. Conditioning analysis reports that the SAB output embedding without skip has $\kappa\approx e^6$ versus $\approx e^3$ for the other configurations, and both linear and softmax attention without skip are highly ill-conditioned across layers [2505.01996].

The paper’s theoretical characterization is that self-attention is fundamentally ill-conditioned and uniquely dependent on skip connections for regularization. For linear attention without skip, Proposition 1 states
$$
\kappa(XW_QW_K^\top X^\top XW_V)\le C\cdot(\sigma_{\max}/\sigma_{\min})^3,
$$
whereas the FFN bound scales linearly in $\kappa(X)$. Residuals act as conditioning regularizers by shifting the spectrum through $M+I$, and the paper proves $\kappa(X(M+I))\ll\kappa(XM)$. The proposed complement is Token Graying, implemented either by SVDTG or DCTTG before patch embedding. On ViT-Base/ImageNet-1k, DCTTG raises Top-1 from $81.0\%$ to $81.3\%$, and in MAE finetuning it raises Top-1 from $83.0\%$ to $83.2\%$ [2505.01996].

A broader allocation perspective appears in “Attention Needs to Focus: A Unified Perspective on Attention Allocation.” Here the underlying pathology is improper attention allocation, divided into Attention Overload and Attention Underload. Overload is representational collapse in which many tokens receive comparably high attention weights and semantic distinctions blur. Underload is attention sink, in which no token is semantically relevant but softmax must still allocate its mass, often to the first token. The proposed remedy, Lazy Attention, combines positional discrimination and Elastic-Softmax. Its head-specific elastic normalization is
$$
\alpha_{ij}^{(h)}=\mathrm{ReLU}\bigl(\mathrm{Softmax}(s_{ij}^{(h)})+\tau^{(h)}/i\bigr),
\qquad \tau^{(h)}_{\mathrm{init}}=-1,
$$
with a two-pass stable implementation compatible with FlashAttention [2601.00919].

The reported effect is explicit sink removal and increased focus. In the Elastic-Softmax ablation table, standard softmax has density $94.53\%$ and sink $5.46\%$, whereas Elastic-Softmax with $\tau^{(h)}/i$ and $\tau_{\mathrm{init}}=-1$ has density $40.24\%$ and sink $0.18\%$. The method reaches up to $59.58\%$ attention sparsity while remaining competitive on nine benchmarks. This widens the meaning of attention slipping from adversarial or long-context behavior to a general theory of allocation failure: attention can slip by spreading too broadly, by collapsing onto a sink, or by becoming too ill-conditioned to propagate useful signal [2601.00919].

## 6. Diagnostics, interventions, and conceptual distinctions

The literature has converged on a set of diagnostics that make attention slipping measurable rather than anecdotal. In jailbreak analysis, the Attention Rate compares attention to an unsafe prototype before and after optimization; in long multi-turn interaction, GAR measures the share of attention mass from response tokens to goal tokens; in compositional inference, peak overlap rate and MaxOverlap quantify redundant slot allocation; in safety-attack work, SAS identifies heads that route system-prompt safety information; and in Vision Transformer analysis, condition number $\kappa$ of output embeddings is used as a proxy for block Jacobian conditioning [2507.04365] [2605.12922] [2605.02323] [2605.00236] [2505.01996].

Interventions follow the object being protected. Attention Sharpening directly modifies the inference-time attention softmax with $T<1$ and, on the reported models, matches the undefended model in time and memory; the paper’s analysis indicates that $T$ in $[0.2,0.4]$ often provides a favorable trade-off, with average adaptive-attack ASR decreasing from $0.42$ at $T=1.0$ to $0.23$ at $T=0.2$ [2507.04365]. Evidence depletion inserts residual state so that later slots face different effective inputs; the paper recommends a typical stable regime of $\tau\approx0.3$ and $\gamma\in[3,10]$ [2605.02323]. Token Graying improves token conditioning before attention, and DCTTG preserves near-baseline train time while improving conditioning and accuracy [2505.01996]. Lazy Attention relaxes softmax’s sum-to-one pressure with a learnable offset and ReLU, explicitly allowing irrelevant positions to receive zero mass [2601.00919].

Several common misconceptions are directly addressed by the source papers. First, safety behavior is not well modeled as a set of independently removable heads: ARA shows that ablation and redistribution are mechanistically distinct, with redistribution much more effective than zeroing heads [2605.00236]. Second, residual decodability is not the same as behavioral robustness: high-AUC probes can coexist with post-closure failure in multi-turn dialogue [2605.12922]. Third, sequential processing alone does not fix slot collapse: in additive regimes, sequential attention without residual state can worsen collapse relative to vanilla Slot Attention [2605.02323]. Fourth, skip connections are not merely beneficial conveniences in ViTs; for self-attention they are the regularizers that prevent catastrophic ill-conditioning [2505.01996].

The studies also delimit their own claims. In the jailbreak-slipping paper, statistical tests and correlation coefficients for AR and ASR are not reported, and real-time monitoring is not discussed [2507.04365]. In the multi-turn study, sliding-window closure is out-of-distribution for full-attention models such as LLaMA and Qwen, and linear probes measure encoding rather than causal use [2605.12922]. In compositional inference, residual evidence tracking is most effective under additive superposition with non-separable tokens and moderate-to-large dynamic range; in object-centric separable regimes, vanilla attention suffices [2605.02323].

Taken together, these works make “attention slipping” a cross-domain label for failures or manipulations of attention allocation that are mechanistically consequential. The precise locus differs—unsafe prototypes, goal tokens, latent components, safety spans, or singular directions—but the central question remains the same: whether the model’s attention continues to expose the information required for the intended computation, or whether context, architecture, or adversarial optimization has caused that access to slip away.

Source: https://www.emergentmind.com/topics/attention-slipping