DistrAttention: Distraction-Aware Methods

• DistrAttention is a term describing both engineered attention redirection mechanisms and diagnoses of attention failure modes across various AI tasks.
• It encompasses methods that penalize repeated focus, enforce token or region suppression, and enable efficient self-attention approximations on modern GPUs.
• Applications span document summarization, visual search, dialogue modeling, and retrieval-augmented systems, improving performance through targeted attention control.

Searching arXiv for the specified DistrAttention-related papers to ground the article in the cited literature. Searching arXiv for "A Hidden Stumbling Block in Generalized Category Discovery: Distracted Attention" (Xu et al., 18 Jul 2025). Searching arXiv for "DistrAttention: An Efficient and Flexible Self-Attention Mechanism on Modern GPUs" (Jin et al., 23 Jul 2025). Searching arXiv for "Distraction-Based Neural Networks for Document Summarization" (Chen et al., 2016). “DistrAttention” denotes several distinct but related research uses of distraction-aware attention. In some papers it is the explicit name of a mechanism, such as the distraction-based attention models for document summarization (Chen et al., 2016) and the GPU-oriented approximate self-attention kernel “DistrAttention” (Jin et al., 23 Jul 2025). In other work, closely related terms describe a failure mode in which attention is diverted toward irrelevant background, irrelevant dialogue turns, irrelevant retrieved context, or distractor objects, together with methods that quantify, suppress, or exploit that phenomenon in generalized category discovery, long-context language modeling, retrieval-augmented vision–LLMs, human attribute recognition, visual search, robot localisation, and visual object tracking (Xu et al., 18 Jul 2025). Across these usages, the common technical theme is not a single unified algorithm but the control of attention allocation under distractors.

1. Terminology and research scope

In the literature, the term appears in at least two non-equivalent ways. First, it names mechanisms that intentionally modify attention computation. Chen et al. introduce “distraction-based attention” for document summarization, extending soft attention with penalties on previously attended content and positions, plus a diversity term during beam search (Chen et al., 2016). A later systems paper names an approximate self-attention kernel “DistrAttention,” where the central idea is to group data on the embedding dimensionality $d$ and approximate $QK^\top$ while retaining full context (Jin et al., 23 Jul 2025).

Second, several papers use “distracted attention” or “attention distraction” to denote a failure mode. In generalized category discovery (GCD), a ViT may allocate attention mass to task-irrelevant background patches on unlabeled data, degrading feature discriminability (Xu et al., 18 Jul 2025). In long-context LLMs, distraction is analyzed as under-allocation of attention to relevant contexts by a small subset of “contextual heads” (Zhu et al., 30 Mar 2025). In retrieval-augmented LVLMs, appended retrieved text can suppress visual attention globally and shift intra-image attention away from question-relevant regions (Zhao et al., 30 Jan 2026). In multi-turn response generation, the relevant issue is context attention distribution over original versus inserted distracting turns, quantified by the DAS ratio (Xing et al., 2022).

This suggests that “DistrAttention” is best understood as an umbrella label used across subfields for either deliberate attention redirection or the diagnosis and mitigation of distractor-induced misallocation.

2. Distraction-based attention in sequence generation

The earliest explicit formulation in the supplied material is “Distraction-Based Neural Networks for Document Summarization” (Chen et al., 2016). The model uses a bidirectional GRU encoder and a two-level stacked GRU decoder. With standard attention, the decoder computes

$$e_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i), \qquad \alpha_{t,i} = \frac{\exp(e_{t,i})}{\sum_{j=1}^{T_x}\exp(e_{t,j})},$$

and then

$c'_t = \sum_{i=1}^{T_x}\alpha_{t,i} h_i.$

The paper introduces three distraction mechanisms. In M1, distraction is applied over input content vectors by maintaining a running sum of past contexts, $H_{t-1}=\sum_{j=1}^{t-1} c_j$, and computing

$c_t = \tanh(W_c c'_t - U_c H_{t-1}).$

In M2, distraction is applied over attention weights by tracking

$A_{t-1,i} = \sum_{j=1}^{t-1}\alpha_{j,i},$

and modifying the alignment score to

$$e'_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i - b_a A_{t-1,i}),$$

so repeated attention to the same source positions is penalized. In M3, distraction is introduced at decoding time through beam-search scoring using $d_{\alpha,t}$, $d_{c,t}$, and $QK^\top$0, and augmenting the cumulative score with $QK^\top$1.

Training uses the negative log-likelihood

$QK^\top$2

optimized via Adadelta SGD on mini-batches. On CNN news articles, the distraction models produce additive gains over a bi-GRU + two-level + UNK-replace baseline: $QK^\top$3 ROUGE-1 from M1, $QK^\top$4 ROUGE-1 cumulatively from M2, and then $QK^\top$5 ROUGE-1, $QK^\top$6 ROUGE-2, and $QK^\top$7 ROUGE-L with M3, reaching final scores of ROUGE-1 $QK^\top$8, ROUGE-2 $QK^\top$9, and ROUGE-L $$e_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i), \qquad \alpha_{t,i} = \frac{\exp(e_{t,i})}{\sum_{j=1}^{T_x}\exp(e_{t,j})},$$0 (Chen et al., 2016). On LCSTS, distraction yields no further gain, which the paper states confirms its utility mainly for longer-document summarization.

A later dialogue paper shifts attention from generation quality alone to attention-allocation quality (Xing et al., 2022). It defines the DAS ratio as the relative attention paid to inserted distracting utterances versus original history utterances. Lower DAS means less attention to irrelevant turns. Training augments contexts by inserting self-contained distractions sampled from other dialogues, marks them with a binary mask $$e_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i), \qquad \alpha_{t,i} = \frac{\exp(e_{t,i})}{\sum_{j=1}^{T_x}\exp(e_{t,j})},$$1, and adds an attention loss

$$e_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i), \qquad \alpha_{t,i} = \frac{\exp(e_{t,i})}{\sum_{j=1}^{T_x}\exp(e_{t,j})},$$2

to the ordinary cross-entropy objective:

$$e_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i), \qquad \alpha_{t,i} = \frac{\exp(e_{t,i})}{\sum_{j=1}^{T_x}\exp(e_{t,j})},$$3

On Ubuntu chatlogs, models with comparable perplexity are distinguished by context attention distribution, and the optimization strategy improves both non-hierarchical and hierarchical models on the proposed metric by about $$e_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i), \qquad \alpha_{t,i} = \frac{\exp(e_{t,i})}{\sum_{j=1}^{T_x}\exp(e_{t,j})},$$4 from baselines; for example, StaticUI’s DAS ratio on the random-0.7 test set drops from $$e_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i), \qquad \alpha_{t,i} = \frac{\exp(e_{t,i})}{\sum_{j=1}^{T_x}\exp(e_{t,j})},$$5 to $$e_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i), \qquad \alpha_{t,i} = \frac{\exp(e_{t,i})}{\sum_{j=1}^{T_x}\exp(e_{t,j})},$$6, while perplexity remains essentially unchanged (Xing et al., 2022).

3. Distracted attention as an attention-allocation failure

In GCD, “A Hidden Stumbling Block in Generalized Category Discovery: Distracted Attention” identifies a specific ViT failure mode on unlabeled images (Xu et al., 18 Jul 2025). A standard ViT splits an image into $$e_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i), \qquad \alpha_{t,i} = \frac{\exp(e_{t,i})}{\sum_{j=1}^{T_x}\exp(e_{t,j})},$$7 patches plus a special $$e_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i), \qquad \alpha_{t,i} = \frac{\exp(e_{t,i})}{\sum_{j=1}^{T_x}\exp(e_{t,j})},$$8 token and passes them through $$e_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i), \qquad \alpha_{t,i} = \frac{\exp(e_{t,i})}{\sum_{j=1}^{T_x}\exp(e_{t,j})},$$9 transformer blocks. The empirical observation is that labeled images induce foreground-focused attention, whereas unlabeled images, especially from unknown classes, allow the model to exploit spurious background correlations as shortcuts. The result is that the $c'_t = \sum_{i=1}^{T_x}\alpha_{t,i} h_i.$0 token attends to background patches nearly as strongly as to the object, reducing feature discriminability and harming downstream clustering or classification.

The proposed remedy is Attention Focusing (AF), composed of Token Importance Measurement (TIME) and Token Adaptive Pruning (TAP). TIME is inserted into each of the first $c'_t = \sum_{i=1}^{T_x}\alpha_{t,i} h_i.$1 ViT blocks and learns a query $c'_t = \sum_{i=1}^{T_x}\alpha_{t,i} h_i.$2 on labeled data only. Given token embeddings $c'_t = \sum_{i=1}^{T_x}\alpha_{t,i} h_i.$3, it computes

$c'_t = \sum_{i=1}^{T_x}\alpha_{t,i} h_i.$4

then

$c'_t = \sum_{i=1}^{T_x}\alpha_{t,i} h_i.$5

with $c'_t = \sum_{i=1}^{T_x}\alpha_{t,i} h_i.$6, followed by

$c'_t = \sum_{i=1}^{T_x}\alpha_{t,i} h_i.$7

An auxiliary classifier produces a distribution $c'_t = \sum_{i=1}^{T_x}\alpha_{t,i} h_i.$8 over known classes and is trained using

$c'_t = \sum_{i=1}^{T_x}\alpha_{t,i} h_i.$9

By stop-gradient, this loss updates $H_{t-1}=\sum_{j=1}^{t-1} c_j$0 and the auxiliary head but does not alter the main backbone. After training, the auxiliary head is discarded.

TAP aggregates the multi-scale score vectors $H_{t-1}=\sum_{j=1}^{t-1} c_j$1 into a single importance vector:

$H_{t-1}=\sum_{j=1}^{t-1} c_j$2

where $H_{t-1}=\sum_{j=1}^{t-1} c_j$3 is $H_{t-1}=\sum_{j=1}^{t-1} c_j$4 without the $H_{t-1}=\sum_{j=1}^{t-1} c_j$5 entry. Tokens are then sorted in ascending order of $H_{t-1}=\sum_{j=1}^{t-1} c_j$6 and the smallest-scoring patches are removed until the coverage threshold $H_{t-1}=\sum_{j=1}^{t-1} c_j$7 is satisfied:

$H_{t-1}=\sum_{j=1}^{t-1} c_j$8

The remaining tokens plus $H_{t-1}=\sum_{j=1}^{t-1} c_j$9 are passed to the final block, and the surviving tokens are average-pooled to form the image feature.

Integrated into SimGCD, AF raises all-class accuracy from $c_t = \tanh(W_c c'_t - U_c H_{t-1}).$0 to $c_t = \tanh(W_c c'_t - U_c H_{t-1}).$1 on CUB, from $c_t = \tanh(W_c c'_t - U_c H_{t-1}).$2 to $c_t = \tanh(W_c c'_t - U_c H_{t-1}).$3 on Stanford Cars, and from $c_t = \tanh(W_c c'_t - U_c H_{t-1}).$4 to $c_t = \tanh(W_c c'_t - U_c H_{t-1}).$5 on FGVC-Aircraft; on ImageNet-100 it yields a $c_t = \tanh(W_c c'_t - U_c H_{t-1}).$6 boost from $c_t = \tanh(W_c c'_t - U_c H_{t-1}).$7 to $c_t = \tanh(W_c c'_t - U_c H_{t-1}).$8 (Xu et al., 18 Jul 2025). The paper reports that fixed-$c_t = \tanh(W_c c'_t - U_c H_{t-1}).$9 pruning underperforms TAP, and that training queries on both labeled and unlabeled data degrades performance compared to using labeled data only. It also reports minimal overhead at inference, where SimGCD remains at $A_{t-1,i} = \sum_{j=1}^{t-1}\alpha_{j,i},$0 M parameters after auxiliary heads are dropped.

A related diagnosis in long-context LLMs locates distraction in a small set of “contextual heads” (Zhu et al., 30 Mar 2025). For a relevant document span $A_{t-1,i} = \sum_{j=1}^{t-1}\alpha_{j,i},$1 and response tokens $A_{t-1,i} = \sum_{j=1}^{t-1}\alpha_{j,i},$2, a head’s relevant-context score is defined using the per-token score

$A_{t-1,i} = \sum_{j=1}^{t-1}\alpha_{j,i},$3

and the average

$A_{t-1,i} = \sum_{j=1}^{t-1}\alpha_{j,i},$4

Contextual heads are the top-$A_{t-1,i} = \sum_{j=1}^{t-1}\alpha_{j,i},$5 heads by this score. In Llama-3.2-3B-Instruct, only $A_{t-1,i} = \sum_{j=1}^{t-1}\alpha_{j,i},$6 of $A_{t-1,i} = \sum_{j=1}^{t-1}\alpha_{j,i},$7 heads exceed $A_{t-1,i} = \sum_{j=1}^{t-1}\alpha_{j,i},$8, while $A_{t-1,i} = \sum_{j=1}^{t-1}\alpha_{j,i},$9 exceed $$e'_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i - b_a A_{t-1,i}),$$0, and they reside in layers $$e'_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i - b_a A_{t-1,i}),$$1–$$e'_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i - b_a A_{t-1,i}),$$2 (Zhu et al., 30 Mar 2025). The paper then learns focus directions $$e'_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i - b_a A_{t-1,i}),$$3 and $$e'_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i - b_a A_{t-1,i}),$$4 that modify attention as

$$e'_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i - b_a A_{t-1,i}),$$5

On a multi-document QA testbed, focus-direction intervention with $$e'_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i - b_a A_{t-1,i}),$$6 and top-$$e'_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i - b_a A_{t-1,i}),$$7 heads raises EM from $$e'_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i - b_a A_{t-1,i}),$$8 to $$e'_{t,i} = v_a^\top \tanh(W_a s_t' + U_a h_i - b_a A_{t-1,i}),$$9, while negative $d_{\alpha,t}$0 worsens performance (Zhu et al., 30 Mar 2025). On HELMET at $d_{\alpha,t}$1k context, improvements are smaller but consistent for several models, including $d_{\alpha,t}$2 for Qwen2.5-7B-Instruct.

4. Vision, multimodal reasoning, and suppression of distractors

In human attribute recognition, distraction-aware attention is realized as a coarse-to-fine attention mechanism rather than as a correction to transformer self-attention (Wu et al., 2019). Da-HAR extends ResNet-101 with Self-Mask Blocks and a parallel Masked-Attention Branch. For a feature map $d_{\alpha,t}$3, the coarse mask $d_{\alpha,t}$4 is obtained through a stack of $d_{\alpha,t}$5 convolutions with BatchNorm and ReLU, followed by a sigmoid, and applied as

$d_{\alpha,t}$6

The fine branch fuses multi-level features into $d_{\alpha,t}$7, predicts a refined mask $d_{\alpha,t}$8, and computes an SRN-style attention map $d_{\alpha,t}$9, producing

$d_{c,t}$0

Training combines branch-wise classification losses and a mask-supervision loss:

$d_{c,t}$1

On WIDER-Attribute, Da-HAR reaches $d_{c,t}$2 mAP over $d_{c,t}$3 attributes, compared with $d_{c,t}$4 for the ResNet-101 baseline and $d_{c,t}$5 for DIAA (Wu et al., 2019). On RAP, Da-HAR + weighted BCE achieves $d_{c,t}$6 and the highest recall of $d_{c,t}$7.

A different vision application aims not to refocus attention but to make a representation blind to distractor classes (Mendez et al., 2021). The objective is to force the latent code $d_{c,t}$8 to satisfy

$d_{c,t}$9

using a Siamese hierarchical VQ-VAE-2 trained on pairs of clean and distractor-overlaid images. The method combines codebook loss, reconstruction losses, a Siamese latent alignment loss $QK^\top$00, and a Siamese reconstruction-overlap loss. The resulting blind latent code is frozen and used for pose regression. On a six-floor multistorey carpark localisation task, the Car-Blind VQ-VAE PoseNet reduces median error on floor 1 from $QK^\top$01 m to $QK^\top$02 m on D1T1 and from $QK^\top$03 m to $QK^\top$04 m on D1T2, while in a full six-floor setting the blind model improves from $QK^\top$05 m to $QK^\top$06 m on D1T1 and from $QK^\top$07 m to $QK^\top$08 m on D2T1 (Mendez et al., 2021).

Predictive modeling of visual distraction during visual search uses both region-level and object-level formulations (Samiei et al., 2022). The pixel-level model is a two-stream encoder–decoder with a search stream and a target stream, shared VGG-16 feature extractors, ASPP, top-down modulation by cross-correlation, and KL-divergence loss:

$QK^\top$09

On COCO-Search18, it achieves AUC-Judd $QK^\top$10, AUC-Borji $QK^\top$11, sAUC $QK^\top$12, NSS $QK^\top$13, KLD $QK^\top$14, CC $QK^\top$15, SIM $QK^\top$16, and IG $QK^\top$17 (Samiei et al., 2022). The object-based method fine-tunes Mask R-CNN to classify target versus distractor instances and reports average $QK^\top$18, $QK^\top$19, and $QK^\top$20 over bottle, bowl, and car.

5. Retrieval-augmented and memory-augmented multimodal systems

In retrieval-augmented LVLMs, attention distraction is formalized as both cross-modal suppression and intra-image drift (Zhao et al., 30 Jan 2026). Let $QK^\top$21, $QK^\top$22, and $QK^\top$23 denote image tokens, question tokens, and retrieved context tokens, with attention weights $QK^\top$24 at generation step $QK^\top$25. The per-step image-attention ratio and context-attention ratio are

$QK^\top$26

Cross-modal distraction is the average drop in image attention from closed-book to RAG:

$QK^\top$27

The paper reports $QK^\top$28 across models and datasets (Zhao et al., 30 Jan 2026). Intra-image distraction is measured by the $QK^\top$29 distance between average image-token heat maps under closed-book and RAG.

The proposed mitigation, MAD-RAG, is training-free and uses a dual-question prompt

$QK^\top$30

instead of $QK^\top$31. $QK^\top$32 grounds on the image without access to $QK^\top$33, while $QK^\top$34 integrates the retrieved context. Attention mixing then injects a fraction of purely visual attention into the context-conditioned question:

$QK^\top$35

or equivalently

$QK^\top$36

With $QK^\top$37, MAD-RAG improves LLaVA-1.5-7B from $QK^\top$38 to $QK^\top$39 on OK-VQA, from $QK^\top$40 to $QK^\top$41 on E-VQA, and from $QK^\top$42 to $QK^\top$43 on InfoSeek, while recovering up to $QK^\top$44 of the “closed-book correct, RAG wrong” failure cases on OK-VQA (Zhao et al., 30 Jan 2026). The paper reports only a $QK^\top$45 inference-time overhead.

Memory-augmented video tracking addresses distractors through a different mechanism (Videnovic et al., 17 Sep 2025). DAM4SAM replaces SAM2’s single FIFO memory with a Recent-Appearance Memory (RAM) and a Distractor-Resolving Memory (DRM), with $QK^\top$46, $QK^\top$47, and $QK^\top$48 plus a reserved anchor slot for the initial frame. Readout uses multi-head cross-attention over memory keys and values:

$QK^\top$49

DRM updates occur only when the tracker is reliable and a distractor is detected, using conditions based on $QK^\top$50, $QK^\top$51, $QK^\top$52, and $QK^\top$53 (Videnovic et al., 17 Sep 2025). On DiDi, DAM4SAM raises the custom Q-score from $QK^\top$54 to $QK^\top$55; on VOT2022 it raises EAO from $QK^\top$56 to $QK^\top$57; on LaSoT it raises AUC from $QK^\top$58 to $QK^\top$59; and on LVOS v2 it raises $QK^\top$60 from $QK^\top$61 to $QK^\top$62.

6. DistrAttention as an efficient self-attention kernel

A distinct usage of the term is the approximate self-attention mechanism “DistrAttention” for modern GPUs (Jin et al., 23 Jul 2025). The starting point is the distributive decomposition

$QK^\top$63

where $QK^\top$64 is the $QK^\top$65-th column of $QK^\top$66 and $QK^\top$67 the $QK^\top$68-th row of $QK^\top$69. The method partitions $QK^\top$70 into groups $QK^\top$71 of size $QK^\top$72, samples one representative $QK^\top$73 per group, and fuses the corresponding key rows,

$QK^\top$74

yielding the approximation

$QK^\top$75

The theoretical multiply cost drops from $QK^\top$76 to $QK^\top$77.

Grouping is performed with locality-sensitive hashing. For a column $QK^\top$78, the method computes $QK^\top$79 using a random projection matrix $QK^\top$80 with $QK^\top$81, binarizes with $QK^\top$82, maps to an integer through Gray-code, sorts hash values, and partitions the permuted indices into consecutive runs of length $QK^\top$83. A block-wise grouping framework is then aligned with FlashAttention-2’s double-loop structure so that LSH, grouping, and fusion can be implemented inside the outer loop over $QK^\top$84-blocks. The paper gives the entrywise $QK^\top$85 error expression

$QK^\top$86

and argues that LSH keeps group members close, thereby limiting approximation error (Jin et al., 23 Jul 2025).

Empirically, the paper reports that DistrAttention is $QK^\top$87 faster than FlashAttention-2 on calculating self-attention. In ViT inference, standard attention achieves ACC1 $QK^\top$88, ACC5 $QK^\top$89, and inference time $QK^\top$90 s, whereas DistrAttention+Flash2 achieves ACC1 $QK^\top$91, ACC5 $QK^\top$92, and time $QK^\top$93 s. In Llama3-1B with $QK^\top$94 prefix, standard attention yields $QK^\top$95 s and ACC $QK^\top$96, while DistrAttention+Flash2 yields $QK^\top$97 s and ACC $QK^\top$98 (Jin et al., 23 Jul 2025). This is a computational, rather than semantic, interpretation of distraction: the mechanism does not suppress task-irrelevant content but approximates full-context attention more efficiently by reducing effective dimensionality.

7. Conceptual synthesis and recurring design patterns

Across these papers, several recurring design patterns are explicit. One pattern is history-aware penalization: M1 and M2 in summarization subtract previously used content or attention mass, while dialogue DistrAttention penalizes attention on masked distracting positions during training (Chen et al., 2016). A second is token or region suppression: AF prunes low-importance patches in ViT-based GCD, Da-HAR masks coarse and fine distraction regions, and Neural Blindness removes distractor-class information from the latent space itself (Xu et al., 18 Jul 2025). A third is architectural decoupling: MAD-RAG separates visual grounding from context integration through the dual-question formulation, and DAM4SAM separates short-term appearance memory from distractor-resolving memory (Zhao et al., 30 Jan 2026). A fourth is lightweight intervention at inference time: focus directions modify only the key and query activations of selected contextual heads, and MAD-RAG is explicitly training-free (Zhu et al., 30 Mar 2025).

A common misconception would be to treat all uses of “DistrAttention” as instances of the same method. The literature in the supplied corpus does not support that interpretation. The summarization mechanism, the dialogue optimization strategy, the ViT background-pruning module, the long-context head-steering intervention, the LVLM dual-question remedy, the distraction-aware CNNs for HAR and visual search, the blindness-based localisation method, the distractor-aware tracker memory, and the GPU kernel named DistrAttention are technically different objects. Their shared concern is the allocation, suppression, or approximation of attention under distractors, but the operational definitions of distraction differ across tasks.

A plausible implication is that the term has evolved from a sequence-modeling design principle—“penalize what you have already seen”—into a broader diagnostic and systems concept covering background patches, irrelevant dialogue turns, retrieved text, distractor objects, and compute-efficient attention approximations. The supplied papers jointly indicate that distraction can be treated as a measurable failure mode, an optimization target, an architectural prior, or a kernel-level efficiency problem, depending on the application domain.