Decoupling and Aggregating Attention Control

• Decoupling and aggregating attention control is a design framework that separates attention functions into distinct computational roles, enabling explicit control over information flow.
• It employs mechanisms like variational bottlenecks, selective head aggregation, and adaptive gating to refine signal extraction and noise suppression in transformers.
• This approach enhances model interpretability and performance by isolating subcircuits, controlling communication budgets, and enabling targeted interventions.

Decoupling and aggregating attention control denotes a family of design and intervention strategies in which functions ordinarily entangled inside a single attention mechanism are separated into distinct computational roles and then recombined under explicit control. Recent work instantiates this pattern by making attention-mediated communication an explicitly budgeted information channel in vision transformers, localizing arbitrary concepts to sparse attention-head modules for scalar intervention, separating signal extraction from noise control, decoupling attention routing from value magnitude at inference time, splitting text and image conditioning or spatial and spectral dependencies in multimodal generators, and distinguishing erase from write in recurrent linear-attention memory updates (Murphy, 4 Feb 2026, Su et al., 20 Jun 2025, Lim et al., 8 Oct 2025, Zhao et al., 1 Jun 2026, Yu et al., 29 Dec 2025, Li et al., 10 Jun 2025, Hatamizadeh et al., 21 May 2026). The common objective is not merely to alter attention weights, but to control information flow, specialization, or interference in a way that is measurable, sparse, or structurally constrained.

1. Conceptual pattern and recurring design axes

Across the recent literature, attention control is implemented by separating distinct roles that standard attention often conflates: communication versus local computation, concept-specific subcircuits versus the rest of the model, signal versus noise, routing versus gain, text semantics versus subject identity, subject versus subject, spatial versus spectral dependency, erase versus write, and structure refinement versus message weighting (Murphy, 4 Feb 2026, Su et al., 20 Jun 2025, Lim et al., 8 Oct 2025, Zhao et al., 1 Jun 2026, Yu et al., 29 Dec 2025, Li et al., 10 Jun 2025, Hatamizadeh et al., 21 May 2026, Chen et al., 2021). This suggests that “decoupling” is not a single mechanism but a recurring systems principle: isolate the controllable subproblem, then define an aggregation rule, gate, or budget that governs reintegration.

Work	Decoupled elements	Aggregation or control mechanism
(Murphy, 4 Feb 2026)	Attention-mediated writes to the residual stream	Per-head VIBs with a tunable information cost
(Su et al., 20 Jun 2025)	Concept-associated head module vs rest of transformer	TopK head selection and scalar rescaling
(Park et al., 7 Apr 2026)	Relevant vs irrelevant diffusion cross-attention heads	Selective aggregation of top-relevance heads
(Lim et al., 8 Oct 2025)	Signal-preserving vs noise-control attention groups	Subtractive attention with ratio-aware head allocation
(Zhao et al., 1 Jun 2026)	Attention routing vs information magnitude	Raw-score-based gain on values with unchanged softmax
(Yu et al., 29 Dec 2025)	Text vs image conditioning; subject vs subject regions	Summation of decoupled streams and crop-and-merge
(Li et al., 10 Jun 2025)	Spatial vs spectral attention	Adaptive fusion gate and GFFN
(Hatamizadeh et al., 21 May 2026)	Erase vs write in linear attention	Separate channel-wise gates
(Chen et al., 2021)	Hard label-based structure attention vs soft feature attention	Stable graph re-weighting in EM
(Jiang et al., 9 Feb 2026)	Meta-path attention weights vs explanation-derived importance	MP-AEA alignment analysis

A central implication of this pattern is that aggregation is treated as a first-class design choice rather than a passive consequence of dense attention. In some systems aggregation is additive, as in $$Z_{\mathrm{out}} = Z_{\mathrm{text}} + Z_{\mathrm{image}}$$ in AnyMS or the fused spatial–spectral stream in STNet; in others it is selective, such as TopK head modules or top-relevance head averaging; in still others it is subtractive or gain-modulated, as in Grouped Differential Attention and RCA.

2. Information-budgeted coordination in vision transformers

A particularly explicit formulation appears in "From independent patches to coordinated attention: Controlling information flow in vision transformers" (Murphy, 4 Feb 2026). That work inserts variational information bottlenecks on every attention head’s write to the residual stream, without changing the surrounding ViT architecture. If $\Delta_i^\ell$ is the attention update for patch $i$ at layer $\ell$, the deterministic write is replaced by a stochastic channel,

$$\Delta_i^\ell \rightarrow z_i^\ell \sim q_\phi(z \mid \Delta_i^\ell) \rightarrow \hat{\Delta}_i^\ell = g_\theta(z_i^\ell),$$

with Gaussian posterior $$q_\phi(z \mid \Delta)=\mathcal{N}(\mu_\phi(\Delta), \mathrm{diag}(\sigma_\phi^2(\Delta)))$$ and standard normal prior $p(z)=\mathcal{N}(0,I)$. Training minimizes a classification term plus a KL penalty scaled by $\beta$, making the transmitted information of each head an explicit cost. In ViT-Tiny from timm this yields 36 bottlenecks, one per head per layer, and produces a continuous spectrum from a zero-communication limit—independent patch processing with global average pooling and a DeepSets-like patch-voting interpretation—to the usual globally mixing ViT. On ImageNet-100, the measured spectrum spans roughly $69.2\%$ to $78.2\%$ top-1 accuracy.

The empirical importance of this construction is that classification behavior and information routing vary continuously rather than binarily. At low communication budgets, patches disagree strongly, producing many distinct patch-level top predictions and large logit ranges; the paper quantifies this with the inverse Simpson index for diversity of patch votes and the min-to-max range of patch logits. As the budget increases, patch predictions become more coordinated and consensus-like. This establishes a concrete operational meaning of decoupling: patches begin as independent voters and then become coordinated as attention-mediated communication is gradually restored.

The same framework is also used as a mechanistic probe. In low-information regimes, most heads collapse to the prior and become effectively inactive. A head is defined as active if its KL exceeds $\Delta_i^\ell$0 nats for at least one validation patch. In the strongest bottleneck regime only one head is active; with a milder bottleneck four heads are active; under weaker constraints progressively more heads participate until all 36 contribute nontrivially. The earliest useful communication channels appear to encode coarse redundancy or repetition signals. Under patch copy-paste augmentations, the latent mean of the active low-bandwidth head shifts systematically when a patch is duplicated to multiple locations, supporting the claim that one of the most information-efficient early global messages is repetition-like context. Head behavior is compared across models and random seeds with a normalized mutual information analysis based on Monte Carlo estimates of $\Delta_i^\ell$1 and $\Delta_i^\ell$2, and the repetition-sensitive head is reported to be robust across seeds.

3. Sparse head modules and selective aggregation

A second major line of work treats attention heads as sparse, concept-associated modules that can be discovered and manipulated without modifying the whole model. "From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformers" introduces Scalable Attention Module Discovery (SAMD), which represents a concept as a vector $\Delta_i^\ell$3, scores every attention head by the average cosine similarity between head output and that concept vector over a concept dataset, and selects the TopK heads globally across layers as the module (Su et al., 20 Jun 2025). The residual-stream decomposition is written as

$\Delta_i^\ell$4

and the key discovery rule is

$\Delta_i^\ell$5

Scalar Attention Module Intervention (SAMI) then rescales only the selected heads by a scalar $\Delta_i^\ell$6, leaving the rest of the transformer intact. The reported module sizes are small: typically 5 heads for SAE concepts and reasoning, 10 for safety, and 3 for ViT label concepts. The empirical results are correspondingly targeted: on FQuAD validation data, default $\Delta_i^\ell$7 reduces the French response ratio from $\Delta_i^\ell$8 to $\Delta_i^\ell$9, outperforming SAE steering’s best reported $i$0; on HarmBench, negative intervention on a safety module yields attack success rates of $i$1 for Llama-2 7B, $i$2 for Qwen 7B, and $i$3 for Gemma 7B; on GSM8K, positive intervention improves Llama-3.1-8B-Instruct from $i$4 to $i$5 and Gemma-7B-Base from $i$6 to $i$7, with roughly unchanged CommonsenseQA, little to no degradation on Humaneval+, minimal change on MBPP+, and negligible decline on MT-bench. In ViTs, label-associated recognition modules of only 3 heads, mostly in the final layers, can be intervened on so that recognition of the attacked label falls to $i$8 while much of overall generalization on other labels is preserved.

Selective head aggregation is also the central move in diffusion-model interpretability. "Selective Aggregation of Attention Maps Improves Diffusion-Based Visual Interpretation" argues that DAAM’s average over all heads and timesteps mixes relevant and irrelevant attention and replaces it with aggregation over only the most concept-relevant heads identified by HRV (Park et al., 7 Apr 2026). In Stable Diffusion v1.4, the method selects the top $i$9 of heads by relevance score and uses 30 heads out of 128 in the main experiments. On the Animals benchmark with prompts of the form “photo of a {animal}” and Grounded-SAM masks, mean IoU improves from $\ell$0 to $\ell$1 at threshold $\ell$2, from $\ell$3 to $\ell$4 at $\ell$5, and from $\ell$6 to $\ell$7 at $\ell$8. The contrast between the 30 most relevant and 30 least relevant heads is large: at threshold $\ell$9, mean IoU drops from $$\Delta_i^\ell \rightarrow z_i^\ell \sim q_\phi(z \mid \Delta_i^\ell) \rightarrow \hat{\Delta}_i^\ell = g_\theta(z_i^\ell),$$0 to $$\Delta_i^\ell \rightarrow z_i^\ell \sim q_\phi(z \mid \Delta_i^\ell) \rightarrow \hat{\Delta}_i^\ell = g_\theta(z_i^\ell),$$1. The paper also reports that 30 selected heads outperform 20 or 40, and that concept-specific head subsets can separate prompt ambiguities such as animal “mouse” versus computer “mouse.”

Taken together, these works recast aggregation from indiscriminate averaging to relevance-conditioned composition. In one case the output of a sparse head set is rescaled; in the other, only a sparse head set is averaged. Both approaches rely on the claim that concept-linked behavior can often be localized to a small, distributed subset of heads.

4. Signal–noise and routing–gain decompositions

Another strand of the literature decomposes the internals of attention by function. "Grouped Differential Attention" starts from Differential Attention, in which a signal attention map and a noise attention map are computed in parallel and the latter is subtracted from the former (Lim et al., 8 Oct 2025). The novelty of GDA is to remove the rigid $$\Delta_i^\ell \rightarrow z_i^\ell \sim q_\phi(z \mid \Delta_i^\ell) \rightarrow \hat{\Delta}_i^\ell = g_\theta(z_i^\ell),$$2 head split and instead allocate more heads to the signal-preserving group and fewer to the noise-control group, with the smaller noise branch stabilized through controlled repetition akin to GQA. If the total number of heads is $$\Delta_i^\ell \rightarrow z_i^\ell \sim q_\phi(z \mid \Delta_i^\ell) \rightarrow \hat{\Delta}_i^\ell = g_\theta(z_i^\ell),$$3, GDA divides them into $$\Delta_i^\ell \rightarrow z_i^\ell \sim q_\phi(z \mid \Delta_i^\ell) \rightarrow \hat{\Delta}_i^\ell = g_\theta(z_i^\ell),$$4 groups and gives the signal side $$\Delta_i^\ell \rightarrow z_i^\ell \sim q_\phi(z \mid \Delta_i^\ell) \rightarrow \hat{\Delta}_i^\ell = g_\theta(z_i^\ell),$$5 heads and the noise side $$\Delta_i^\ell \rightarrow z_i^\ell \sim q_\phi(z \mid \Delta_i^\ell) \rightarrow \hat{\Delta}_i^\ell = g_\theta(z_i^\ell),$$6. The paper reports that moderate imbalance is best in 0.9B-parameter pretraining with 48 heads: $$\Delta_i^\ell \rightarrow z_i^\ell \sim q_\phi(z \mid \Delta_i^\ell) \rightarrow \hat{\Delta}_i^\ell = g_\theta(z_i^\ell),$$7 gives the best average gain of $$\Delta_i^\ell \rightarrow z_i^\ell \sim q_\phi(z \mid \Delta_i^\ell) \rightarrow \hat{\Delta}_i^\ell = g_\theta(z_i^\ell),$$8, $$\Delta_i^\ell \rightarrow z_i^\ell \sim q_\phi(z \mid \Delta_i^\ell) \rightarrow \hat{\Delta}_i^\ell = g_\theta(z_i^\ell),$$9 is near-neutral at $$q_\phi(z \mid \Delta)=\mathcal{N}(\mu_\phi(\Delta), \mathrm{diag}(\sigma_\phi^2(\Delta)))$$0, and $$q_\phi(z \mid \Delta)=\mathcal{N}(\mu_\phi(\Delta), \mathrm{diag}(\sigma_\phi^2(\Delta)))$$1 degrades by $$q_\phi(z \mid \Delta)=\mathcal{N}(\mu_\phi(\Delta), \mathrm{diag}(\sigma_\phi^2(\Delta)))$$2. In progressive continual training from roughly 400M to roughly 1.6B parameters, $$q_\phi(z \mid \Delta)=\mathcal{N}(\mu_\phi(\Delta), \mathrm{diag}(\sigma_\phi^2(\Delta)))$$3 yields the best result with a $$q_\phi(z \mid \Delta)=\mathcal{N}(\mu_\phi(\Delta), \mathrm{diag}(\sigma_\phi^2(\Delta)))$$4 average gain over the balanced baseline, while $$q_\phi(z \mid \Delta)=\mathcal{N}(\mu_\phi(\Delta), \mathrm{diag}(\sigma_\phi^2(\Delta)))$$5 gives $$q_\phi(z \mid \Delta)=\mathcal{N}(\mu_\phi(\Delta), \mathrm{diag}(\sigma_\phi^2(\Delta)))$$6. The architectural claim is that differential denoising is useful, but equal resource allocation to signal and noise is not.

"Resonant Context Anchoring: Decoupling Attention Routing and Signal Gain at Inference Time" separates a different pair of functions: where attention routes information and how much energy the routed information carries (Zhao et al., 1 Jun 2026). Standard attention computes raw scores $$q_\phi(z \mid \Delta)=\mathcal{N}(\mu_\phi(\Delta), \mathrm{diag}(\sigma_\phi^2(\Delta)))$$7, softmax routing probabilities $$q_\phi(z \mid \Delta)=\mathcal{N}(\mu_\phi(\Delta), \mathrm{diag}(\sigma_\phi^2(\Delta)))$$8, and then $$q_\phi(z \mid \Delta)=\mathcal{N}(\mu_\phi(\Delta), \mathrm{diag}(\sigma_\phi^2(\Delta)))$$9. RCA leaves the softmax distribution untouched but uses the raw pre-softmax scores as an instantaneous metric of semantic alignment to construct a gain field,

$p(z)=\mathcal{N}(0,I)$0

and modulates values in situ via $p(z)=\mathcal{N}(0,I)$1. The method is inserted at the value projection stage inside self-attention, is training-free, and is described as computationally negligible. The paper’s SNR argument is that relevant contextual values are amplified while the noise term remains approximately unchanged, so contextual SNR increases monotonically. On XSum, RCA improves Llama-3-8B-Instruct from $p(z)=\mathcal{N}(0,I)$2 to $p(z)=\mathcal{N}(0,I)$3 FactKB and from $p(z)=\mathcal{N}(0,I)$4 to $p(z)=\mathcal{N}(0,I)$5 AlignScore, while ROUGE-L moves from $p(z)=\mathcal{N}(0,I)$6 to $p(z)=\mathcal{N}(0,I)$7; on Llama-3-70B-Instruct, FactKB rises from $p(z)=\mathcal{N}(0,I)$8 to $p(z)=\mathcal{N}(0,I)$9 and AlignScore from $\beta$0 to $\beta$1, with ROUGE-L changing from $\beta$2 to $\beta$3. On NQ-Swap, exact match improves from $\beta$4 to $\beta$5 for 8B and from $\beta$6 to $\beta$7 for 70B. On MemoTrap, Llama-3-70B micro accuracy increases from $\beta$8 to $\beta$9 and macro accuracy from $69.2\%$0 to $69.2\%$1. TruthfulQA, TriviaQA, and PopQA change negligibly. The reported sweet spots are $69.2\%$2 for Llama-3-8B and $69.2\%$3 for Llama-3-70B; performance begins to drop around $69.2\%$4, and by $69.2\%$5 generation quality and perplexity worsen sharply.

These two approaches differ mechanistically—subtractive head grouping versus value-norm amplification—but both reject the idea that attention should be a single undifferentiated operation. One decouples denoising from representational capacity; the other decouples normalized routing from residual-stream magnitude.

5. Multimodal, layout, and spectral–spatial decoupling

In diffusion-based image synthesis, decoupling often targets conditioning conflicts. "AnyMS: Bottom-up Attention Decoupling for Layout-guided and Training-free Multi-subject Customization" addresses the competition among text alignment, subject identity preservation, and layout control by introducing a bottom-up dual-level attention decoupling mechanism (Yu et al., 29 Dec 2025). At the global level, text cross-attention and image cross-attention are computed separately and summed as

$69.2\%$6

At the local level, each subject image $69.2\%$7 with box $69.2\%$8 is restricted to its designated latent subregion through crop-and-merge attention: the query subregion $69.2\%$9 is extracted, attention is computed only within that region, and the result is written back to $78.2\%$0. For overlapping regions, conflicts are resolved by semantic priority, such as $78.2\%$1 or $78.2\%$2. The system uses pre-trained image adapters, instantiated with IP-Adapter in the experiments, so no subject learning, LoRA tuning, or adapter training is required. On 11 multi-subject composition cases with 2–5 subjects, the reported table gives AP50 $78.2\%$3, mIOU $78.2\%$4, CLIP-I $78.2\%$5, DreamSim $78.2\%$6, and CLIP-T $78.2\%$7. Ablations on seven combinations with more than three subjects show that removing local decoupling or crop-and-merge harms performance: without local decoupling, CLIP-I is $78.2\%$8 and CLIP-T $78.2\%$9; without crop-and-merge, mIOU is $\Delta_i^\ell$00, CLIP-I $\Delta_i^\ell$01, and CLIP-T $\Delta_i^\ell$02; full AnyMS reaches mIOU $\Delta_i^\ell$03, CLIP-I $\Delta_i^\ell$04, and CLIP-T $\Delta_i^\ell$05.

A related but domain-specific instance appears in hyperspectral image classification. "Hyperspectral Image Classification via Transformer-based Spectral-Spatial Attention Decoupling and Adaptive Gating" introduces STNet, a 3D-DenseNet-style framework with a SpatioTemporalTransformer module that explicitly separates spatial attention from spectral attention (Li et al., 10 Jun 2025). The spatial branch attends over intra-band spatial dependencies by reshaping spatial positions into the sequence dimension, while the spectral branch mean-pools over spatial dimensions and attends over inter-band spectral correlations. Their outputs are combined by an adaptive attention fusion gate,

$\Delta_i^\ell$06

and the post-attention transformation is further filtered by a gated feed-forward network, $\Delta_i^\ell$07. The model also uses learnable and interpolatable 3D positional encoding. Reported dataset-level results are strong: on Indian Pines, STNet achieves OA $\Delta_i^\ell$08, AA $\Delta_i^\ell$09, and Kappa $\Delta_i^\ell$10; on Pavia University, OA $\Delta_i^\ell$11, AA $\Delta_i^\ell$12, and Kappa $\Delta_i^\ell$13; the text states that STNet also performs best on KSC. STNet-base versus STNet-large on Indian Pines changes OA from $\Delta_i^\ell$14 to $\Delta_i^\ell$15, AA from $\Delta_i^\ell$16 to $\Delta_i^\ell$17, and Kappa from $\Delta_i^\ell$18 to $\Delta_i^\ell$19, which the paper interprets as support for the importance of module design rather than only model scale.

Both AnyMS and STNet treat decoupling as a way to prevent interference among heterogeneous information sources. In AnyMS the competing sources are prompt semantics, subject identity, and layout constraints; in STNet they are spatial and spectral dependencies in high-dimensional HSI cubes. In both cases aggregation is adaptive rather than fixed.

6. Memory editing and graph propagation

The decoupling motif extends beyond standard softmax attention. "Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention" argues that prior delta-rule and KDA-style recurrent memories use a single scalar gate to control two logically distinct actions: erasing old content on the key side and committing new content on the value side (Hatamizadeh et al., 21 May 2026). Gated DeltaNet-2 introduces a channel-wise erase gate $\Delta_i^\ell$20, a channel-wise write gate $\Delta_i^\ell$21, and keeps channel-wise decay $\Delta_i^\ell$22. The core recurrence is

$\Delta_i^\ell$23

When both gates collapse to the same scalar the model reduces to KDA; when decay also collapses to a scalar it reduces to Gated DeltaNet. The paper derives a fast-weight update view, a chunkwise WY algorithm with channel-wise decay absorbed into asymmetric erase factors, and a gate-aware backward pass that preserves efficient parallel training. At 1.3B parameters trained on 100B FineWeb-Edu tokens, Gated DeltaNet-2 achieves the strongest overall results among Mamba-2, Gated DeltaNet, KDA, and Mamba-3 variants across language modeling, commonsense reasoning, and retrieval. Its most pronounced advantage is on long-context RULER multi-key retrieval: on MK-NIAH-1, the recurrent setting reaches $\Delta_i^\ell$24 at 1K / 2K / 4K and the hybrid setting $\Delta_i^\ell$25.

Graph learning offers another explicit split between structure and weighting. "Graph Decoupling Attention Markov Networks for Semi-supervised Graph Node Classification" introduces GDAMN, which separates hard attention on labels from soft attention on features (Chen et al., 2021). Hard attention refines the graph structure by suppressing likely inter-class edges through

$\Delta_i^\ell$26

while soft attention then learns aggregation weights on the refined graph using feature relations. The model is trained in an EM framework: the M-step learns refined structure and soft weights, and the E-step performs feature propagation on a stable graph re-weighting

$\Delta_i^\ell$27

Reported results show GDAMN at $\Delta_i^\ell$28 on Cora, $\Delta_i^\ell$29 on Citeseer, $\Delta_i^\ell$30 on Pubmed, $\Delta_i^\ell$31 on Coauthor-CS, $\Delta_i^\ell$32 on Coauthor-Phy, and $\Delta_i^\ell$33 on ogbn-arXiv, with ablations indicating that removing ENT, Hard, or Soft harms performance. The model is presented as a response to “negative disturbance” from inter-class edges: structure learning and message weighting are treated as separate problems.

These recurrent and graph-based formulations reinforce the broader point that attention control often becomes most transparent when decomposed into edit primitives. In linear attention the primitives are forgetting, erasing, and writing; in graph propagation they are structure refinement and neighborhood weighting.

7. Interpretability, explanation, and limits of control

A persistent issue in this literature is whether attention-based control yields faithful explanation or only useful intervention. "Is Meta-Path Attention an Explanation? Evidence of Alignment and Decoupling in Heterogeneous GNNs" addresses this directly for meta-path-based heterogeneous GNNs (Jiang et al., 9 Feb 2026). The paper introduces MetaXplain, which keeps explanations in the native meta-path view domain through view-factorized explanations, schema-valid channel-wise perturbations, and fusion-aware attribution, and proposes Meta-Path Attention–Explanation Alignment (MP-AEA), a rank-correlation measure between learned semantic attention weights and explanation-derived meta-path contribution scores across random runs. On ACM, DBLP, and IMDB with HAN and HAN-GCN, meta-path-aware explainers generally outperform type-matched random baselines under standard faithfulness metrics, but MP-AEA reveals both high-alignment and statistically significant decoupling regimes. IMDB shows consistently high correlation for both HAN and HAN-GCN, whereas DBLP with HAN shows statistically insignificant correlation. Controlled attention interventions further show that learned attention can over-commit: on IMDB, especially with HAN-GCN, a balanced mixture can outperform the learned attention. Retraining on explanation-induced subgraphs can preserve or even improve predictive performance in some noisy regimes, which the paper describes as an explanation-as-denoising effect.

Other works in the attention-control literature make related caveats. SAMD is explicitly correlation-based and offers no causality guarantee from discovery alone; the identified module may be overcomplete or incomplete, concept monosemanticity is not guaranteed, concept-vector quality matters, and TopK is a heuristic boundary rather than a theoretically derived one (Su et al., 20 Jun 2025). RCA relies on calibration of $\Delta_i^\ell$34, uses a linearized subspace perspective, is evaluated mainly on the Llama-3 series, and can hurt fluency and perplexity when $\Delta_i^\ell$35 becomes too large (Zhao et al., 1 Jun 2026). By contrast, the variationally bottlenecked ViT work frames explicit communication control as an aid to mechanistic analysis because low-information models become sparse, low-bandwidth, and quantitatively ordered by information cost, rather than remaining a dense and entangled attention stack (Murphy, 4 Feb 2026).

The cumulative picture is therefore nuanced. Decoupling attention often increases tractability, controllability, or robustness by isolating specific forms of interference and then imposing an explicit aggregation rule, gate, or budget. Yet the same literature also shows that attention weights, modules, or channels should not automatically be read as faithful explanations. In several domains, the most reliable use of decoupled attention is not as a narrative about what the model “really means,” but as an experimentally testable interface for controlling information flow.