Attention-Augmented State Space Models

• A2SSM is a state-space architecture where multi-scale attention maps are injected to modulate hidden-state processing, effectively integrating cross-attention-like operations.
• It addresses the limitations of sequential SSMs by dynamically capturing two-dimensional spatial dependencies in image data.
• Empirical results demonstrate that A2SSM improves visual recognition performance, yielding notable gains in metrics like ImageNet top-1 accuracy and ADE20K mIoU.

Searching arXiv for the cited A2SSM-related papers to ground the article. arXiv search: A2Mamba attention-augmented state space models visual recognition (Lou et al., 22 Jul 2025) Attention-Augmented State Space Models (A2SSMs) are state-space-based architectures in which attention-derived signals modify the state-space computation itself rather than merely appearing in adjacent or alternating blocks. In the most explicit formulation presently associated with the term, A2SSM denotes the core mechanism inside A2Mamba, where multi-scale attention maps are injected into hidden-state processing within a Mamba-style state-space path for visual recognition (Lou et al., 22 Jul 2025). More broadly, recent work places A2SSMs within a larger landscape of hybrid sequence and vision models that combine recurrent state evolution, semiseparable or state-space structure, and attention-style routing, normalization, or multiplicative interaction (Hu et al., 6 Oct 2025). The central technical theme is a deeper integration between attention and state-space dynamics: attention may spatially aggregate hidden states, parameterize a feedback operator, supply multiplicative statistics for in-context adaptation, or provide post-SSM adaptive scaling, depending on the architecture (Lou et al., 22 Jul 2025).

1. Definition and problem setting

In A2Mamba, A2SSM is introduced to address a specific limitation of prior Transformer–Mamba hybrids for visual recognition: existing methods often combine attention and state-space modules by simple stacking, so attention extracts local context and the SSM performs sequence modeling afterward, but “the two mechanisms remain largely independent” (Lou et al., 22 Jul 2025). The stated problem is that standard scan-based SSMs are not naturally matched to images, because images have two-dimensional spatial structure with non-sequential dependencies, whereas sequential scanning is natural for language. Even with vision-specific scan variants such as SS2D or multiple scan directions, the paper argues that SSMs still suffer from “inherent causality from sequential scanning,” “difficulty comprehensively understanding 2D spatial structure,” and “insufficient dynamic modeling of spatially varying dependencies” (Lou et al., 22 Jul 2025).

A2SSM is presented as the response to that mismatch. It does not replace the state-space backbone with ordinary self-attention, nor does it simply append an attention layer after an SSM. Instead, it introduces what the paper calls “a variant of cross-attention” by spatially aggregating SSM hidden states using precomputed multi-scale attention maps (Lou et al., 22 Jul 2025). In this formulation, the recurrent state evolution remains central, but the hidden-state representation is explicitly modulated by image-space affinities.

This distinguishes A2SSM from several neighboring categories. Relative to vanilla Mamba or a plain SSM, A2SSM augments hidden-state processing rather than relying only on scan-produced hidden states and standard output modulation (Lou et al., 22 Jul 2025). Relative to local-attention transformers, it retains the SSM as the main computational pathway and uses attention maps to aggregate recurrent hidden states rather than performing ordinary token-to-token self-attention (Lou et al., 22 Jul 2025). Relative to earlier hybrids such as SegMAN/LASS or MambaVision, its novelty is the interaction mechanism itself: shared attention maps directly aggregate SSM hidden states rather than merely preceding or following them in depth (Lou et al., 22 Jul 2025).

A broader theoretical context sharpens this definition. Structured State-Space Duality shows that some structured SSMs and some masked attention operators are exactly the same sequence transformation under different algorithmic realizations, but this exact duality applies to semiseparable masked attention, not to standard softmax attention (Hu et al., 6 Oct 2025). This suggests that some apparent attention–SSM hybrids may be algebraically redundant, whereas others—especially those involving softmax attention or non-semiseparable spatial operators—are genuine hybrids. A2SSM as used in A2Mamba belongs to the latter category, because the paper explicitly frames its hidden-state augmentation as a cross-attention-like operation rather than an exact reparameterization of an SSM (Lou et al., 22 Jul 2025).

2. A2SSM in A2Mamba: architectural placement and data flow

A2Mamba is a four-stage hierarchical pyramid backbone for visual recognition, and each A2Mamba block contains a residual $3\times 3$ depthwise convolution for positional enhancement, the proposed MASS token mixer, and a ConvFFN (Lou et al., 22 Jul 2025). The hierarchy is explicit: an A2Mamba block contains MASS, and MASS contains A2SSM as its SSM core (Lou et al., 22 Jul 2025).

MASS, short for Multi-scale Attention-enhanced State Space Model, has two parts: Adaptive Multi-scale Attention (AMA), which generates multi-scale attention maps and associated attended features, and A2SSM, which consumes those maps to augment SSM hidden states (Lou et al., 22 Jul 2025). AMA therefore provides the spatial affinity information, and A2SSM uses that information to enhance SSM-based sequence modeling.

The data flow is defined procedurally. Given input feature map $\mathbf{X}$, MASS first splits channels into two branches. One branch applies sliding local attention (SLA) to produce a local attention map $\mathbf{A}_1$ and output feature $\mathbf{X}_1'$; another branch applies dilated sliding attention (DLA) to produce a dilated attention map $\mathbf{A}_2$ and output feature $\mathbf{X}_2'$ (Lou et al., 22 Jul 2025). These attended features are concatenated to form $\mathbf{Y}$, and $\mathbf{Y}$, together with $\mathbf{A}_1$ and $\mathbf{A}_2$, is fed into A2-SSM (Lou et al., 22 Jul 2025). Inside A2SSM, the model computes SSM hidden states, splits them into two groups, spatially aggregates them with $\mathbf{X}$0 and $\mathbf{X}$1, concatenates the aggregated hidden states, and then performs output modulation and residual/gating as in a Mamba-style SSM (Lou et al., 22 Jul 2025). MASS ends with

$\mathbf{X}$2

The remaining SSM operations follow vanilla SSM, including a weighted residual connection through learnable $\mathbf{X}$3, and A2Mamba blocks use residual structure around the token mixer and FFN (Lou et al., 22 Jul 2025).

The paper gives the overall MASS token mixer as

$\mathbf{X}$4

The source notes that there appears to be a likely typo in the DLA branch, which is written as $\mathbf{X}$5, though symmetry may suggest $\mathbf{X}$6; preserving the paper as written is necessary (Lou et al., 22 Jul 2025).

The adaptive dilation rate for multi-scale attention is

$\mathbf{X}$7

Here $\mathbf{X}$8 and $\mathbf{X}$9 are the spatial height and width of the input feature map, $\mathbf{A}_1$0 is window size, and $\mathbf{A}_1$1 is the dilation rate used by the dilated sliding attention branch (Lou et al., 22 Jul 2025). The stated purpose is to make the dilated sliding window have the same effective size as the input feature map, regardless of resolution, so that attention covers the entire input space (Lou et al., 22 Jul 2025).

3. Core mechanism: hidden-state augmentation as cross-attention-like aggregation

The paper does not provide the full continuous or discrete SSM equations in the visible method section, but it clearly states that A2SSM follows vanilla SSM or Mamba except for the new hidden-state aggregation using attention maps (Lou et al., 22 Jul 2025). The hidden-state handling is therefore the defining operation.

The feature tensor fed into A2SSM is $\mathbf{A}_1$2, and the hidden states produced by the SSM are reshaped and split into two channel-wise halves,

$\mathbf{A}_1$3

The attention maps $\mathbf{A}_1$4 and $\mathbf{A}_1$5 are then applied to $\mathbf{A}_1$6 and $\mathbf{A}_1$7, treating the hidden-state halves as “value” components, producing enhanced hidden-state maps $\mathbf{A}_1$8 (Lou et al., 22 Jul 2025). These are concatenated, multiplied element-wise with reshaped $\mathbf{A}_1$9, and combined with the weighted residual through learnable $\mathbf{X}_1'$0 and input $\mathbf{X}_1'$1, following vanilla SSM (Lou et al., 22 Jul 2025).

The paper explicitly interprets this as “a variant of cross-attention by spatially aggregating the SSM’s hidden states using the multi-scale attention maps” (Lou et al., 22 Jul 2025). The phrase is carefully delimited. It does not mean standard Transformer cross-attention with explicit $\mathbf{X}_1'$2 projections from two token sequences. Rather, the attention maps are precomputed by AMA from the feature map, and those maps are used to spatially reweight and aggregate SSM hidden states (Lou et al., 22 Jul 2025). Thus the affinities come from one computational stream and the values from another, which is why the operation is characterized as cross-attention-like rather than self-attention (Lou et al., 22 Jul 2025).

The mechanism is described in five stages. AMA computes $\mathbf{X}_1'$3 from regular sliding local attention and $\mathbf{X}_1'$4 from dilated sliding attention, so the attention maps are dynamic, input-dependent, and inherently two-dimensional (Lou et al., 22 Jul 2025). The concatenated feature $\mathbf{X}_1'$5 passes through the SSM path, producing hidden states $\mathbf{X}_1'$6, preserving efficient long-range state propagation and dynamic token-dependent modeling (Lou et al., 22 Jul 2025). Those hidden states are split into $\mathbf{X}_1'$7, interpreted as value tensors, and spatially aggregated using $\mathbf{X}_1'$8 (Lou et al., 22 Jul 2025). The resulting $\mathbf{X}_1'$9 are concatenated, multiplied with $\mathbf{A}_2$0, and combined with the weighted residual $\mathbf{A}_2$1 to produce the final SSM output (Lou et al., 22 Jul 2025).

The role of multi-scale attention is central. AMA combines SLA for dense short-range affinities and DLA for sparse long-range affinities, motivated by the presence of both fine local structures and larger contextual relationships in images (Lou et al., 22 Jul 2025). The adaptive dilation rate lets the dilated branch scale with resolution and effectively span the input space, which the paper states is especially useful for dense prediction (Lou et al., 22 Jul 2025). The resulting hidden-state maps are claimed to have “dynamically enhanced spatial coherence and dependency pertaining to a two-dimensional space instead of a one-dimensional sequence,” with reduced causality effects from sequential scanning (Lou et al., 22 Jul 2025).

This design also informs how A2SSM relates to a more abstract theory of attention and SSMs. A unified operator framework shows that attention-style models and SSMs differ in whether pairwise operators are explicitly factorized or implicitly induced by latent dynamics, and the paper proves that stable linear dynamics exhibit distance-dependent gradient attenuation whereas attention layers admit inputs with distance-independent gradient paths (Ghodsi, 17 Dec 2025). A plausible implication is that A2SSM-style augmentation can be read as an attempt to preserve the structured dynamical advantages of an SSM while importing some of the routing flexibility associated with attention, though A2Mamba itself does not formulate the design in those theoretical terms (Lou et al., 22 Jul 2025).

4. Mathematical and theoretical context

Several papers situate attention-augmented SSMs within a common mathematical landscape. Their role is not to define A2SSM in the narrow A2Mamba sense, but to clarify when attention and SSM operations are exactly equivalent, when they are merely analogous, and when true hybridization is unavoidable.

The most direct structural result is Structured State-Space Duality. For a time-varying linear SSM

$\mathbf{A}_2$2

with $\mathbf{A}_2$3, unrolling yields

$\mathbf{A}_2$4

Thus an SSM induces a lower-triangular sequence-to-sequence matrix $\mathbf{A}_2$5 (Hu et al., 6 Oct 2025). When $\mathbf{A}_2$6, the resulting operator is exactly a $\mathbf{A}_2$7-semiseparable masked attention operator: $\mathbf{A}_2$8 The 2025 note extends this from scalar-times-identity transitions to general diagonal $\mathbf{A}_2$9, proving that diagonal SSMs admit an additive decomposition into $\mathbf{X}_2'$0 scalar-SSD channels and, under full-rank diagonal transitions, can be rewritten as a single $\mathbf{X}_2'$1-semiseparable masked attention operator (Hu et al., 6 Oct 2025).

The same paper states a necessary and sufficient condition for when a general $\mathbf{X}_2'$2-dimensional SSM has a $\mathbf{X}_2'$3-SS masked-attention dual: the corresponding $\mathbf{X}_2'$4-SS matrix must have several diagonal blocks containing all the nonzero entries, and each diagonal block must have at most $\mathbf{X}_2'$5 new columns (Hu et al., 6 Oct 2025). It also proves that this duality fails for standard softmax attention because exponentiation produces rank explosion (Hu et al., 6 Oct 2025). In the context of A2SSM, this matters because it distinguishes exact semiseparable dualities from genuine hybrids involving attention operations that are not reducible to recurrence.

A second line of theory, the Dynamical Systems Framework, rewrites attention, SSMs, selective SSMs such as S6/Mamba, and some RNNs as instances of a common linear structured time-varying recurrence

$\mathbf{X}_2'$6

with diagonal $\mathbf{X}_2'$7 (Sieber et al., 2024). Under this view, linear or separable attention can also be written recurrently if the attention kernel is separable, but softmax attention requires an infinite-dimensional hidden state (Sieber et al., 2024). The paper argues that selective, input-dependent parameterization in SSMs plays a role analogous to attention-like content selectivity, and that normalization structure may matter more than simply increasing the number of transition parameters (Sieber et al., 2024). This suggests that “attention augmentation” need not always mean an explicit softmax attention module; it may also take the form of input-dependent query/key/state-normalization structure inside an SSM-like recurrence (Sieber et al., 2024).

A third theory paper frames the comparison in terms of effective interaction operators $\mathbf{X}_2'$8, proving an Interaction Rank Gap for single-head factorized attention-style models, an Equivalence (Head-Count) Theorem for multi-head factorized models representing linear SSM lag-operator families, and a Gradient Highway Result showing that attention admits distance-independent gradient paths while stable linear dynamics attenuate gradients with temporal distance (Ghodsi, 17 Dec 2025). Its core message for hybrid models is that SSMs provide structured lag-dependent operator families, while attention provides flexible input-dependent mixing and optimization-friendly long-range shortcuts (Ghodsi, 17 Dec 2025).

Across these analyses, a consistent picture emerges. Some masked linear or semiseparable attention operators are exactly recurrent SSMs under another parameterization (Hu et al., 6 Oct 2025). Softmax attention is not (Hu et al., 6 Oct 2025, Sieber et al., 2024). Stable linear dynamics have structured expressive advantages but may suffer from long-range gradient attenuation (Ghodsi, 17 Dec 2025). Attention introduces dynamic routing and gradient highways, but factorized attention-style models can be operator-rank limited unless head count is sufficient (Ghodsi, 17 Dec 2025). This suggests that the practical meaning of A2SSM varies by design: in some cases it is a true hybrid, in others a dual representation, and in still others an SSM with attention-inspired parameterization.

5. Architectural variants and neighboring formulations

Although A2SSM is most explicitly named in A2Mamba, several other recent models instantiate closely related principles.

Sessa places attention inside a feedback path rather than beside a recurrent block. Its mixer constructs a forward causal attention signal $\mathbf{X}_2'$9 and a strict-past feedback attention distribution $\mathbf{Y}$0, together with a scalar token-dependent gain

$\mathbf{Y}$1

These define a strictly lower-triangular routing matrix

$\mathbf{Y}$2

and the mixer output is the solution of

$\mathbf{Y}$3

equivalently

$\mathbf{Y}$4

The paper explicitly characterizes this as attention inserted into the recurrent path rather than used in a single direct read, yielding “many-path aggregation” through the finite Neumann expansion

$\mathbf{Y}$5

on a finite horizon (Horbatko, 20 Apr 2026). In this taxonomy, Transformers are direct-read systems, standard and selective SSMs are single-chain feedback systems, and Sessa is a feedback system whose recurrent operator is attention-routed (Horbatko, 20 Apr 2026). This places it close to the strongest possible architectural meaning of attention-augmented state space modeling.

A different form of augmentation appears in “State-space models can learn in-context by gradient descent,” which proves that a structured SSM layer with multiplicative input and output gating can reproduce the outputs of an implicit linear model after one step of gradient descent (Sushma et al., 2024). In the vector case, the recurrence stores a matrix-valued gradient accumulator

$\mathbf{Y}$6

and the readout applies it to the next query,

$\mathbf{Y}$7

Crucially, a local self-attention window of size $\mathbf{Y}$8 is used to construct terms such as $\mathbf{Y}$9 and select $\mathbf{Y}$0, because a pure first-order recurrence over single tokens cannot access the needed second-order interactions (Sushma et al., 2024). The paper’s mechanistic claim is that attention-like multiplicative preprocessing and multiplicative output gating are the missing ingredients that make SSMs capable of transformer-like in-context optimization (Sushma et al., 2024). In that sense, attention augmentation provides algebraic primitives rather than merely longer-range routing.

SPADE exemplifies a different hybrid pattern: an SSM-augmented Transformer rather than an attention-augmented SSM. Its bottom “global layer” processes the same normalized input in parallel through a local-attention branch and an S4 branch,

$\mathbf{Y}$1

followed by concatenation, projection, residual addition, and FFN (Zuo et al., 2022). The architectural conclusion is that one SSM-equipped bottom layer can inject global context and positional bias, after which upper layers perform local attention-based refinement (Zuo et al., 2022). Although this is not an A2SSM in the narrow sense, it strongly informs the same design space by showing that structured global context can be supplied early and refined later.

Vision restoration provides yet another case study. TSANet introduces a “State space augmented cross-attention” design for HybridEVS demosaicing, where the Cross-Swin State Block splits features into a QCSA branch and an RVSS branch: $\mathbf{Y}$2 Here cross-attention handles local position-aware fusion while the state-space branch supplies global dependencies with linear complexity (Zhou et al., 8 Aug 2025). The two branches run in parallel rather than serially, and the paper explicitly describes the division of labor as local high-frequency textures versus global low-frequency structures (Zhou et al., 8 Aug 2025).

A more tentative variant appears in GFSSM, which is not a standard attention+SSM hybrid but rather an SSM augmented with attention-inspired mechanisms. It partitions the recurrence into groups and introduces grouped FIR filtering and an “attention sink” inspired prompt or cache mechanism: $\mathbf{Y}$3 with

$\mathbf{Y}$4

in the $\mathbf{Y}$5 exposition (Meng et al., 2024). The sink mechanism is implemented through learnable prompts and cached hidden states rather than explicit query-key-value attention (Meng et al., 2024). The paper is conceptually relevant to A2SSM but does not empirically validate the design (Meng et al., 2024).

6. Empirical evidence, strengths, and open questions

The most direct empirical evidence for A2SSM as a named mechanism comes from A2Mamba. In the roadmap ablation, a baseline LASS-style model with 13M parameters, 14.5G FLOPs, and 176 img/s achieves $\mathbf{Y}$6 ImageNet top-1 and $\mathbf{Y}$7 ADE20K mIoU (Lou et al., 22 Jul 2025). Replacing Natten with AMA yields $\mathbf{Y}$8 and $\mathbf{Y}$9, a gain of $\mathbf{A}_1$0 top-1 and $\mathbf{A}_1$1 mIoU (Lou et al., 22 Jul 2025). Replacing SS2D with vanilla SSM drops performance to $\mathbf{A}_1$2 and $\mathbf{A}_1$3, showing that naïve unidirectional SSM is harmful for vision (Lou et al., 22 Jul 2025). Replacing SSM with A2SSM produces 13M parameters, 13.3G FLOPs, 235 img/s, and $\mathbf{A}_1$4 top-1 and $\mathbf{A}_1$5 mIoU, a gain of $\mathbf{A}_1$6 top-1 and $\mathbf{A}_1$7 mIoU relative to vanilla SSM (Lou et al., 22 Jul 2025). Adding the final gate yields 15M parameters, 14.0G FLOPs, 220 img/s, and $\mathbf{A}_1$8 top-1 and $\mathbf{A}_1$9 mIoU (Lou et al., 22 Jul 2025). This is the cleanest evidence that the hidden-state augmentation mechanism, rather than generic scaling alone, accounts for a substantial fraction of the gains.

Further ablations compare A2SSM to other local mixers. Baseline A2Mamba-T with A2SSM reaches $\mathbf{A}_2$0 top-1 and $\mathbf{A}_2$1 mIoU, whereas replacing it with dilated RepConv gives $\mathbf{A}_2$2 and $\mathbf{A}_2$3, and replacing it with DCNv2 gives $\mathbf{A}_2$4 and $\mathbf{A}_2$5 (Lou et al., 22 Jul 2025). Adaptive dilation in AMA is also supported empirically: the adaptive-dilation baseline gives $\mathbf{A}_2$6 and $\mathbf{A}_2$7, while fixed dilations 3, 5, 7 and a four-branch $\mathbf{A}_2$8 alternative yield lower or equal results, especially on segmentation (Lou et al., 22 Jul 2025). In token-mixer comparison within the same encoder setting, MASS yields $\mathbf{A}_2$9 compared with LASS at $\mathbf{X}$00, improving by $\mathbf{X}$01 top-1 and $\mathbf{X}$02 mIoU (Lou et al., 22 Jul 2025).

At the model level, A2Mamba-S/B/L reach $\mathbf{X}$03, $\mathbf{X}$04, and $\mathbf{X}$05 top-1 on ImageNet-1K, respectively (Lou et al., 22 Jul 2025). On ADE20K with UperNet, A2Mamba-S/B/L achieve $\mathbf{X}$06, $\mathbf{X}$07, and $\mathbf{X}$08 mIoU for single-scale/multi-scale evaluation (Lou et al., 22 Jul 2025). On COCO Cascade Mask R-CNN, A2Mamba-S/B/L obtain $\mathbf{X}$09, $\mathbf{X}$10, and $\mathbf{X}$11 for AP$\mathbf{X}$12/AP$\mathbf{X}$13 (Lou et al., 22 Jul 2025). The paper also reports that A2Mamba-L reaches $\mathbf{X}$14 top-1 on ImageNet-1K, A2Mamba-B exceeds CAFormer-S36 by $\mathbf{X}$15 mIoU in semantic segmentation while exhibiting higher efficiency, and A2Mamba-S surpasses MambaVision-B by $\mathbf{X}$16 in AP$\mathbf{X}$17/AP$\mathbf{X}$18 with $\mathbf{X}$19 fewer parameters (Lou et al., 22 Jul 2025).

Sessa provides a different empirical picture, focused on long-context language modeling. Under matched architectures and training budgets, it achieves $\mathbf{X}$20 on SymbolSoup versus $\mathbf{X}$21 for Transformer and $\mathbf{X}$22 for Mamba2, and $\mathbf{X}$23 on Diffuse MQAR versus $\mathbf{X}$24 for Transformer and $\mathbf{X}$25 for Mamba2 (Horbatko, 20 Apr 2026). On SimpleStories short-context language modeling, however, Sessa is slightly worse, with perplexity $\mathbf{X}$26 versus $\mathbf{X}$27 for Transformer and $\mathbf{X}$28 for Mamba2; removing the feedback branch improves it to $\mathbf{X}$29 (Horbatko, 20 Apr 2026). The paper therefore treats attention-routed feedback as especially beneficial in long-context settings rather than universally advantageous (Horbatko, 20 Apr 2026).

Adversarial-robustness experiments contribute a different criterion. A comparative study of SSM variants under adversarial training reports that among the tested SSM families, adding attention in Mega gives the strongest robustness–generalization trade-off on CIFAR-10, but also aggravates robust overfitting (Qi et al., 2024). The paper’s interpretation is that attention acts as adaptive scaling of the SSM output sequence, aligning clean and adversarial features more effectively than fixed SSM dynamics alone (Qi et al., 2024). This does not establish a general theorem for all A2SSMs, but it does suggest that attention augmentation can improve trainability under difficult optimization criteria beyond standard clean accuracy (Qi et al., 2024).

The main strengths attributed to A2SSM-like designs across these papers are consistent. In A2Mamba, the benefits are better two-dimensional spatial dependency modeling, suppression of sequential causality bias, dynamic multi-scale spatial enhancement through shared attention maps, stronger dense prediction performance, competitive efficiency, and larger effective receptive field with good local sensitivity (Lou et al., 22 Jul 2025). In Sessa, the distinctive gain is many-path aggregation within a recurrent layer and the possibility of heavy-tail memory $\mathbf{X}$30 for $\mathbf{X}$31, with tighter $\mathbf{X}$32 under explicit uniform diffuse routing (Horbatko, 20 Apr 2026). In the in-context-learning construction, the benefit is access to multiplicative statistics and query-conditioned readout inside a recurrent architecture (Sushma et al., 2024).

Several limitations are equally clear. A2Mamba still depends on the quality of attention maps from AMA, uses a handcrafted two-branch multi-scale structure with stage-specific windows, and does not expose a fully detailed tensor equation for the cross-attention-like hidden-state aggregation in the visible method text (Lou et al., 22 Jul 2025). Sessa’s strongest theory holds under specific diffuse-routing and finite-horizon family regimes, and its dense implementation remains quadratic rather than linear-time in the default architecture (Horbatko, 20 Apr 2026). Exact state-space/attention dualities do not extend to standard softmax attention (Hu et al., 6 Oct 2025, Sieber et al., 2024). Factorized attention-style models may require many heads to emulate structured dynamical operator families (Ghodsi, 17 Dec 2025). Some attention-inspired SSM proposals, such as GFSSM, remain conceptually suggestive but empirically unvalidated (Meng et al., 2024).

A common misconception is that any architecture containing both attention and an SSM is automatically an A2SSM. The literature is more precise. Some models simply stack attention and SSM layers without deep interaction, which A2Mamba explicitly argues is insufficient (Lou et al., 22 Jul 2025). Some are SSM-augmented Transformers rather than attention-augmented SSMs, as in SPADE (Zuo et al., 2022). Some achieve exact equivalence only for semiseparable masked attention rather than standard attention (Hu et al., 6 Oct 2025). The most specific contemporary use of the term refers to attention-guided hidden-state enhancement inside the SSM path itself (Lou et al., 22 Jul 2025).

The overall trajectory suggests that A2SSM is best understood not as a single fixed architecture but as a design family centered on one principle: attention should alter the memory dynamics, hidden-state aggregation, or recurrent readout of an SSM in a task-relevant way, rather than merely coexist with it. In A2Mamba this is realized by multi-scale attention-guided hidden-state aggregation for vision (Lou et al., 22 Jul 2025). In Sessa it is attention-defined feedback routing (Horbatko, 20 Apr 2026). In GD-capable SSMs it is multiplicative local attention for constructing gradient statistics (Sushma et al., 2024). Theoretical work indicates why these efforts continue to matter: exact equivalence is limited, softmax attention is not reducible to finite-state semiseparable recurrence, and attention and state-space dynamics exhibit complementary trade-offs in operator expressivity, normalization structure, and long-range gradient propagation (Hu et al., 6 Oct 2025, Sieber et al., 2024, Ghodsi, 17 Dec 2025).