Spatial Structured Attention Block

• Spatial Structured Attention Block is a family of feature refinement modules that use explicit structural priors to constrain spatial interactions in neural networks.
• It employs diverse mechanisms—such as pooling, local self-attention, graph partitioning, autoregressive masks, and geometric representations—to impose structure on spatial features.
• The design enables computational efficiency and improved performance across applications like medical segmentation, weather forecasting, and neural operator modeling.

Spatial structured attention block denotes a family of feature-refinement modules in which spatial attention is not learned as an unconstrained position-wise mask, but is instead shaped by an explicit structural prior. Across the literature, the label has been used for several related operators: CNN spatial gates built from pooled descriptors, overlapping local self-attention blocks, graph-partitioned spatiotemporal modules, autoregressive mask generators, probabilistic spatial-channel factorizations, low-rank latent mixers, and geometrically parameterized masks. The common objective is to preserve or exploit spatial organization while imposing structure on where and how interactions are computed (Woo et al., 2018, Chen et al., 10 Sep 2025, Jiang et al., 2019, Khandelwal et al., 2019).

1. Formal basis and relation to standard spatial attention

A useful reference point is the spatial branch of CBAM. Given an intermediate feature map $F\in\mathbb R^{C\times H\times W}$, CBAM collapses the channel axis at each spatial location by average-pooling and max-pooling: $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$ These two maps are concatenated into

$$T=[F_{avg};F_{max}]\in\mathbb R^{2\times H\times W},$$

processed by a $7\times 7$ convolution with kernel size $7$, stride $1$, padding $3$, and mapped through a sigmoid to obtain

$$M_s(F)=\sigma\bigl(f^{7\times 7}([F_{avg};F_{max}])\bigr)\in\mathbb R^{1\times H\times W}.$$

The refined output is

$$F' = M_s(F)\otimes F,\qquad F'_i(x,y)=M_s(x,y)\cdot F_i(x,y).$$

Because stride $=1$ and padding $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$0, the $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$1 spatial dimensions remain unchanged and no down-sampling is introduced inside the block (Woo et al., 2018).

This formulation already contains the essential ingredients of many later spatial structured modules: a reduced spatial descriptor, a constrained operator over that descriptor, and multiplicative feature refinement. In CBAM the structure is mild, because the attention map is still a dense $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$2 mask, but its estimation is bottlenecked through pooled summaries and a single large-kernel convolution. The block is therefore a baseline for later methods that impose stronger structure over the attention variables.

A closely related instantiation appears in the Residual Spatial Attention Block used by RSAN for retinal vessel segmentation. There, the spatial gate is applied not to the original input but to the residual features after a pre-activation two-layer residual path with DropBlock. If $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$3 denotes the residual features, then

$$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$4

and

$$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$5

Broadcasting $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$6 over channels gives

$$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$7

followed by the residual output

$$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$8

This variant shows how a standard spatial gate can be embedded into a residual learning pipeline rather than used as a stand-alone multiplicative mask (Guo et al., 2020).

2. Recurrent structuring principles

The surveyed designs impose structure in several distinct ways.

Variant	Structuring principle	Representative paper
CBAM-style spatial gate	Channel-axis pooling + $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$9 convolution	(Woo et al., 2018)
Local block-wise SSAB	Overlapping $$T=[F_{avg};F_{max}]\in\mathbb R^{2\times H\times W},$$0 blocks with stacked layers	(Jiang et al., 2019)
Graph-partitioned SSA	Intra-subgraph attention + inter-subgraph attention	(Chen et al., 10 Sep 2025)
Autoregressive spatial attention	Bi-directional diagonal LSTMs over mask variables	(Khandelwal et al., 2019)
Variational structured attention	CP-rank spatial-channel factorization	(Yang et al., 2021)
Rectangular attention	Five-parameter soft rectangle	(Nguyen et al., 13 Mar 2025)
Low-rank spatial attention	Learned latent bottleneck of width $$T=[F_{avg};F_{max}]\in\mathbb R^{2\times H\times W},$$1	(Yang et al., 4 Apr 2026)
Tile-based sparse SSAB	Adaptive tiling, dual windows, frame-group masks	(Li et al., 18 Aug 2025)

These variants indicate that “structured” does not refer to a single mechanism. In some works, structure is a locality prior; in others, it is a graph partition, an autoregressive dependency, a low-rank factorization, or a geometric support constraint. A plausible implication is that the term is best understood functionally: a spatial structured attention block is any attention module whose spatial interaction pattern is deliberately constrained so that the resulting mask or affinity tensor reflects prior assumptions about regularity, hierarchy, sparsity, or geometry.

The contrast with unconstrained position-wise attention is explicit in several works. “Convolutional Rectangular Attention Module” states that standard position-wise spatial attention generates masks in a position-wise fashion and that this results in “very irregular boundaries,” whereas a rectangle parametrized by only five parameters yields a smoother connected region and a lower-dimensional hypothesis space (Nguyen et al., 13 Mar 2025). AttentionRNN similarly argues that prior attention frameworks lack the ability to explicitly model structural dependencies among attention variables, making it difficult to predict consistent attention masks (Khandelwal et al., 2019).

3. Locality, hierarchy, and graph partitioning

One major line of work structures attention by decomposing the spatial domain into blocks or subgraphs. In the local block-wise self-attention method for normal organ segmentation, the input feature tensor $$T=[F_{avg};F_{max}]\in\mathbb R^{2\times H\times W},$$2 is linearly projected into $$T=[F_{avg};F_{max}]\in\mathbb R^{2\times H\times W},$$3, $$T=[F_{avg};F_{max}]\in\mathbb R^{2\times H\times W},$$4, and $$T=[F_{avg};F_{max}]\in\mathbb R^{2\times H\times W},$$5 by $$T=[F_{avg};F_{max}]\in\mathbb R^{2\times H\times W},$$6 convolutions. The $$T=[F_{avg};F_{max}]\in\mathbb R^{2\times H\times W},$$7 grid is then covered by overlapping blocks of size $$T=[F_{avg};F_{max}]\in\mathbb R^{2\times H\times W},$$8 and stride $$T=[F_{avg};F_{max}]\in\mathbb R^{2\times H\times W},$$9: $7\times 7$0 Within each block, attention is computed as

$7\times 7$1

Because blocks overlap, each pixel belongs to multiple blocks, and the final feature at $7\times 7$2 is the average of the corresponding block outputs covering that location. Stacking $7\times 7$3 such layers enlarges the receptive field; the reported effective receptive field is

$7\times 7$4

This design exchanges information through overlap and depth rather than through full-image attention (Jiang et al., 2019).

A graph-structured counterpart appears in global station weather forecasting. Let $7\times 7$5 be a spatial graph of $7\times 7$6 stations with adjacency matrix $7\times 7$7, where $7\times 7$8 if stations $7\times 7$9 are within $7$0 km. A fast graph-partitioning procedure such as METIS splits $7$1 into $7$2 non-overlapping subgraphs, each with at most $7$3 nodes. Node embeddings $7$4 are reshaped into $7$5 with padding and masks. For each subgraph $7$6,

$7$7

and intra-subgraph attention is computed with an added shortest-path-distance bias matrix: $7$8 Subgraph summaries

$7$9

are stacked into $1$0, passed through inter-subgraph attention, and then expanded back to per-node shape before a linear fusion: $1$1 The same work stacks $1$2 such blocks at progressively coarser partitions, with default $1$3 and $1$4, so that subgraphs grow larger at higher layers (Chen et al., 10 Sep 2025).

These two designs share a common logic. Dense attention is preserved inside a constrained local unit—an image block or graph partition—while longer-range exchange is recovered through overlap, pooling, or multiscale repartitioning. This suggests that structure is often introduced not to eliminate global context, but to stage it hierarchically.

4. Autoregressive, probabilistic, and geometric formulations

A second line of work structures spatial attention by constraining the attention variables themselves. AttentionRNN models the attention mask $1$5 autoregressively: $1$6 To implement the two-dimensional dependency efficiently, it uses two diagonal LSTMs: one scans from top-left to bottom-right and the other from top-right to bottom-left. At each location, hidden summaries from the two passes are mapped to partial Gaussians, then combined into a final Gaussian approximation for the attention value. In practice, the deterministic choice is

$1$7

The key idea is that each attention value depends not only on local image or contextual information, but also on previously predicted attention values, which explicitly enforces structure over the spatial mask (Khandelwal et al., 2019).

VISTA-Net goes further by jointly modeling spatial and channel attention within a variational CRF. For each ordered scale pair $1$8, the structured attention tensor is constrained to have CP-rank $1$9: $3$0 where each $3$1 is a binary spatial map and each $3$2 is a one-hot channel selector vector. The refined feature at receiver scale $3$3 is

$3$4

Mean-field updates for the Gaussian posterior over $3$5, Bernoulli posterior over spatial gates $3$6, categorical posterior over channel vectors $3$7, and Gaussian posterior over kernels $3$8 are implemented by differentiable tensor operations and unrolled for end-to-end training (Yang et al., 2021).

A geometrically constrained alternative is the rectangular attention module. Instead of assigning an unrestricted scalar weight $3$9 to every location, it predicts a rectangle parametrized by five parameters

$$M_s(F)=\sigma\bigl(f^{7\times 7}([F_{avg};F_{max}])\bigr)\in\mathbb R^{1\times H\times W}.$$0

corresponding to center, half-width, half-height, and rotation. Using a sigmoid $$M_s(F)=\sigma\bigl(f^{7\times 7}([F_{avg};F_{max}])\bigr)\in\mathbb R^{1\times H\times W}.$$1 and sharpness parameter $$M_s(F)=\sigma\bigl(f^{7\times 7}([F_{avg};F_{max}])\bigr)\in\mathbb R^{1\times H\times W}.$$2, the unrotated soft window is

$$M_s(F)=\sigma\bigl(f^{7\times 7}([F_{avg};F_{max}])\bigr)\in\mathbb R^{1\times H\times W}.$$3

and the rotated two-dimensional mask is obtained by inverse rotation followed by tensorized windows. The gated feature update is residual: $$M_s(F)=\sigma\bigl(f^{7\times 7}([F_{avg};F_{max}])\bigr)\in\mathbb R^{1\times H\times W}.$$4 Here, structure is imposed directly on the support geometry of the attention map rather than on its computation graph (Nguyen et al., 13 Mar 2025).

Taken together, these methods show that structured spatial attention can be sequential, probabilistic, or geometric. The shared principle is the replacement of a free $$M_s(F)=\sigma\bigl(f^{7\times 7}([F_{avg};F_{max}])\bigr)\in\mathbb R^{1\times H\times W}.$$5 weight field by a lower-entropy object: a causal mask process, a low-rank tensor, or a soft rectangle.

5. Scalability, sparsity, and hardware alignment

Structured attention is frequently motivated by complexity reduction. In the graph-partitioned weather model, the intra-subgraph cost is

$$M_s(F)=\sigma\bigl(f^{7\times 7}([F_{avg};F_{max}])\bigr)\in\mathbb R^{1\times H\times W}.$$6

while inter-subgraph attention costs

$$M_s(F)=\sigma\bigl(f^{7\times 7}([F_{avg};F_{max}])\bigr)\in\mathbb R^{1\times H\times W}.$$7

The total is therefore

$$M_s(F)=\sigma\bigl(f^{7\times 7}([F_{avg};F_{max}])\bigr)\in\mathbb R^{1\times H\times W}.$$8

Choosing $$M_s(F)=\sigma\bigl(f^{7\times 7}([F_{avg};F_{max}])\bigr)\in\mathbb R^{1\times H\times W}.$$9 balances the terms and yields

$$F' = M_s(F)\otimes F,\qquad F'_i(x,y)=M_s(x,y)\cdot F_i(x,y).$$0

which is sub-quadratic in $$F' = M_s(F)\otimes F,\qquad F'_i(x,y)=M_s(x,y)\cdot F_i(x,y).$$1 (Chen et al., 10 Sep 2025).

For local block-wise self-attention, the reported total cost per SSAB layer is

$$F' = M_s(F)\otimes F,\qquad F'_i(x,y)=M_s(x,y)\cdot F_i(x,y).$$2

and for $$F' = M_s(F)\otimes F,\qquad F'_i(x,y)=M_s(x,y)\cdot F_i(x,y).$$3 stacked layers

$$F' = M_s(F)\otimes F,\qquad F'_i(x,y)=M_s(x,y)\cdot F_i(x,y).$$4

The same work contrasts this with global self-attention at $$F' = M_s(F)\otimes F,\qquad F'_i(x,y)=M_s(x,y)\cdot F_i(x,y).$$5 when $$F' = M_s(F)\otimes F,\qquad F'_i(x,y)=M_s(x,y)\cdot F_i(x,y).$$6, as well as criss-cross attention at $$F' = M_s(F)\otimes F,\qquad F'_i(x,y)=M_s(x,y)\cdot F_i(x,y).$$7 and point-wise spatial attention at $$F' = M_s(F)\otimes F,\qquad F'_i(x,y)=M_s(x,y)\cdot F_i(x,y).$$8 (Jiang et al., 2019).

Low-Rank Spatial Attention for neural operators adopts a different route. It compresses pointwise features $$F' = M_s(F)\otimes F,\qquad F'_i(x,y)=M_s(x,y)\cdot F_i(x,y).$$9 into $=1$0 latent tokens by cross-attention with learnable queries $=1$1, performs self-attention in latent space, and reconstructs spatial updates by a second cross-attention. Its total cost is

$=1$2

compared with $=1$3 for full softmax attention. Because $=1$4, the method scales nearly linearly in $=1$5, uses only standard Transformer primitives, and is directly compatible with FlashAttention-2, xFormers, and Triton kernels (Yang et al., 4 Apr 2026).

In video generation, Compact Attention replaces full attention over $=1$6 tokens by tile-based sparse attention. Tokens are partitioned into spacetime tiles of size $=1$7, and a binary tile-pair mask $=1$8 specifies active interactions. The method combines adaptive tiling, dual windows that approximate local, cross-shaped, and global patterns, and frame-group-wise sparsity that varies with temporal distance. Its implementation is explicitly hardware-aware: block-wise FlashAttention 2 kernels, bit-packed masks, contiguous $=1$9 layouts, warp-level parallelism across tile pairs, and single-pass dual-window fusion are all reported design choices (Li et al., 18 Aug 2025).

These complexity results indicate that “structured” often means “computationally selective.” The selection mechanism, however, varies substantially: partitions, overlaps, latent ranks, tile masks, or progressive coarse-to-fine scales.

6. Domain-specific instantiations and reported results

The empirical record for spatial structured attention is broad, spanning image classification, detection, medical segmentation, hyperspectral unmixing, weather forecasting, video generation, and scientific machine learning.

Domain	Reported outcome	Paper
Global station weather forecasting	Up to 16.8% improvement over time series baselines	(Chen et al., 10 Sep 2025)
Head and neck organ segmentation	Highest Dice up to $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$00 with +66.7% time and +0.15% parameters	(Jiang et al., 2019)
Retinal vessel segmentation	RSAN improves sensitivity, F1, ACC, and AUC on DRIVE and CHASE DB1	(Guo et al., 2020)
Medical image segmentation decoder	Synapse DSC from 81.35% to 83.92% after ACFA, TFFA, and SMMM	(Zhang et al., 5 Dec 2025)
Hyperspectral unmixing	Jasper Ridge avg SAD from $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$01 to $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$02	(Qi et al., 2023)
Neural operators	Average relative $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$03 error reduction of over 17% vs. second-best methods	(Yang et al., 4 Apr 2026)
Video generation	1.65$$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$04 to 2.51$$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$05 attention speedup with comparable visual quality	(Li et al., 18 Aug 2025)

In retinal vessel segmentation, the contribution of the Residual Spatial Attention Block was quantified by ablations on DRIVE and CHASE DB1. On DRIVE, the backbone achieved Sensitivity $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$06, F1 $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$07, ACC $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$08, and AUC $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$09; adding DropBlock raised these to SEN $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$10, F1 $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$11, ACC $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$12, and AUC $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$13; adding spatial attention to obtain RSAN yielded SEN $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$14, F1 $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$15, ACC $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$16, and AUC $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$17. On CHASE DB1, the progression was from SEN $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$18, F1 $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$19, ACC $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$20, AUC $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$21 to SEN $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$22, F1 $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$23, ACC $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$24, AUC $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$25, and finally SEN $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$26, F1 $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$27, ACC $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$28, AUC $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$29 (Guo et al., 2020).

In normal organ segmentation, the local block-wise design was evaluated on 48 internal headneck CT scans for training and 48 external PDDCA scans for testing. The reported Dice scores for the best configuration were $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$30 and $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$31 for left and right parotid glands, $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$32 and $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$33 for left and right submandibular glands, $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$34 for mandible, and $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$35 for brain stem. The same work reports a +66.7% computing-time increase per image and a 0.15% increase in model parameters compared with standard U-Net, whereas point-wise spatial attention achieved comparable accuracy but with a 516.7% increase in computing time and an 8.14% increase in parameters (Jiang et al., 2019).

In the decoder-oriented medical segmentation framework built from ACFA, TFFA, and SMMM, the SSAB is not a single self-attention matrix but a composite fusion block. ACFA combines channel and spatial gates together with learnable planar, horizontal, and vertical guidance tensors; TFFA fuses spatial, Fourier, Difference-of-Gaussians, and Mexican Hat branches; SMMM merges decoder and skip features using multi-scale depthwise convolutions, structural saliency masks, and a dilated-convolution refinement. On Synapse, the baseline decoder reached 81.35% DSC, +ACFA reached 82.04%, +ACFA+TFFA reached 83.23%, and +ACFA+TFFA+SMMM reached 83.92%. On ISIC ’18, the TFFA internal ablation reports 90.48% DSC for only Fourier, 90.57% for +MexicanHat, and 90.71% for +DoG+MH (Zhang et al., 5 Dec 2025).

Hyperspectral unmixing offers another interpretation of structured spatial attention. SAWU-Net first applies pixel attention,

$$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$36

then maps the center pixel to a full $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$37 window-attention matrix

$$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$38

which is used to re-weight coarse abundances within a local patch. On Jasper Ridge, the baseline autoencoder obtained avg SAD $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$39, SSAE obtained $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$40, SAWU-Net without Pixel Attention obtained $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$41, and full SAWU-Net obtained $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$42 (Qi et al., 2023).

The larger-scale computational variants also report substantial empirical gains. Compact Attention reaches 1.65$$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$43 speedup on Wan2.1 at up to 33.99% sparsity, with end-to-end latency 663.8 s versus 1092.2 s and SSIM $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$44, PSNR $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$45; on Hunyuan it reaches 2.51$$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$46 speedup at up to 62.36% sparsity, with 546.5 s versus 1370.7 s while preserving PSNR $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$47 and SSIM $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$48 (Li et al., 18 Aug 2025). LRSA reports an average relative $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$49 error reduction of over 17% relative to second-best methods across six PDE benchmarks, remains within 1–2% of FP32 accuracy in both FP16 and BF16, and on ShapeNet-Car in FP16 is 3.2$$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$50 faster per step and uses 2.1$$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$51 less peak GPU memory versus Transolver in FP32 (Yang et al., 4 Apr 2026).

7. Interpretation, misconceptions, and recurrent design tensions

A common misconception is that spatial structured attention is synonymous with a standard position-wise spatial mask. The surveyed literature shows the opposite. Several methods introduce structure precisely because position-wise attention may be noisy, irregular, or too expensive. The rectangular attention paper states that position-wise maps often have very irregular boundaries and that jagged masks can overfit to texture-level idiosyncrasies; its five-parameter rectangle is intended to reduce the hypothesis space and improve generalization (Nguyen et al., 13 Mar 2025).

Another misconception is that imposing structure necessarily sacrifices global context. The evidence is mixed. Graph-partitioned SSA combines local dense attention within subgraphs with global attention among pooled subgraph summaries, and the multiscale version progressively expands subgraph scales (Chen et al., 10 Sep 2025). LRSA preserves global coupling through low-rank latent interactions rather than dense all-to-all attention (Yang et al., 4 Apr 2026). Compact Attention explicitly models local, cross-shaped, and global patterns within a sparse tile framework (Li et al., 18 Aug 2025). This suggests that structured attention often seeks parsimonious global interaction rather than purely local processing.

A further design tension concerns how much structure is beneficial. In local block-wise self-attention, increasing block size from 24 to 36 to 48 improves Dice only marginally, by $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$52, but doubles compute if stride is held fixed; stacking from $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$53 to $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$54 yields about +0.02 average Dice, whereas $$F_{avg}(x,y)=\frac{1}{C}\sum_{i=1}^C F_i(x,y),\qquad F_{max}(x,y)=\max_{i=1,\ldots,C}F_i(x,y).$$55 has diminishing returns and adds linear cost (Jiang et al., 2019). In VISTA-Net, separate channel-only and spatial-only attention underperforms the joint structured approach by about 3 mIoU, and replacing variational inference by direct deterministic gates degrades performance by about 1.9 mIoU (Yang et al., 2021). In LRSA, removing latent self-attention raises error by up to 20% on time-dependent tasks, while tying compression and reconstruction also degrades performance (Yang et al., 4 Apr 2026).

The overall trajectory of the field therefore points toward a broad principle rather than a single canonical block. Spatial structure can be injected through pooling, overlap, graph topology, autoregressive order, variational factorization, geometric support, sparsity patterns, or latent compression. What remains stable across these formulations is the attempt to regularize where attention is placed and which spatial interactions are permitted, so that feature refinement becomes more coherent, more scalable, or more compatible with the inductive biases of the target domain.