Spatially Modulated Attention (SMA)

• SMA is a design pattern that injects spatial biases into attention mechanisms to refine feature selection in tasks ranging from object detection to crowd counting.
• It is implemented in various forms, including regression-aware co-attention in DETR, spatial decay masks in transformer encoders, and multiplicative gating in CNNs.
• While SMA enhances convergence and context adaptation, its performance depends on careful calibration of spatial priors and modulation parameters to avoid suppressing useful global context.

Searching arXiv for the cited SMA/SMCA papers to ground the article and citations. to=arxiv_search.search 北京赛车怎么json qq天天中彩票 {"^^^^^^^^11query11 OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^^^^^", "^^^^^^^^11max_results11 ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11^^^^, "^^^^^^^^11sort_by11 "^^^^^^^^11relevance11 to=arxiv_search.search Refining the search by title to ensure the exact papers are identified. to=arxiv_search.search 在天天中彩票 天天中彩票粤json {"^^^^^^^^11query11 Convergence of DETR with Spatially Modulated Co-Attention^^^^^^^^11\11 OR ^^^^^^^^11\11 Dual-Modulation Framework for RGB-T Crowd Counting via Spatially Modulated Attention and Adaptive Fusion^^^^^^^^11\11 OR ^^^^^^^^11\11 Neural Network Model of Spatial and Feature-Based Attention^^^^^^^^11\11 OR ^^^^^^^^11\11 Top-Down Attention Module for Contextually Guided Feature Selection in CNNs^^^^^^^^11\11 OR ^^^^^^^^11\11 Modulation of Visual Spatial Integration: Psychophysical Evidence Supported by Population Coding Modeling^^^^^^^^11\11 "^^^^^^^^11max_results11 ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11^^^^, "^^^^^^^^11sort_by11 "^^^^^^^^11relevance11 to=arxiv_search.search Spatially Modulated Attention (SMA) denotes a family of mechanisms that inject explicit spatial structure into attention or attention-like modulation. In the cited literature, the term does not refer to a single standardized operator. In object detection, it appears as Spatially Modulated Co-Attention (SMCA), where decoder cross-attention logits are biased by a ^^^^^^^^11query11 spatial prior (&&&^^^^11query11^^^^&&&). In RGB-T crowd counting, it is a self-attention mechanism with a learnable Spatial Decay Mask that penalizes distant token interactions (&&&^^^^11max_results11^^^^&&&). In cognition-inspired neural models and top-down CNN modules, it is realized as multiplicative gating over spatial locations and feature channels rather than as ^^^^^^^^11query11 attention (&&&^^^^11sort_by11^^^^&&&); (&&&^^^^11relevance11^^^^&&&). In vision science, SMA refers to attention-dependent reweighting of spatial integration weights over neural populations (&&&^^^^11query11^^^^&&&). A plausible implication is that SMA is best understood as a design pattern—explicitly spatializing selection—rather than as a single architecture.

^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^. Terminology and conceptual scope

The broadest commonality across SMA formulations is the introduction of a spatial inductive bias into an otherwise weakly constrained selection mechanism. What varies is where the bias enters: decoder co-attention, modality-specific self-attention, multiplicative gates over CNN feature maps, or post-encoding integration weights in a population code. The DETR literature explicitly distinguishes “Spatially Modulated Attention” as a broad idea from “Spatially Modulated Co-Attention,” emphasizing that the modified operator is decoder cross-attention rather than self-attention (&&&^^^^11query11^^^^&&&).

Formulation	Core mechanism	Domain
SMCA (&&&^^^^11query11^^^^&&&)	Add PRESERVED_PLACEHOLDER_^^^^11query11^^^^ to decoder co-attention logits	DETR object detection
SMA (&&&^^^^11max_results11^^^^&&&)	Learnable Spatial Decay Mask over token distances	RGB-T crowd counting
SMA (&&&^^^^11sort_by11^^^^&&&)	Contextual gate PRESERVED_PLACEHOLDER_^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^	Multi-digit visual attention model
TDAM (&&&^^^^11relevance11^^^^&&&)	Top-down “searchlight” for channel and spatial modulation	CNN feature selection
SMA (&&&^^^^11query11^^^^&&&)	Attention-dependent tuning of spatial integration weights	Psychophysics and population coding
SMA (&&&^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11 OR \11^^^^&&&)	Sparse Modular Activation controlling where attention runs	Sequence modeling

This terminological spread matters because “spatial modulation” can mean very different operations. Some SMA variants preserve global attention and add a parametric spatial prior; others suppress long-range interactions; others do not use self-attention or cross-attention at all. The acronym is therefore semantically overloaded, and any technical discussion must specify the mechanism, the locus of modulation, and the learning signal.

^^^^11max_results11^^^^. Regression-aware co-attention in DETR

In DETR, the motivating problem is slow convergence. Vanilla DETR requires about ^^^^11query11query11query11^^^^ epochs from scratch because decoder co-attention between object queries and encoded image features is agnostic to the boxes it will later regress. Each ^^^^^^^^11query11^^^^^^^^ must discover where to attend through training alone, so ^^^^^^^^11query11 assignment emerges inefficiently. SMCA addresses this by making decoder co-attention regression-aware: each ^^^^^^^^11query11^^^^^^^^ predicts an initial box center and anisotropic scales, and the resulting Gaussian-like spatial prior constrains attention responses to be high near the estimated box (&&&^^^^11query11^^^^&&&).

The baseline decoder cross-attention is

PRESERVED_PLACEHOLDER_^^^^11max_results11^^^^

SMCA augments this with a spatial prior derived from the ^^^^^^^^11query11

PRESERVED_PLACEHOLDER_^^^^11sort_by11^^^^

followed by

PRESERVED_PLACEHOLDER_^^^^11relevance11^^^^

and logit modulation

PRESERVED_PLACEHOLDER_^^^^11query11^^^^

The prior therefore acts as a regression-aware logit prior. The parameter PRESERVED_PLACEHOLDER_^^^^11\11^^^^ controls bandwidth and can be scheduled or tuned so that the prior is wide early in training.

The coupling between attention and regression is strengthened further at box prediction time. After spatially modulated co-attention, decoder features produce class and box outputs, and the pre-sigmoid prior center is added to the pre-sigmoid box center outputs. This explicitly aligns final box prediction with the center that shaped the attention prior. The paper also generalizes the shared prior to head-specific priors by predicting head-specific offsets and scales relative to a shared ^^^^^^^^11query11^^^^^^^^ center. Different heads can thus focus on complementary object regions.

The full system adds multi-scale encoding and scale-selection attention. Backbone features PRESERVED_PLACEHOLDER_^^^^11 OR \11^^^^ are processed by an encoder with ^^^^11max_results11^^^^ intra-scale self-attention blocks, ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^ multi-scale self-attention block, and ^^^^11max_results11^^^^ more intra-scale blocks (“^^^^11max_results11^^^^Intra–Multi–^^^^11max_results11 with weight sharing across scales. In the decoder, each ^^^^^^^^11query11^^^^^^^^ predicts

PRESERVED_PLACEHOLDER_^^^^11 OR \11^^^^

and the final per-head co-attention output is an PRESERVED_PLACEHOLDER_^^^^11 OR \11^^^^-weighted sum across scales. SMCA changes only the decoder’s co-attention; residual connections, post-norm, Hungarian bipartite matching, and DETR’s end-to-end set prediction are retained.

Training also stays close to DETR, with focal loss replacing cross-entropy for classification, L^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^ loss and GIoU loss for boxes, and coefficients PRESERVED_PLACEHOLDER_^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11^^^^, PRESERVED_PLACEHOLDER_^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^, and PRESERVED_PLACEHOLDER_^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11max_results11^^^^. Typical schedules are ^^^^11query11query11^^^^ or ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11 OR \11^^^^ epochs, with a learning-rate drop at epoch ^^^^11relevance11query11^^^^ by a factor of ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11^^^^, AdamW optimization, learning rate PRESERVED_PLACEHOLDER_^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11sort_by11^^^^ for the Transformer encoder-decoder and PRESERVED_PLACEHOLDER_^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11relevance11^^^^ for the backbone, and ^^^^11sort_by11query11query11^^^^ queries. On COCO val, DETR-DC^^^^11query11^^^^ with ResNet-^^^^11query11query11^^^^ achieves ^^^^11relevance11sort_by11^^^^.^^^^11sort_by11^^^^ AP at ^^^^11query11query11query11^^^^ epochs, whereas full multi-scale SMCA reaches ^^^^11relevance11sort_by11^^^^.^^^^11 OR \11^^^^ AP at ^^^^11query11query11^^^^ epochs and ^^^^11relevance11query11^^^^.^^^^11\11^^^^ AP at ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11 OR \11^^^^ epochs; the ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11 OR \11^^^^-epoch AP by object size is PRESERVED_PLACEHOLDER_^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11^^^^ for small, medium, and large objects. Ablations show a gain from ^^^^11sort_by11relevance11^^^^.^^^^11 OR \11^^^^ AP to ^^^^11relevance11query11^^^^.^^^^11max_results11^^^^ AP for head-shared spatial modulation at ^^^^11query11query11^^^^ epochs, a further increase to ^^^^11relevance11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11query11^^^^ AP with multi-head spatial modulation, and ^^^^11relevance11sort_by11^^^^.^^^^11 OR \11^^^^ AP with the ^^^^11max_results11^^^^Intra–Multi–^^^^11max_results11 encoder. The computational overhead is modest in single-scale form, increasing inference time from ^^^^11query11^^^^.^^^^11query11sort_by11 OR \11^^^^ s to ^^^^11query11^^^^.^^^^11query11relevance11sort_by11^^^^ s, while the full multi-scale model runs at ^^^^11query11^^^^.^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11query11^^^^ s versus ^^^^11query11^^^^.^^^^11query11 OR \11 OR \11^^^^ s for DETR-DC^^^^11query11^^^^ and uses ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11max_results11^^^^ GFLOPs versus ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11 OR \11 OR \11^^^^ GFLOPs for DETR-DC^^^^11query11^^^^ (&&&^^^^11query11^^^^&&&).

A recurring limitation is sensitivity to inaccurate early priors. The paper mitigates this with a wide PRESERVED_PLACEHOLDER_^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11\11^^^^ early in training and with multi-head priors that diversify focus. Small-object performance can remain weaker than methods centered on local sampling, such as Deformable DETR, whereas medium and large objects benefit from preserved global attention.

^^^^11sort_by11^^^^. Spatial decay masks in RGB-T crowd counting

In RGB-T crowd counting, SMA addresses a different failure mode: Transformer self-attention is permutation-invariant and lacks ^^^^11max_results11^^^^D spatial inductive bias, so attention can spread into irrelevant background regions. The proposed SMA sharpens localization by penalizing long-range token interactions according to their Euclidean distance on the feature grid. Unlike SMCA in DETR, this formulation is applied inside modality-specific self-attention encoders, and there is no cross-attention (&&&^^^^11max_results11^^^^&&&).

A shared VGG-^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11 OR \11^^^^ backbone extracts parallel feature maps for RGB and thermal inputs, denoted PRESERVED_PLACEHOLDER_^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11 OR \11^^^^ and PRESERVED_PLACEHOLDER_^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11 OR \11^^^^. Each modality then passes through a separate transformer encoder. The convolutional feature maps are flattened into PRESERVED_PLACEHOLDER_^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11 OR \11^^^^ tokens arranged on a ^^^^11max_results11^^^^D grid, and SMA uses token positions to compute a pairwise Euclidean distance matrix PRESERVED_PLACEHOLDER_^^^^11max_results11query11^^^^ with

PRESERVED_PLACEHOLDER_^^^^11max_results11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^

For each attention head, two learnable parameters are introduced, PRESERVED_PLACEHOLDER_^^^^11max_results11max_results11^^^^ and PRESERVED_PLACEHOLDER_^^^^11max_results11sort_by11^^^^, transformed as

PRESERVED_PLACEHOLDER_^^^^11max_results11relevance11^^^^

then

PRESERVED_PLACEHOLDER_^^^^11max_results11query11^^^^

The mask decreases with distance beyond the learned bias threshold, so nearby interactions are preserved while distant ones are penalized. The paper presents the modified attention as

PRESERVED_PLACEHOLDER_^^^^11max_results11\11^^^^

and its algorithmic description states that the effective operation is modulation of attention logits by the per-head mask before the softmax.

This per-head parameterization creates an implicit multi-scale encoder: some heads learn rapid decay and local focus, while others learn gentler decay and retain broader context. The resulting SMA-enhanced features are then fused by Adaptive Fusion Modulation (AFM), where a scene-aware gate weighs RGB and thermal contributions. Density regression is supervised by Bayesian Loss rather than auxiliary regularizers on the decay mask.

The implementation uses a two-layer transformer encoder per modality, ^^^^11 OR \11^^^^ attention heads, ^^^^11relevance11query11query11^^^^ training epochs, batch size ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^, Adam with learning rate PRESERVED_PLACEHOLDER_^^^^11max_results11 OR \11^^^^ and weight decay PRESERVED_PLACEHOLDER_^^^^11max_results11 OR \11^^^^, random horizontal flips, and random PRESERVED_PLACEHOLDER_^^^^11max_results11 OR \11^^^^ crops. On RGBT-CC, the full framework achieves GAME scores of ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11^^^^.^^^^11 OR \11query11^^^^, ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11relevance11^^^^.^^^^11max_results11query11^^^^, ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11 OR \11^^^^.^^^^11 OR \11relevance11^^^^, and ^^^^11max_results11sort_by11^^^^.^^^^11 OR \11max_results11^^^^ for PRESERVED_PLACEHOLDER_^^^^11sort_by11query11^^^^ to PRESERVED_PLACEHOLDER_^^^^11sort_by11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^, with RMSE ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11 OR \11^^^^.^^^^11 OR \11 OR \11^^^^. On DroneRGBT, it attains GAME(^^^^11query11^^^^) = ^^^^11\11^^^^.^^^^11 OR \11 OR \11^^^^ and RMSE = ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11sort_by11query11^^^^. The SMA ablation on RGBT-CC reduces GAME(^^^^11query11^^^^) from ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11max_results11^^^^.^^^^11query11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^ to ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11query11 OR \11^^^^ and RMSE from ^^^^11max_results11\11^^^^.^^^^11\11 OR \11^^^^ to ^^^^11max_results11query11^^^^.^^^^11query11sort_by11 and adding AFM yields ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11^^^^.^^^^11 OR \11query11^^^^ and ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11 OR \11^^^^.^^^^11 OR \11 OR \11^^^^. Fixed decay settings such as PRESERVED_PLACEHOLDER_^^^^11sort_by11max_results11^^^^ give GAME(^^^^11query11^^^^) = ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11relevance11max_results11^^^^ and RMSE = ^^^^11max_results11query11^^^^.^^^^11query11\11 whereas learnable per-head decay performs best. The reported overhead remains modest because SMA adds PRESERVED_PLACEHOLDER_^^^^11sort_by11sort_by11^^^^ distance computation and element-wise modulation, matching the asymptotic order of attention’s PRESERVED_PLACEHOLDER_^^^^11sort_by11relevance11^^^^ term (&&&^^^^11max_results11^^^^&&&).

The main limitations are also explicitly spatial: severe RGB-thermal misalignment can make distance-based local focus emphasize mismatched regions, isotropic decay may be insufficient for unusual scene layouts, and overly aggressive decay can suppress useful global context.

^^^^11relevance11^^^^. Context-driven multiplicative gating in cognition-inspired models

A distinct use of SMA appears in a neural network model of spatial and feature-based attention inspired by human cognition. Here SMA is not self-attention or cross-attention. It is a context-driven, multiplicative gate over an intermediate CNN feature map, varying jointly over spatial position and feature channel. A contextual network transforms top-down cues into a gate

PRESERVED_PLACEHOLDER_^^^^11sort_by11query11^^^^

which modulates a frozen function network by

PRESERVED_PLACEHOLDER_^^^^11sort_by11\11^^^^

The gating is therefore spatially structured and channel-specific (&&&^^^^11sort_by11^^^^&&&).

The architecture contains two networks. The function network is a CNN pre-trained on MNIST single-digit classification, achieving ^^^^11 OR \11max_results11^^^^% test accuracy, and then frozen. The contextual network is trained to generate gates at the second convolutional layer of the function network, where the feature map has PRESERVED_PLACEHOLDER_^^^^11sort_by11 OR \11^^^^ channels. For spatial attention tasks, the contextual network receives only a one-hot spatial signal PRESERVED_PLACEHOLDER_^^^^11sort_by11 OR \11^^^^ and uses deconvolution layers to generate the gate:

PRESERVED_PLACEHOLDER_^^^^11sort_by11 OR \11^^^^

For feature-based attention tasks, it receives both the image and the top-down cue:

PRESERVED_PLACEHOLDER_^^^^11relevance11query11^^^^

The paper also defines diagnostic summaries: a channel-averaged spatial attention map

PRESERVED_PLACEHOLDER_^^^^11relevance11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^

and a spatially averaged feature attention profile

PRESERVED_PLACEHOLDER_^^^^11relevance11max_results11^^^^

Three tasks are used. In two-digit spatial attention, the cue specifies left or right; in three-digit spatial attention, left, middle, or right; in feature-based attention, the cue specifies Group ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^ (digits ^^^^11query11^^^^–^^^^11relevance11 or Group ^^^^11max_results11^^^^ (digits ^^^^11query11^^^^–^^^^11 OR \11^^^^). The contextual network is trained by cross-entropy through the fixed function network. The reported datasets are ^^^^11 OR \11query11^^^^,^^^^11query11query11query11^^^^ train and ^^^^11max_results11query11^^^^,^^^^11query11query11query11^^^^ test for the two-digit and three-digit spatial tasks, each trained for ^^^^11query11^^^^ epochs, and ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11\11query11^^^^,^^^^11query11query11query11^^^^ train and ^^^^11relevance11query11^^^^,^^^^11query11query11query11^^^^ test for the feature-based task, trained for ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11^^^^ epochs.

The quantitative effect of spatial modulation is large. For two-digit spatial attention, the baseline frozen function network scores ^^^^11sort_by11 OR \11^^^^.^^^^11\11 OR \11^^^^% ± ^^^^11sort_by11^^^^.^^^^11\11query11 while FN+CN reaches ^^^^11 OR \11sort_by11^^^^.^^^^11\11max_results11^^^^% ± ^^^^11 OR \11^^^^.^^^^11 OR \11max_results11^^^^E-^^^^11query11relevance11^^^^. For three digits, the scores are ^^^^11max_results11relevance11^^^^.^^^^11 OR \11 OR \11^^^^% ± ^^^^11 OR \11^^^^.^^^^11query11max_results11^^^^E- versus ^^^^11 OR \11 OR \11^^^^.^^^^11relevance11relevance11^^^^% ± ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11query11max_results11^^^^E- For feature-based attention, the scores are ^^^^11sort_by11relevance11^^^^.^^^^11 OR \11\11^^^^% ± ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11query11sort_by11^^^^E- versus ^^^^11 OR \11query11^^^^.^^^^11query11max_results11^^^^% ± ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11query11query11^^^^E- Visualizations show that spatial tasks produce coherent left/right or left/middle/right gate structures across most channels, while feature-based tasks produce more complex channel-specific patterns that still suppress non-target content. The limitations are equally explicit: the setting is confined to MNIST digits, only one layer is gated, and no explicit sparsity or entropy regularization is imposed on PRESERVED_PLACEHOLDER_^^^^11relevance11sort_by11^^^^ (&&&^^^^11sort_by11^^^^&&&).

^^^^11query11^^^^. Top-down searchlight modulation in CNN blocks

TDAM presents another non-Transformer formulation of spatially modulated attention. It inserts iterative top-down feedback inside a convolutional block and computes a “visual searchlight” that first performs channel selection and then induces a spatial attention map. The central claim is that bottom-up attention modules such as SE, CBAM, ECA, and FCA are limited by the local statistics of a single feature map, whereas TDAM uses semantically richer top features to decide both what and where to amplify (&&&^^^^11relevance11^^^^&&&).

At computation step PRESERVED_PLACEHOLDER_^^^^11relevance11relevance11^^^^, a block maps bottom input PRESERVED_PLACEHOLDER_^^^^11relevance11query11^^^^ to top output PRESERVED_PLACEHOLDER_^^^^11relevance11\11^^^^. A searchlight

PRESERVED_PLACEHOLDER_^^^^11relevance11 OR \11^^^^

for joint attention, or

PRESERVED_PLACEHOLDER_^^^^11relevance11 OR \11^^^^

for top-only attention, is obtained by global pooling followed by small MLPs. The exact joint form is

PRESERVED_PLACEHOLDER_^^^^11relevance11 OR \11^^^^

with reduction ratio PRESERVED_PLACEHOLDER_^^^^11query11query11^^^^. Attention is then applied by

PRESERVED_PLACEHOLDER_^^^^11query11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^

where PRESERVED_PLACEHOLDER_^^^^11query11max_results11^^^^ denotes element-wise product and PRESERVED_PLACEHOLDER_^^^^11query11sort_by11^^^^ denotes pointwise convolution. In effect, the searchlight weights channels and also acts as a PRESERVED_PLACEHOLDER_^^^^11query11relevance11^^^^ filter that produces a spatial map on the lower-layer feature grid.

TDAM is inserted before residual addition in ResNet bottleneck blocks, works best with small feedback distance PRESERVED_PLACEHOLDER_^^^^11query11query11^^^^, and uses different batch-normalization layers for each step to stabilize training. The mechanism does not require top and bottom features to share spatial size because the top-down signal is injected through pooled channel vectors rather than spatial upsampling.

The reported ImageNet results quantify both overhead and gain. ResNet-^^^^11query11query11^^^^ baseline has ^^^^11max_results11query11^^^^.^^^^11query11\11 parameters, ^^^^11relevance11^^^^.^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11max_results11^^^^G FLOPs, and ^^^^11 OR \11 OR \11^^^^.^^^^11query11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^% Top-^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^ on ImageNet-V^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^. TDjoint with PRESERVED_PLACEHOLDER_^^^^11query11\11^^^^ has ^^^^11max_results11 OR \11^^^^.^^^^11\11query11^^^^M parameters, ^^^^11relevance11^^^^.^^^^11query11 OR \11^^^^G FLOPs, and ^^^^11 OR \11 OR \11^^^^.^^^^11 OR \11\11^^^^% Top-^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^ with ^^^^11 OR \11relevance11^^^^.^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11 OR \11^^^^% Top-^^^^11query11^^^^; on ImageNet-V^^^^11max_results11^^^^ it reaches ^^^^11\11 OR \11^^^^.^^^^11\11\11^^^^% Top-^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^. TDtop with PRESERVED_PLACEHOLDER_^^^^11query11 OR \11^^^^ yields ^^^^11 OR \11 OR \11^^^^.^^^^11 OR \11max_results11^^^^%, and TDtop with PRESERVED_PLACEHOLDER_^^^^11query11 OR \11^^^^ yields ^^^^11 OR \11 OR \11^^^^.^^^^11 OR \11query11^^^^%. For ResNet-^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^, the baseline is ^^^^11relevance11relevance11^^^^.^^^^11query11query11 parameters, ^^^^11 OR \11^^^^.^^^^11 OR \11query11^^^^G FLOPs, and ^^^^11 OR \11query11^^^^.^^^^11sort_by11\11^^^^% Top-^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^, while TDjoint with PRESERVED_PLACEHOLDER_^^^^11query11 OR \11^^^^ reaches ^^^^11relevance11\11^^^^.^^^^11 OR \11query11^^^^M, ^^^^11 OR \11^^^^.^^^^11sort_by11 OR \11^^^^G FLOPs, and ^^^^11 OR \11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11\11max_results11^^^^% Top-^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^. In weakly supervised localization, TDjoint gives ^^^^11\11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11query11query11^^^^% and TDtop with PRESERVED_PLACEHOLDER_^^^^11\11query11^^^^ gives ^^^^11\11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11 OR \11 OR \11^^^^%, exceeding CBAM at ^^^^11query11 OR \11^^^^.^^^^11 OR \11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^%. The paper also reports that at PRESERVED_PLACEHOLDER_^^^^11\11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^ resolution TDAM improves about ^^^^11max_results11^^^^% over baseline whereas CBAM degrades. The principal limitations are instability at large feedback distance and degradation when the number of steps becomes too large, with performance dropping beyond about PRESERVED_PLACEHOLDER_^^^^11\11max_results11^^^^–^^^^11sort_by11^^^^ (&&&^^^^11relevance11^^^^&&&).

^^^^11\11^^^^. Vision-science formulation, misconceptions, and adjacent usages

In psychophysics and population coding, SMA is formulated at a different level of analysis. Rather than modifying a neural-network attention layer, attention modulates how neural populations pool information across space. The formal statement is

PRESERVED_PLACEHOLDER_^^^^11\11sort_by11^^^^

where PRESERVED_PLACEHOLDER_^^^^11\11relevance11^^^^ is the population response after encoding and PRESERVED_PLACEHOLDER_^^^^11\11query11^^^^ is the attention-dependent spatial weighting. In the cited study, spatial attention produces strong, localized reductions in integration at the attended eccentricity, whereas feature-based attention produces modest, global reductions in integration across the field (&&&^^^^11query11^^^^&&&).

The experimental design uses gaze-contingent search to induce different attentional modes. Search detections cluster near the fovea in the baseline condition, with median ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11query11^^^^ ± ^^^^11query11^^^^.^^^^11 OR \11max_results11^^^^ deg, but shift outward under visual deprivation and information deprivation, with medians ^^^^11 OR \11^^^^.^^^^11max_results11 OR \11^^^^ ± ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11 OR \11 OR \11^^^^ deg and ^^^^11 OR \11^^^^.^^^^11 OR \11 OR \11^^^^ ± ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11query11query11^^^^ deg. In the subsequent ^^^^11max_results11^^^^-AFC task, isolated-target thresholds do not differ significantly across conditions, but crowded-target integration strength changes: visual deprivation yields a strong, localized reduction at ^^^^11 OR \11^^^^ deg with PRESERVED_PLACEHOLDER_^^^^11\11\11^^^^, and information deprivation yields a modest, global reduction across eccentricities with PRESERVED_PLACEHOLDER_^^^^11\11 OR \11^^^^. The corresponding population-code model implements Gaussian pooling kernels

PRESERVED_PLACEHOLDER_^^^^11\11 OR \11^^^^

with baseline slope ^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^.^^^^11sort_by11\11query11^^^^ and attention-dependent modulation factors PRESERVED_PLACEHOLDER_^^^^11\11 OR \11^^^^ and PRESERVED_PLACEHOLDER_^^^^11 OR \11query11^^^^. The study’s central mechanistic statement is that “attention acts beyond the neuronal encoding stage to tune the spatial integration weights of neural populations” (&&&^^^^11query11^^^^&&&).

Several common misconceptions can be resolved by comparing these literatures. First, SMA is not synonymous with self-attention: it can modify cross-attention in DETR, self-attention in RGB-T encoders, multiplicative CNN gates, or post-encoding integration weights. Second, spatial modulation does not necessarily imply hard locality. SMCA in DETR preserves global attention and biases logits with a Gaussian-like prior, whereas RGB-T SMA penalizes distant interactions continuously rather than via fixed windows. Third, the acronym “SMA” can denote something adjacent but distinct. In SeqBoat, SMA means Sparse Modular Activation, a differentiable mechanism that decides per sequence position whether the GAU attention sub-module should run, via a binary activation PRESERVED_PLACEHOLDER_^^^^11 OR \11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11^^^^ and confidence PRESERVED_PLACEHOLDER_^^^^11 OR \11max_results11^^^^. Activated tokens are compressed into a subsequence, processed by GAU, and then scattered back, yielding training complexity PRESERVED_PLACEHOLDER_^^^^11 OR \11sort_by11^^^^ and inference complexity PRESERVED_PLACEHOLDER_^^^^11 OR \11relevance11^^^^ under local attention on the compressed sequence (&&&^^^^11id:(Gao et al., 2021) OR id:(Gao et al., 2021) OR id:(Feng et al., 21 Sep 2025) OR id:(Hu et al., 5 Jun 2025) OR id:(Jaiswal et al., 2021) OR id:(Grillini et al., 2019)11 OR \11^^^^&&&).

A plausible implication of these differences is that “spatially modulated attention” names a recurring principle rather than a canonical block: useful attention mechanisms often become more data-efficient, better localized, or more task-aligned when they are explicitly constrained by spatial structure. The cited works instantiate that principle at markedly different levels—token geometry, object queries, convolutional feature maps, iterative top-down search, and neural population integration—and their limitations are correspondingly different.