Map-Level Attention Processing (MAP)

• MAP is a design principle that applies attention over explicit map representations—ranging from egocentric grids to semantic tensors—to integrate spatial and contextual cues.
• It employs varied computational patterns like self-attention over map cells, multi-layer fusion, and direct map processing to enhance navigation, perception, and decoding tasks.
• Empirical results show that MAP boosts sample efficiency, reduces processing overhead, and improves performance in tasks like embodied navigation, automated driving, and language decoding.

Map-Level Attention Processing (MAP) denotes a family of designs in which attention is applied to, guided by, or computed over an explicit map-like representation rather than being confined to transient per-frame or per-token features. Depending on the setting, the underlying map may be an egocentric semantic grid for embodied navigation, a situation-conditioned relevance map for perception scheduling, a quality or occupancy map for dense prediction, an attention-score tensor treated as a feature map, or a layer–token semantic map assembled during decoding. Across these variants, the common objective is to expose structured global context—geometry, semantics, relevance, or distributed factual evidence—to downstream inference, control, or decoding in a form that is more persistent, interpretable, or computationally actionable than standard token-local attention alone (Seymour et al., 2021, Henning et al., 2021, Li et al., 3 Aug 2025).

1. Terminological scope and lineage

The literature uses MAP both explicitly and implicitly. In embodied navigation, MaAST defines map-level attention as computing attention over a structured, egocentric, top-down spatial memory rather than directly over raw image frames (Seymour et al., 2021). In automated driving, a Multi-Layer Attention Map (MLAM) implements a map-level controller that derives relevant regions from lanes, ego path, and nearby objects, then uses that map to configure perception modules and gate intra-module processing (Henning et al., 2021). In LVLM decoding, MAP treats hidden states across layers and tokens as a two-dimensional semantic map and applies structured attention over that map to aggregate factual cues (Li et al., 3 Aug 2025).

Selected instantiations illustrate the breadth of the term.

Setting	Map object	MAP role
Embodied navigation	Egocentric semantic top-down map	Policy reads persistent spatial memory
Automated driving perception	Multi-Layer Attention Map	Module selection and ROI gating
Transformer and LVLM decoding	Attention-score map or layer–token semantic map	Logit refinement or factual aggregation

Several other papers do not name their method MAP, but they operate on the same principle. Explicitly modeled spatial attention maps replace learned $Q/K$ interactions with geometric kernels in image classification (Tan et al., 2020). Multi-scale attention maps are retained and merged across decoder stages in remote-sensing segmentation (Yang et al., 2024). Learnable masks regulate the full attention matrix in multimodal Transformers (Barrios et al., 2024). Attention maps are also supervised directly by human edits in Attention Branch Networks, reused across Transformer layers for efficiency, or processed as feature maps for length extrapolation (Mitsuhara et al., 2019, Shim et al., 2023, Zheng et al., 2024). This suggests that MAP is best understood as a design principle: attention becomes an operation over an explicit map substrate, not merely a hidden similarity kernel.

2. Representational substrates

A central distinction among MAP systems lies in what counts as the map. In MaAST, the map is a local egocentric crop $M \in \mathbb{R}^{H\times W\times C}$ centered on the agent and rotated so the agent’s heading points up. Its channels encode object semantics as multi-hot cell labels, traversable space, and unexplored areas; the map is built from RGB, depth, semantic labels, camera intrinsics, and pose, yielding a persistent memory that fuses geometry, exploration state, and semantics (Seymour et al., 2021). In MLAM, the map is a Cartesian grid of size $151\times151$ at $1\ \text{m/cell}$, with a polar counterpart at $1^\circ$ resolution; each cell aggregates unit-valued layer activations from lanes, ego path, and nearby objects, and the fused map is interpreted as a spatial field of performance requirements (Henning et al., 2021).

Other MAP variants broaden the notion of map beyond explicit navigation grids. In QA-Net for multi-image super-resolution, binary quality maps in $\{0,1\}$ identify clear versus disturbed regions and are transformed by Quality Map Encoding Modules into learned quality-feature tensors that guide per-pixel attention across images and final fusion weights (Lee, 2022). In abnormal occupancy-grid recognition, the underlying occupancy grid is itself the map, and channel-and-spatial recalibration is applied directly to $H\times W\times C$ feature tensors derived from that grid (Deng et al., 2021). In Attention Branch Networks, the map is a single-channel spatial attention map $A \in \mathbb{R}^{H\times W}$ that multiplicatively gates intermediate CNN features and can be aligned to human-edited targets through an $\ell_2$ attention-map loss (Mitsuhara et al., 2019). PHAM similarly externalizes attention maps by stacking the original image and its inferred attention map as a multi-dimensional input to a hypercomplex classifier (Lopez et al., 2023).

A distinct line of work treats attention maps themselves as the primary substrate. DAPE V2 interprets the pre-softmax score tensor $S \in \mathbb{R}^{B\times H\times L\times L}$ as a stack of 2D feature maps, one per head, and applies convolution before the softmax to refine those maps for length extrapolation (Zheng et al., 2024). AMMUNet stores decoder attention maps explicitly and merges them across scales with a fixed mask template, rather than discarding them after computing attention outputs (Yang et al., 2024). In multimodal Transformers with Learnable Attention Mask, the map is the full token-pair matrix $M \in \mathbb{R}^{H\times W\times C}$0 or $M \in \mathbb{R}^{H\times W\times C}$1, generated per layer and fused with attention scores (Barrios et al., 2024). In LVLM hallucination mitigation, all hidden states $M \in \mathbb{R}^{H\times W\times C}$2 form a semantic map $M \in \mathbb{R}^{H\times W\times C}$3 spanning layer and token dimensions (Li et al., 3 Aug 2025).

The surveyed literature therefore suggests that “map” is representational rather than exclusively geometric. It may denote a top-down spatial memory, a relevance raster, a semantic occupancy field, a full attention matrix, or a layer–token latent lattice. What unifies these cases is that structure is made explicit and then processed directly.

3. Core computational patterns

One recurring MAP pattern is direct self-attention over map cells. MaAST embeds each map cell by summing a semantic embedding and a learned positional embedding derived from discretizing a Gaussian centered on the agent, then applies a lightweight Transformer with 2 layers, 4 heads per layer, and embedding size 128. With $M \in \mathbb{R}^{H\times W\times C}$4, attention is standard,

$M \in \mathbb{R}^{H\times W\times C}$5

but the linear projections are implemented as $M \in \mathbb{R}^{H\times W\times C}$6 convolutions on the 2-D grid, preserving map-native processing before flattening and pooling (Seymour et al., 2021).

A second pattern is explicit fusion of multiple relevance layers into a control signal. In MLAM, active layers are summed with equal weights, so the per-region requirement is the sum of active layer scores, and the optimal active module set minimizes total cost subject to coverage and regional performance constraints. The paper formalizes this as

$M \in \mathbb{R}^{H\times W\times C}$7

with additional constraints $M \in \mathbb{R}^{H\times W\times C}$8 and $M \in \mathbb{R}^{H\times W\times C}$9 for relevant regions (Henning et al., 2021). Here attention is not merely feature weighting inside a neural block; it becomes an externalized spatial controller for inter-module optimization.

A third pattern is direct processing of attention maps as feature maps. AMMUNet computes local $151\times151$0 granular attention maps, places them on the block diagonal of a full attention matrix, and merges them with deeper global maps by

$151\times151$1

where $151\times151$2 is a fixed block-diagonal identity mask aligned to the current granularity (Yang et al., 2024). DAPE V2 similarly operates pre-softmax on attention-score maps: masked logits and bias are concatenated across channels, processed by two convolutions of kernel size $151\times151$3, then added back residually before softmax (Zheng et al., 2024). Learnable Attention Mask applies a per-layer mask generator $151\times151$4 and uses post-softmax fusion $151\times151$5 as its primary mechanism (Barrios et al., 2024). These designs share a common premise: the attention map is itself an object worth refining, regularizing, or reusing.

A fourth pattern is map-level aggregation across orthogonal axes of a latent tensor. In LVLM MAP, Layer-Wise Criss-Cross Attention defines for a query cell $151\times151$6 a row neighborhood within the current layer and a column neighborhood across layers at the same token position, then combines both with a trade-off parameter $151\times151$7. At the end of decoding, Global-Local Logit Fusion combines logits before and after global attention as

$151\times151$8

with $151\times151$9 used in practice (Li et al., 3 Aug 2025). This generalizes MAP from explicit spatial grids to a latent semantic map over depth and sequence position.

Finally, some works realize MAP without pairwise self-attention at all. Explicitly modeled attention maps parameterize spatial attention directly by distance, for example with a Gaussian kernel and a single learnable radius $1\ \text{m/cell}$0, then apply the normalized map to value features (Tan et al., 2020). Attention map reuse computes a standard attention map once per group of layers and reuses it in subsequent layers of the group, reinterpreting the map as a reusable layer-level artifact rather than a per-layer disposable intermediate (Shim et al., 2023).

4. Integration with learning, control, and decoding

MAP is typically embedded into a larger system rather than used in isolation. In MaAST, a 3-layer CNN encodes RGB-D into $1\ \text{m/cell}$1, the map Transformer produces $1\ \text{m/cell}$2, and both are fused and passed with the previous action to a 1-layer GRU; PPO then optimizes the actor-critic with navigation reward, exploration reward, and no auxiliary reconstruction or segmentation losses (Seymour et al., 2021). In QA-Net, map-level processing is intertwined with feature extraction: at each pixel, low-resolution image features and quality-map features are stacked across the image dimension, self-attention and QM-associated attention are applied across images, and the final feature is a quality-weighted mean

$1\ \text{m/cell}$3

This uses the map not only for attention but also for fusion (Lee, 2022).

In automated driving, MLAM sits beside an existing perception chain and controls it at two levels: module subset selection and intra-module gating. The proof-of-concept stack includes lidar object detection with PointPillars, radar object detection with a PointNet-like model, tracking with a Labeled Multi-Bernoulli filter or a longitudinal-only variant, and plausibilization. Situation detection activates attention layers; optimization chooses the module set; the MLAM is then projected into each active module to filter detections, search regions, or track initiation outside regions of interest (Henning et al., 2021). Here MAP is an operational controller over a heterogeneous perception stack.

Several systems use MAP as an explicit conditioning signal. PHAM computes attention maps offline with PatchConvNet, stacks them with the original image, and feeds the result to PHResNet, whose Parameterized Hypercomplex Convolution enforces structured channel mixing through

$1\ \text{m/cell}$4

For mammography, $1\ \text{m/cell}$5 corresponds to image plus attention map; for histopathology, $1\ \text{m/cell}$6 corresponds to RGB plus attention map (Lopez et al., 2023). In Attention Branch Networks, the attention branch outputs a single-channel map that gates perception features by $1\ \text{m/cell}$7, and human-edited attention targets are injected through $1\ \text{m/cell}$8 during fine-tuning (Mitsuhara et al., 2019).

MAP also appears in sequential decision-making beyond dense perception. In learning-assisted MAPF, each agent observes a 10-channel spatial tensor containing obstacles, starts, goals, cost-to-go fields, and predicted future occupancy of other agents; a CNN compresses it to an $1\ \text{m/cell}$9 map, a tokenizer forms $1^\circ$0 tokens, and a 16-layer, 16-head Visual Transformer produces a single-agent policy that is then integrated into M* as LM* (Virmani et al., 2021). In the Map-then-Act Paradigm, the term MAP refers to a three-stage procedure—Global Exploration, Task-Specific Mapping, and Knowledge-Augmented Execution—rather than a specific neural attention block. The paper does not introduce explicit neural attention over map nodes or edges, but it does describe a procedural focus over a structured cognitive map $1^\circ$1 conditioned by knowledge increment and state novelty (Liu et al., 13 May 2026). This marks a conceptual extension from attention over maps to reasoning grounded in maps.

5. Empirical record across domains

Reported results associate MAP with gains in sample efficiency, path efficiency, throughput, hallucination mitigation, and resource reduction, although the exact benefit depends on the substrate and system objective.

Domain	System	Reported outcome
Indoor navigation	MaAST	SPL 47, Success 54 at 14M steps
Automated driving perception	MLAM	59.0% processing-time reduction, 6.5% overhead
Remote-sensing segmentation	AMMUNet	mIoU 75.48% on Vaihingen, 77.90% on Potsdam
LVLM decoding	MAP	MME 1529.34 on LLaVA-1.5
Long-context language modeling	DAPE V2 MAP	Books3 perplexity 23.52 at train 128, test 8192
Transformer inference	Attention map reuse	Up to 2.58× GPU and 3.68× CPU speedup at $1^\circ$2

In navigation, MaAST improves Matterport3D validation performance from SPL 34 and Success 51 for RGB-D PPO to SPL 47 and Success 54, and from SPL 43 and Success 52 for RGBD+OCC to SPL 47 and Success 54. The paper attributes the improvement mainly to path efficiency, and it reports that 14M training steps correspond to an approximately 81% reduction in experience relative to a cited 75M-step reference, consistent with the abstract’s “80% decrease” (Seymour et al., 2021).

In driving perception, the combined effect of module subset selection and intra-module MLAM gating yields a 59.0% reduction in accumulated processing time relative to a naive full-stack baseline, with awareness-processing overhead of 6.5% of baseline processing time, median CPU load reduction of 25.4%, and median GPU load reduction of 48.7% (Henning et al., 2021). In remote sensing, UNetFormer + GMSA + AMMM achieves mIoU 75.48 and mAcc 82.97 on Vaihingen, and AMMUNet achieves mIoU 77.90 and mAcc 85.92 on Potsdam, while also improving FPS relative to patch-free MSA variants (Yang et al., 2024).

In sequence modeling, DAPE V2 reframes length extrapolation as attention-map processing. On Books3 with training length 128 and test length 8192, Kerple reports perplexity 66.23, DAPE–Kerple 25.01, and MAP–Kerple 23.52; on arXiv with training length 1024 and test length 8192, RoPE reports 256.12 while MAP–Kerple reports 3.37 (Zheng et al., 2024). Attention map reuse targets efficiency rather than accuracy improvements: in Conformer-M ASR with sequence length $1^\circ$3, the 8×2 reuse configuration reduces GPU latency from 37.82 ms to 14.63 ms and CPU latency from 1033.59 ms to 281.11 ms, while 2×8 and 4×4 reuse preserve WER more closely than 8×2 (Shim et al., 2023).

In LVLM hallucination mitigation, map-level decoding improves MME perception total score from 1491.56 to 1529.34 on LLaVA-1.5, from 1459.54 to 1466.36 on mPLUG-Owl2, and from 1271.43 to 1302.72 on InstructBLIP; MAP also reaches an overall MMHal-Bench score of 2.43 and improves POPE accuracy/F1 in several settings, such as 72.57/77.70 versus 68.77/75.73 on GQA-Adversarial for LLaVA-1.5 (Li et al., 3 Aug 2025). In multimodal medical classification, PHResNet18 with attention-map augmentation reaches AUC $1^\circ$4 on INbreast and $1^\circ$5 on CBIS-DDSM, while PHResNet50 with attention-map augmentation reaches accuracies $1^\circ$6, $1^\circ$7, and $1^\circ$8 on BreakHis at 100X, 200X, and 400X, respectively (Lopez et al., 2023).

6. Limitations, misconceptions, and research trajectory

A common misconception is that MAP is synonymous with standard Transformer self-attention over a spatial tensor. The record is broader. Some systems use full self-attention over explicit maps, as in MaAST; others use maps as resource-allocation controllers, as in MLAM; still others regulate, merge, prune, or reuse the attention maps themselves, as in AMMUNet, Learnable Attention Mask, attention-map-guided pruning, and attention-map reuse (Seymour et al., 2021, Henning et al., 2021, Yang et al., 2024, Barrios et al., 2024, Mao et al., 2023, Shim et al., 2023). MAP is therefore not a single architectural primitive but a family of mechanisms centered on explicit map substrates.

The limits of MAP vary with the substrate. Spatial-memory MAP in navigation depends on accurate pose, depth, and semantic channels; MaAST notes that SLAM drift would degrade stable egocentric maps, semantic noise lowers SPL, and artifacts in 3-D reconstructions can cause failures (Seymour et al., 2021). MLAM depends on correct situation detection, expert-derived module performance values, and equal-weight fusion across layers; the paper identifies quantization artifacts, map errors, and unmodeled scenarios as failure modes, mitigated by conservative dilation, coverage constraints, and escalation rules (Henning et al., 2021). QA-Net likely depends on quality-map accuracy, because mislabelled clear or disturbed pixels can misguide attention and fusion, although the paper does not empirically isolate that sensitivity (Lee, 2022).

Map-level processing of attention tensors also does not automatically solve complexity. MaAST explicitly retains global attention with $1^\circ$9 memory and time over map tokens, mitigating it only through a local crop and a light Transformer (Seymour et al., 2021). DAPE V2 preserves the $\{0,1\}$0 class of attention while adding convolutional processing over score maps, and Learnable Attention Mask leaves quadratic asymptotics unchanged while introducing modest overhead for mask generation (Zheng et al., 2024, Barrios et al., 2024). This suggests that MAP can improve structure and robustness without necessarily reducing asymptotic cost; efficiency gains arise only in specific formulations such as attention-map reuse or task-driven module gating (Shim et al., 2023, Henning et al., 2021).

Several papers also indicate open directions. MLAM suggests more rigorous task-specific performance metrics than expert knowledge and identifies weighted layer fusion as a plausible extension; MaAST notes that larger maps may require downsampling, local windows, or sparse attention; AMMUNet points to mask design and cross-window detail retention as open issues; the Map-then-Act Paradigm explicitly identifies typed-graph maps and learned attention over map nodes as natural next steps, while noting that such attention is not part of the current method (Henning et al., 2021, Seymour et al., 2021, Yang et al., 2024, Liu et al., 13 May 2026). Taken together, these works suggest a continuing shift from implicit, transient attention toward explicit structured substrates—semantic maps, relevance maps, quality maps, score maps, and cognitive maps—that can be inspected, supervised, optimized, and reused.