MapFormer: Structured & Cognitive Mapping

• MapFormer is a dual-framework approach that utilizes Transformer architectures enhanced with structured inductive biases for both conditional change detection and cognitive mapping.
• It integrates multi-modal feature fusion and pixelwise contrastive loss to significantly improve semantic alignment and IoU scores in remote sensing applications.
• The cognitive mapping variant leverages input-dependent positional embeddings based on Lie group theory to achieve robust out-of-distribution generalization in sequential tasks.

MapFormer refers to two distinct classes of architectures, each addressing different scientific domains—conditional change detection in remote sensing and self-supervised learning of cognitive maps in sequential modeling—unified by the goal of augmenting Transformer-based models with structured or semantic inductive biases. Both MapFormer frameworks leverage additional sources of information (pre-change semantic maps or input-dependent positional encodings) to achieve strong performance and robustness in their respective tasks (Bernhard et al., 2023, Rambaud et al., 24 Nov 2025).

1. Conditional Change Detection: MapFormer for Remote Sensing

MapFormer for remote sensing formulates and solves the Conditional Change Detection (CCD) problem, which uses pre-change semantic maps as auxiliary inputs for temporally-aware land-cover change detection. Given pre- ($I_{t-1}$) and post-change ($I_t$) remote sensing images and a pre-change semantic map ($M_{t-1}$), the model outputs a binary mask indicating per-pixel semantic change ($\hat{B}\in\{0,1\}^{H\times W}$), optionally predicting the updated semantic map ($\hat{M}_t$) in the semantic variant (Bernhard et al., 2023).

Key Workflow

1. Image Encoding: Both $I_{t-1}$ and $I_t$ are passed through a shared Mix-Vision-Transformer (MiT, following SegFormer), producing multi-scale feature maps $f_s^{(1)}$, $f_s^{(2)}$ for $s\in \{1,...,4\}$.
2. Map Encoding: The pre-change semantic map $M_{t-1}$ is one-hot encoded and processed by a lightweight CNN (1×1 convolution followed by two 5×5 dilated convolutions with dilation=2) to produce multi-scale feature maps $g_s^{(1)}$.
3. Multi-Modal Feature Fusion: At each spatial position and scale, the model concatenates $f_s^{(1)}$, $f_s^{(2)}$, and $g_s^{(1)}$, passing the resulting vectors through $K$ parallel two-layer MLPs to generate joint representations. Softmax attention weights, conditioned on $g_s^{(1)}$, are used to fuse the joint representations via a weighted sum.
4. Decoder: Fused multi-scale features are upsampled, concatenated, and processed through a SegFormer-style MLP decoder to produce the binary change mask and, optionally, the semantic segmentation output.

2. Cross-Modal Contrastive Supervision and Training Objectives

To enforce semantic alignment of spatial representations, MapFormer employs a supervised, pixelwise cross-modal contrastive loss. For each location $(i,j)$:

• Let $\pi: \mathbb{R}^{D_f}\to \mathbb{R}^{D_g}$ be a learnable linear projection.
• Cosine similarity: $\operatorname{sim}(u, v) = u^{\top}v/\|u\|\|v\|$.
• For pixel $(i,j)$, the loss is:

$$L^{(\text{cont})}_{ij} = -\operatorname{sim}\big(g^{(1)}_{ij}, \pi(f^{(1)}_{ij})\big)$$

with an additional negative or positive term, depending on whether the pixel is changed or unchanged.

• The full loss for training:

$$L = \lambda_{\text{seg}} L_{\text{seg}} + \lambda_{\text{cont}} L_{\text{cont}}$$

where $L_{\text{seg}}$ is a cross-entropy loss on the predicted binary mask (and segmentation label if used), $L_{\text{cont}}$ is the sum of contrastive losses, and $\lambda_{\text{seg}}, \lambda_{\text{cont}}$ are scaling coefficients (empirically set to 1).

3. Empirical Performance and Robustness

MapFormer demonstrates significant improvements over state-of-the-art methods (e.g., ChangeFormer) on benchmark datasets such as DynamicEarthNet and HRSCD. The inclusion of additional semantic map information and the proposed fusion/contrastive mechanisms more than double the IoU scores relative to state-of-the-art ($+11.7\%$ on DynamicEarthNet, $+18.4\%$ on HRSCD). The model remains robust under degraded semantic input (lower spatial resolution or fewer classes) and is insensitive to changes in the fusion module capacity ($K$ in the range $5$–$15$). Ablations confirm the crucial role of the contrastive loss for maximum benefit (Bernhard et al., 2023).

Method	DynENet IoU (%)	HRSCD IoU (%)
FHD	9.4	29.2
ChangerEx	11.8	22.7
ChangeFormer (SOTA)	11.5	29.6
Concatenation Baseline	11.6	25.9
MapFormer (K=10)	23.5 (+11.7)	48.0 (+18.4)

In terms of computational complexity, MapFormer’s per-pixel fusion remains tractable ($O(KD_f)$) for moderate $K$ and high-resolution inputs; the map encoder contributes negligible overhead.

4. MapFormer: Cognitive Map Learning with Input-Dependent Positional Embeddings

The second MapFormer paradigm (Rambaud et al., 24 Nov 2025) addresses the problem of learning cognitive maps—internal models that disentangle “structure” (relations, location) from “content”—in sequential domains. Motivated by principles from neuroscience (hippocampal place/grid-cell coding) and robust OOD generalization, MapFormer replaces fixed positional encodings in Transformers with input-dependent, action-based positional embeddings derived from Lie group theory.

Architecture Overview

• Action Matrices: Each action token $a\in\mathbb{R}^r$ is mapped to an invertible matrix $W_a\in\mathrm{GL}(n)$; position vectors are updated by $p(x+a) = W_a p(x)$. The collection $\{W_a\}$ forms a Lie subgroup $G$.
• Lie Algebra Reduction: Sequential composition is computed in the Lie algebra: $A(t)=\sum_{i=1}^D t_i A_i$, $W_a = \exp(A(t))$. Path integration is realized as a cumulative sum over generators, followed by a (block-diagonal) matrix exponential.
• EM vs. WM Variants:
  • MapFormer-EM (Episodic Memory): Maintains independent parameter slots for structure and content, processing absolute positions with action-based updates.
  • MapFormer-WM (Working Memory): Employs a learned rotary positional encoding, entangling structure and content in a single latent representation.

5. Structure–Content Disentanglement and Self-Supervised Path Integration

MapFormer establishes a principled separation between actions (structure) and observations (content):

• Structure tokens induce large structural displacements with negligible token value.
• Content tokens update token values with minimal structural displacement.
• In MapFormer-EM, structure and content retain independent subspaces, mimicking hippocampal and entorhinal population codes observed in neuroscience. In MapFormer-WM, rotation angles are learned per head, entangling indices similarly to prefrontal working memory circuits.
• Training objective is standard next-token cross-entropy; path integration is unsupervised and discovered through parallel algebraic manipulations.

6. Experimental Evaluation and Out-of-Distribution Generalization

MapFormer is evaluated on selective-copy and forced navigation (2D grid) tasks:

• Selective-copy: Tests content gating and memory.
• Forced navigation: Sequences of interleaved actions and observations, requiring the model to recall previously observed content when revisiting locations.

Results show RoPE and contextual gating methods fail to generalize to out-of-distribution contexts (longer sequences, denser or sparser actions) due to lack of inverse action modeling. In contrast, MapFormer (both EM and WM) achieves near 100% accuracy across both IID and OOD scenarios due to its structural bias and Lie-algebraic path integration mechanism. The EM variant achieves high memory efficiency and scalability; the WM variant is parameter-efficient at the expense of larger required state spaces (Rambaud et al., 24 Nov 2025).

7. Implications and Applications

The conditional change detection MapFormer has operational applicability in urban monitoring, disaster response, agricultural land-use detection, and the systematic updating of geospatial databases. Its architecture can be integrated with SegFormer and ChangeFormer pipelines and maintains near real-time inference for large inputs with moderate GPU compute.

MapFormer for cognitive map modeling provides a neurobiologically and mathematically principled approach to structure–content disentanglement in sequential models, with implications for robust OOD generalization in language, vision, and planning. The Lie-group framework offers direct interpretability in terms of biological substrates (e.g., grid and place cells), and supports future extensions to non-commutative groups and generalized relational world modeling. The model is compatible with current large-scale Transformer pretraining and opens directions toward richer relational reasoning, meta-control, and biologically grounded architectures (Rambaud et al., 24 Nov 2025).

In summary, both MapFormer frameworks exemplify the advance of inductive bias engineering in deep learning, drawing from domain priors or neurocomputational principles to overcome limitations of purely data-driven architectures.