MambaMatcher: Mamba-Based Matching

• MambaMatcher is a family of Mamba-based matching architectures that use selective state-space models to refine 4D correlation maps across various domains.
• It employs a similarity-aware selective scan and descending sorting of correlation tokens to enhance robustness in semantic correspondence tasks.
• The architecture achieves efficient linear-time processing with competitive PCK scores on benchmarks, improving over traditional Transformer-based approaches.

Searching arXiv for papers on “MambaMatcher” and closely related Mamba-based matching models. MambaMatcher denotes a class of Mamba-based matching architectures built on selective state-space models (SSMs), with the name used most explicitly for a semantic correspondence method that refines dense 4D correlation maps by a similarity-aware selective scan derived from Mamba (Kim et al., 29 Sep 2025). In related usage, the term also naturally describes Mamba-based matchers in sequential recommendation, local feature matching, semi-dense image matching, robotic imitation learning, and cross-modal sequence alignment, where the common design pattern is to replace or hybridize Transformer-style attention with selective SSMs to improve efficiency on long sequences or high-dimensional matching tensors (Yuan, 10 Apr 2025). Across these settings, MambaMatcher architectures typically preserve a matching formulation—user-to-item, image-to-image, state-to-action, or music-to-latent-motion—while exploiting Mamba’s linear-complexity scan, recurrent state dynamics, or hybrid Mamba–Transformer blocks.

1. Semantic correspondence formulation

In its most specific usage, MambaMatcher is a semantic correspondence method for establishing pixel- or region-wise correspondences between different instances of the same category, such as two different dogs, cars, or chairs (Kim et al., 29 Sep 2025). Semantic correspondence differs from optical flow, stereo matching, and classical local feature matching because it must remain robust under large intra-class variation, scale and viewpoint changes, and significant appearance differences including color, texture, and background clutter (Kim et al., 29 Sep 2025).

The method is positioned between two dominant paradigms. Feature-metric methods learn dense features and then apply a simple similarity such as dot product or cosine similarity, but do not explicitly model the global structure of the 4D correlation tensor. Correlation-metric methods construct a 4D correlation volume and refine it with 4D CNNs, Transformers, or related modules, but processing a tensor of size $H^2W^2$ is expensive and often forces low feature resolutions, restricted receptive fields, or shallow networks (Kim et al., 29 Sep 2025). MambaMatcher addresses this gap by using strong dense features together with explicit full-resolution 4D correlation refinement via a selective SSM (Kim et al., 29 Sep 2025).

The basic matching object is a multi-level 4D correlation volume. For source and target feature maps $\mathbf{F}'^{(l)}_s$ and $\mathbf{F}'^{(l)}_t$, the level-wise correlation is defined by cosine similarity: $$\mathbf{C}^{(l)}(p_s,p_t)= \frac{\mathbf{F}'^{(l)}_s(p_s)\cdot \mathbf{F}'^{(l)}_t(p_t)} {\|\mathbf{F}'^{(l)}_s(p_s)\|\ \|\mathbf{F}'^{(l)}_t(p_t)\|}.$$ These maps are stacked into

$$\mathbf{C}\in\mathbb{R}^{2L\times H\times W\times H\times W},$$

then flattened into

$$\overline{\mathbf{C}}\in\mathbb{R}^{N\times 2L},\quad N=H^2W^2,$$

so that each token corresponds to one source-target spatial pair and its $2L$-dimensional multi-level similarity vector (Kim et al., 29 Sep 2025).

A central implication is that MambaMatcher treats matching not as independent nearest-neighbor assignment, but as structured reasoning over the entire correlation field. This suggests a shift from local descriptor comparison toward global cost-volume modeling with linear-time sequence operators.

2. Similarity-aware selective state-space modeling

The core contribution of "MambaMatcher" (Kim et al., 29 Sep 2025) is a similarity-aware selective scan on the flattened 4D correlation tensor. Each row of $\overline{\mathbf{C}}$ is treated as a state token, and the sequence is sorted in descending order of a scalar similarity score derived from the final correlation level, i.e., the $2L$-th channel (Kim et al., 29 Sep 2025). High-score tokens are therefore processed first, and low-score or ambiguous matches later.

This ordering is the mechanism that makes the scan "similarity-aware." The stated rationale is that strong matches establish a context that later ambiguous positions can leverage for denoising and disambiguation (Kim et al., 29 Sep 2025). The paper reports ablations over ascending sort, no sorting, and Z-order or bidirectional scans, and finds that descending-similarity sorting gives the best PCK (Kim et al., 29 Sep 2025).

After sorting, the sequence is processed by a Mamba block. The paper presents the continuous-time SSM as

$$\begin{aligned} h'(t) &= \mathbf{A}h(t) + \mathbf{B}x(t),\ y(t) &= \mathbf{C}h(t) + Dx(t), \end{aligned}$$

with Zero-Order Hold discretization

$\mathbf{F}'^{(l)}_s$0

leading to

$\mathbf{F}'^{(l)}_s$1

In Mamba, $\mathbf{F}'^{(l)}_s$2, $\mathbf{F}'^{(l)}_s$3, and the step $\mathbf{F}'^{(l)}_s$4 are input-dependent, and a parallel scan computes the recurrence in linear time over sequence length (Kim et al., 29 Sep 2025).

The concrete Mamba block used by MambaMatcher includes input projection to an inner channel space with expansion factor 16, a local 1D convolution of width 4, parameter generation for the selective SSM, selective scan, output projection, inverse sorting, and reshaping back to a refined 4D correlation map $\mathbf{F}'^{(l)}_s$5 (Kim et al., 29 Sep 2025). The paper uses SSM expansion factor 16, local convolution width 4, and block expansion factor 3 (Kim et al., 29 Sep 2025).

This design places MambaMatcher in a distinct position relative to prior vision SSM models. Vision Mamba, VMamba, and related systems scan pixels or spatiotemporal tokens; MambaMatcher instead scans flattened 4D correlation entries, uses multi-level correlation scores as the state vector, and introduces task-specific similarity-based ordering (Kim et al., 29 Sep 2025).

3. End-to-end pipeline and prediction

The semantic correspondence pipeline begins with DINOv2 ViT-B/14 as backbone, using an input resolution of $\mathbf{F}'^{(l)}_s$6 and patch size 14, giving feature maps with $\mathbf{F}'^{(l)}_s$7 (Kim et al., 29 Sep 2025). From DINOv2, token and value features are extracted from intermediate layers 4 through 11, yielding $\mathbf{F}'^{(l)}_s$8 layers and therefore $\mathbf{F}'^{(l)}_s$9 feature maps per image (Kim et al., 29 Sep 2025). Each feature map is processed by a shared 2-layer 2D convolutional feature aggregator: $\mathbf{F}'^{(l)}_t$0 where the first convolution has kernel size 3 and output channels $\mathbf{F}'^{(l)}_t$1, the second convolution has kernel size 3 and output channels 768, and DINOv2 remains frozen while only the convs and correlation SSM are trained (Kim et al., 29 Sep 2025).

After 4D correlation construction and Mamba-based refinement, the refined correlation $\mathbf{F}'^{(l)}_t$2 is converted into dense flow and then into transferred keypoints using kernel soft-argmax (Kim et al., 29 Sep 2025). For each source position $\mathbf{F}'^{(l)}_t$3, the peak target position is found by

$\mathbf{F}'^{(l)}_t$4

then the normalized correlation is

$\mathbf{F}'^{(l)}_t$5

and dense target coordinates are obtained as

$\mathbf{F}'^{(l)}_t$6

A soft sampler $\mathbf{F}'^{(l)}_t$7 is then used for sub-pixel accuracy: $\mathbf{F}'^{(l)}_t$8 with

$\mathbf{F}'^{(l)}_t$9

The training loss is an $$\mathbf{C}^{(l)}(p_s,p_t)= \frac{\mathbf{F}'^{(l)}_s(p_s)\cdot \mathbf{F}'^{(l)}_t(p_t)} {\|\mathbf{F}'^{(l)}_s(p_s)\|\ \|\mathbf{F}'^{(l)}_t(p_t)\|}.$$0 keypoint regression loss,

$$\mathbf{C}^{(l)}(p_s,p_t)= \frac{\mathbf{F}'^{(l)}_s(p_s)\cdot \mathbf{F}'^{(l)}_t(p_t)} {\|\mathbf{F}'^{(l)}_s(p_s)\|\ \|\mathbf{F}'^{(l)}_t(p_t)\|}.$$1

and supervision is provided directly by keypoint pairs on PF-PASCAL and SPair-71k (Kim et al., 29 Sep 2025).

A notable architectural point is that no auxiliary contrastive or cross-entropy loss on matches is used; the model supervises the entire correlation-refinement pipeline through keypoint position error alone (Kim et al., 29 Sep 2025). This suggests that the correlation tensor itself is treated as the principal latent object of optimization.

4. Efficiency, benchmarks, and ablations

MambaMatcher’s complexity claim rests on replacing expensive 4D processing with linear-time selective scanning. With $$\mathbf{C}^{(l)}(p_s,p_t)= \frac{\mathbf{F}'^{(l)}_s(p_s)\cdot \mathbf{F}'^{(l)}_t(p_t)} {\|\mathbf{F}'^{(l)}_s(p_s)\|\ \|\mathbf{F}'^{(l)}_t(p_t)\|}.$$2, the paper states the following asymptotic costs: 4D convolution with kernel $$\mathbf{C}^{(l)}(p_s,p_t)= \frac{\mathbf{F}'^{(l)}_s(p_s)\cdot \mathbf{F}'^{(l)}_t(p_t)} {\|\mathbf{F}'^{(l)}_s(p_s)\|\ \|\mathbf{F}'^{(l)}_t(p_t)\|}.$$3 has complexity $$\mathbf{C}^{(l)}(p_s,p_t)= \frac{\mathbf{F}'^{(l)}_s(p_s)\cdot \mathbf{F}'^{(l)}_t(p_t)} {\|\mathbf{F}'^{(l)}_s(p_s)\|\ \|\mathbf{F}'^{(l)}_t(p_t)\|}.$$4, 4D dot-product attention has complexity $$\mathbf{C}^{(l)}(p_s,p_t)= \frac{\mathbf{F}'^{(l)}_s(p_s)\cdot \mathbf{F}'^{(l)}_t(p_t)} {\|\mathbf{F}'^{(l)}_s(p_s)\|\ \|\mathbf{F}'^{(l)}_t(p_t)\|}.$$5, and Mamba selective SSM has complexity $$\mathbf{C}^{(l)}(p_s,p_t)= \frac{\mathbf{F}'^{(l)}_s(p_s)\cdot \mathbf{F}'^{(l)}_t(p_t)} {\|\mathbf{F}'^{(l)}_s(p_s)\|\ \|\mathbf{F}'^{(l)}_t(p_t)\|}.$$6 (Kim et al., 29 Sep 2025). For the chosen setting, 4D Conv with $$\mathbf{C}^{(l)}(p_s,p_t)= \frac{\mathbf{F}'^{(l)}_s(p_s)\cdot \mathbf{F}'^{(l)}_t(p_t)} {\|\mathbf{F}'^{(l)}_s(p_s)\|\ \|\mathbf{F}'^{(l)}_t(p_t)\|}.$$7 requires about 33.6 GFLOPs, FastFormer about 1.74 GFLOPs, Mamba selective SSM about 23.2 GFLOPs, and the sorting overhead is estimated at about 0.064 GFLOPs (Kim et al., 29 Sep 2025).

The reported latency and memory figures on a SPair-71k small subset at $$\mathbf{C}^{(l)}(p_s,p_t)= \frac{\mathbf{F}'^{(l)}_s(p_s)\cdot \mathbf{F}'^{(l)}_t(p_t)} {\|\mathbf{F}'^{(l)}_s(p_s)\|\ \|\mathbf{F}'^{(l)}_t(p_t)\|}.$$8 resolution are 1.67 GB and 28.8 ms for FastFormer correlation aggregation, versus 1.64 GB and 16.4 ms for Mamba with similarity-aware scan (Kim et al., 29 Sep 2025). Overall forward time including DINOv2 is approximately 74 ms with memory approximately 2.1 GB for $$\mathbf{C}^{(l)}(p_s,p_t)= \frac{\mathbf{F}'^{(l)}_s(p_s)\cdot \mathbf{F}'^{(l)}_t(p_t)} {\|\mathbf{F}'^{(l)}_s(p_s)\|\ \|\mathbf{F}'^{(l)}_t(p_t)\|}.$$9 resolution (Kim et al., 29 Sep 2025).

The main quantitative results reported for $$\mathbf{C}\in\mathbb{R}^{2L\times H\times W\times H\times W},$$0 input are summarized below.

Benchmark	Metric	MambaMatcher
PF-PASCAL	PCK@$$\mathbf{C}\in\mathbb{R}^{2L\times H\times W\times H\times W},$$1	87.3 / 95.9 / 98.2 at $$\mathbf{C}\in\mathbb{R}^{2L\times H\times W\times H\times W},$$2
SPair-71k	PCK@$$\mathbf{C}\in\mathbb{R}^{2L\times H\times W\times H\times W},$$3	61.6 / 77.8 / 84.3
PF-WILLOW	[email protected]/0.10	56.2 / 81.1 (image) and 47.4 / 72.1 (bbox-kp)

On PF-PASCAL, the method is reported as state-of-the-art at $$\mathbf{C}\in\mathbb{R}^{2L\times H\times W\times H\times W},$$4 and $$\mathbf{C}\in\mathbb{R}^{2L\times H\times W\times H\times W},$$5, and extremely competitive at $$\mathbf{C}\in\mathbb{R}^{2L\times H\times W\times H\times W},$$6 (Kim et al., 29 Sep 2025). On SPair-71k, it outperforms both feature-metric baselines such as DINOv2 and DIFT and correlation-metric baselines such as TransforMatcher, CATs++, and HCCNet (Kim et al., 29 Sep 2025). On PF-WILLOW, several baselines slightly outperform it, which the paper interprets as weaker cross-dataset generalization under this supervised training regime (Kim et al., 29 Sep 2025).

The ablation results sharpen the architectural picture. Token features outperform key and query facets, while token plus value gives most of the gain and is therefore selected (Kim et al., 29 Sep 2025). For feature aggregation, simple 2D convolution with kernel size 3 is both good and efficient, while kernel size 5 is slightly better but costlier (Kim et al., 29 Sep 2025). For correlation aggregation, Mamba with descending similarity sorting reaches 79.3 [email protected] on SPair-71k small, outperforming FastFormer at 78.9, plain Mamba at 78.5, and 4D Conv with $$\mathbf{C}\in\mathbb{R}^{2L\times H\times W\times H\times W},$$7 at 78.2 (Kim et al., 29 Sep 2025).

The resolution study reports SPair-71k PCK values of 26.4 / 39.7 / 46.5 at $$\mathbf{C}\in\mathbb{R}^{2L\times H\times W\times H\times W},$$8, 61.6 / 77.8 / 84.3 at $$\mathbf{C}\in\mathbb{R}^{2L\times H\times W\times H\times W},$$9, and 64.2 / 78.4 / 85.2 at $$\overline{\mathbf{C}}\in\mathbb{R}^{N\times 2L},\quad N=H^2W^2,$$0, with the paper describing $$\overline{\mathbf{C}}\in\mathbb{R}^{N\times 2L},\quad N=H^2W^2,$$1 as a good tradeoff because gains saturate at higher resolution (Kim et al., 29 Sep 2025). Sorting by the last layer’s scores performs slightly better than using the penultimate layer or the mean across layers (Kim et al., 29 Sep 2025).

5. Related uses of the “MambaMatcher” pattern

The term also applies naturally to several Mamba-based matching architectures outside semantic correspondence. In sequential recommendation, the paper "A Novel Mamba-based Sequential Recommendation Method" introduces Hydra, a multi-head latent Mamba architecture for sequential recommendation, and explicitly notes that a “MambaMatcher” is naturally instantiated by Hydra as a recommender that matches users with items from long interaction histories and large catalogs (Yuan, 10 Apr 2025). Hydra uses an item model, stacked Hydra layers, and a prediction layer that scores the last hidden state against all item embeddings (Yuan, 10 Apr 2025). It supports both ID-based embeddings and an Item LLM, and its multi-head latent interaction factorizes the Mamba state into multiple low-dimensional subspaces fused with a RoPE-enhanced item-information stream (Yuan, 10 Apr 2025).

Hydra’s matching stage is standard user-item scoring,

$$\overline{\mathbf{C}}\in\mathbb{R}^{N\times 2L},\quad N=H^2W^2,$$2

where $$\overline{\mathbf{C}}\in\mathbb{R}^{N\times 2L},\quad N=H^2W^2,$$3 summarizes the user history and $$\overline{\mathbf{C}}\in\mathbb{R}^{N\times 2L},\quad N=H^2W^2,$$4 is the item embedding table (Yuan, 10 Apr 2025). The paper reports, on Movies & TV, that Hydra-0.28B achieves R@10 $$\overline{\mathbf{C}}\in\mathbb{R}^{N\times 2L},\quad N=H^2W^2,$$5 and N@10 $$\overline{\mathbf{C}}\in\mathbb{R}^{N\times 2L},\quad N=H^2W^2,$$6 with average epoch time $$\overline{\mathbf{C}}\in\mathbb{R}^{N\times 2L},\quad N=H^2W^2,$$7 s, compared with HSTU-large at R@10 $$\overline{\mathbf{C}}\in\mathbb{R}^{N\times 2L},\quad N=H^2W^2,$$8, N@10 $$\overline{\mathbf{C}}\in\mathbb{R}^{N\times 2L},\quad N=H^2W^2,$$9, average epoch time $2L$0 s, and SASRec-1B at R@10 $2L$1, N@10 $2L$2, average epoch time $2L$3 s (Yuan, 10 Apr 2025). The paper states that Hydra uses 28% of HSTU’s parameters and trains in about 12% of HSTU’s time while yielding better accuracy (Yuan, 10 Apr 2025).

In sparse local feature matching, "MambaGlue: Fast and Robust Local Feature Matching With Mamba" describes a hybrid matcher in which Mamba-based self-attention is combined with cross-attention and a deep confidence score regressor (Ryoo et al., 1 Feb 2025). Its MambaAttention mixer produces a message

$2L$4

combining a self-attention branch, a Mamba selective scan branch, and a non-scan Mamba-style branch (Ryoo et al., 1 Feb 2025). The matcher uses exit tests and feature pruning identical to LightGlue’s procedures, but replaces LightGlue’s shallow confidence estimator with a deeper MLP regressor (Ryoo et al., 1 Feb 2025). On HPatches, with SuperPoint features, the paper reports PR@3px 94.6 and LO-RANSAC AUC@1px / @5px of 39.0 / 79.3, surpassing SuperGlue, SGMNet, and LightGlue in that table (Ryoo et al., 1 Feb 2025). On Aachen Day–Night, it reports 89.0 / 95.3 / 98.7 day recall and 86.7 / 93.9 / 100.0 night recall, with throughput 16.7 pairs/s, comparable to LightGlue’s 17.2 pairs/s (Ryoo et al., 1 Feb 2025).

In semi-dense matching, "VMatcher: State-Space Semi-Dense Local Feature Matching" is a hybrid Mamba–Transformer detector-free matcher that uses MambaVision and gMLP layers as primary coarse-scale sequence encoders, with a small number of downsampled Transformer layers for self- and cross-attention (Youssef, 31 Jul 2025). The paper presents VMatcher-B and VMatcher-T as hybrid patterns in which most layers are Mamba-based, attention is spatially downsampled, and matching follows a LoFTR-style coarse-to-fine semi-dense pipeline (Youssef, 31 Jul 2025). Reported results include HPatches LO-RANSAC AUC@3/5/10 of 70.2 / 79.3 / 87.4 for VMatcher-T and 69.8 / 79.1 / 87.5 for VMatcher-B, with runtimes 29.23 ms and 35.43 ms, respectively (Youssef, 31 Jul 2025). On MegaDepth, VMatcher-B reaches 69.6 / 81.1 / 88.9 LO-RANSAC AUC@5/10/20 at 87.56 ms, compared with ELoFTR at 69.4 / 80.8 / 88.7 at 100.25 ms (Youssef, 31 Jul 2025).

The same pattern extends beyond image or recommendation matching. In robotic imitation learning, Mamba is used as a compact motion encoder that compresses the robot’s joint angle and torque history into a 4-dimensional state, which the paper interprets as continuously matching current motion context to future actions (Tsuji, 2024). In music-to-dance generation, "MatchDance" uses a Mamba–Transformer hybrid to map music into discrete motion codes, functioning as a cross-modal sequence matcher in latent space (Yang et al., 20 May 2025). These uses indicate that “MambaMatcher” has developed into a broader architectural label rather than a single-task term.

6. Limitations, failure modes, and architectural implications

The semantic correspondence MambaMatcher paper explicitly notes three limitations: large viewpoint changes combined with symmetry ambiguity can cause confusion between symmetric parts; multiple similar instances can lead the method to align to a different instance than the annotated one; and cross-dataset generalization on PF-WILLOW is competitive but not state-of-the-art (Kim et al., 29 Sep 2025). The paper also reports that refinement depends strongly on high-confidence initial correlations: removing top-$2L$5 of correlation scores degrades performance more severely for larger $2L$6, and integrated gradients indicate that descending-similarity ordering concentrates useful evidence earlier in the sequence (Kim et al., 29 Sep 2025).

A broader limitation of Mamba-based architectures is discussed in "Revealing and Mitigating the Local Pattern Shortcuts of Mamba" (You et al., 2024). That paper argues that Mamba often relies on positional shortcuts and n-gram shortcuts, performing well when key information is local or templatic but struggling with distributed information (You et al., 2024). It introduces a global selection module that augments the gate $2L$7 with a long-convolution-based global signal: $2L$8 With only 4M extra parameters, the paper reports improvement on a hard distributed-information task from 0 to 80.54 points for a 130M Mamba model (You et al., 2024). This suggests that MambaMatcher-style architectures may benefit from explicit global gating or hybridization when the matching evidence is highly dispersed.

A plausible implication is that different MambaMatcher variants respond to the same underlying tension: Mamba offers linear complexity and efficient long-sequence processing, but task performance depends on how global evidence is routed. Semantic MambaMatcher resolves this by sorting the 4D correlation sequence by confidence before scanning (Kim et al., 29 Sep 2025); Hydra decomposes state into multiple low-dimensional heads and separates historical and item-information streams (Yuan, 10 Apr 2025); MambaGlue and VMatcher retain explicit attention blocks for pairwise interaction (Ryoo et al., 1 Feb 2025, Youssef, 31 Jul 2025). These designs can be read as domain-specific solutions to the selectivity problem identified in (You et al., 2024).

In that sense, MambaMatcher is best understood not merely as a proper noun for a single model, but as a family of matching systems that use Mamba-derived selective state-space modeling to reconcile global reasoning with scalable computation. The explicit semantic correspondence method in (Kim et al., 29 Sep 2025) is the canonical instance, while related Mamba-based recommenders, local matchers, semi-dense matchers, and cross-modal sequence generators show how the same principle generalizes across matching problems with very long histories, large candidate spaces, or high-dimensional correlation structures.