---
title: Dual-Scale-Former for Visual Place Recognition
url: https://www.emergentmind.com/topics/dual-scale-former-dsformer
type: topic
---

# Dual-Scale-Former for Visual Place Recognition

Searching arXiv for the relevant DSFormer variants and the specific "Dual-Scale-Former" paper to ground the article in the current literature.
arXiv search query: 2507.18444 DSFormer Dual-Scale Cross-Learning Transformer Visual Place Recognition

Dual-Scale-Former (DSFormer) is a Transformer-based cross-learning module for Visual Place Recognition (VPR) that integrates dual-scale CNN features, intra-scale self-attention, and shared bidirectional cross-attention to produce a compact 512-dimensional global descriptor for single-stage retrieval [2507.18444]. In the formulation reported on arXiv, DSFormer is paired with a block clustering strategy for the SF-XL training set, with the combined system targeting robustness to viewpoint changes, illumination and seasonal variability, occlusions, and perceptual aliasing while maintaining memory and latency characteristics suitable for large-scale deployment [2507.18444]. The name is potentially ambiguous across domains: other arXiv papers use closely related spellings for a Dual Sparse Transformer in multispectral object detection and a Double Sampling Transformer in multivariate time-series forecasting, but those are distinct models and do not define “Dual-Scale-Former” in the VPR sense [2606.31015], [2308.03274].

## 1. Nomenclature and scope

Within VPR, DSFormer denotes “Dual-Scale-Former,” specifically expanded in the paper title as “A Dual-Scale Cross-Learning Transformer for Visual Place Recognition” [2507.18444]. Its core idea is to fuse the final two CNN feature scales by combining self-attention for within-scale dependency modeling with shared cross-attention for cross-scale information transfer, yielding a compact global embedding for retrieval.

The acronym is not unique across arXiv. In multispectral detection, DSFormer denotes a “Dual Sparse Transformer,” the core block inside DSAFormer; that paper explicitly states that it does not use or define “Dual-Scale-Former” and instead decomposes DSFormer into a Spatial Sparse Transformer and a Channel Sparse Transformer [2606.31015]. In long-term forecasting, DSformer denotes a “Double Sampling Transformer,” built from a double sampling block and a temporal variable attention block for multivariate time series [2308.03274]. This naming overlap makes domain qualification important when citing “DSFormer.”

| Term in paper | Expansion | Domain |
|---|---|---|
| DSFormer | Dual-Scale-Former / Dual-Scale Cross-Learning Transformer | Visual Place Recognition |
| DSFormer | Dual Sparse Transformer | Multispectral object detection |
| DSformer | Double Sampling Transformer | Multivariate time-series long-term prediction |

## 2. Problem setting and design objectives

The VPR formulation addressed by DSFormer is global image retrieval for mobile robot localization under severe environmental and geometric variation. The paper identifies the central difficulties as viewpoint changes, illumination and seasonal variability, occlusions, and perceptual aliasing, together with practical constraints on memory and latency in large-scale systems [2507.18444]. The model is therefore designed to avoid dependence on costly local-feature storage or geometric reranking, instead producing a single global descriptor per image.

The architectural response is dual-scale feature integration. DSFormer extracts the final two CNN layers so that deeper features contribute semantic richness and shallower features contribute spatial detail, then performs bidirectional information transfer between them. This arrangement is meant to preserve long-range dependencies within each scale and complementary correlations between scales. A plausible implication is that the model treats scale interaction not as a late fusion heuristic but as a learned alignment problem embedded directly in the encoder.

The framework is complemented by a block clustering strategy for the SF-XL training dataset. According to the paper, this repartitioning improves class coherence for geographically proximate locations, reduces redundancy, increases robustness to viewpoint changes, and decreases training data by approximately \(25\%\) to \(32\%\) relative to prior SF-XL partitions, with “approximately \(30\%\)” emphasized in the abstract [2507.18444].

## 3. Architectural formulation

The baseline backbone is a pretrained ResNet-50 truncated at the final two layers. All residual blocks except the last two are frozen during training, and linear projections standardize channel dimensions before attention [2507.18444]. The two resulting feature maps are denoted \(f_1 \in \mathbb{R}^{C_1 \times H_1 \times W_1}\) and \(f_2 \in \mathbb{R}^{C_2 \times H_2 \times W_2}\), with token counts \(N_1 = H_1 W_1\) and \(N_2 = H_2 W_2\) after flattening. Positional encodings are then added to preserve geometry.

Within each scale, DSFormer uses Transformer encoders with pre-normalization and residual connections:
$$
z_t = \mathrm{MHA}(\mathrm{LN}(z_{l-1})) + z_{l-1}, \qquad
z_l = \mathrm{FFN}(\mathrm{LN}(z_t)) + z_t.
$$
The self-attention is augmented with IRPE (Improving Relative Position Encoding):
$$
\mathrm{Attention}_{\mathrm{IRPE}}(q,k,v)
=
rpe_v\!\left(
\mathrm{softmax}\!\left(\frac{qk^\top}{\sqrt{d_k}} + rpe_q(q) + rpe_k(k)\right)
\right)v.
$$
This adds relative-position terms to the logits and modulates outputs with learnable relative position functions, improving spatial coherence and long-range reasoning [2507.18444].

The distinctive cross-scale mechanism is a shared-parameter multi-head cross-attention applied in both directions. Let \(z_i \in \mathbb{R}^{N_i \times C}\) denote the token sequence of scale \(i \in \{1,2\}\). Cross-attention from scale \(i\) to scale \(m\), with \(i \neq m\), is:
$$
Q_i = z_i W_{Q_i}, \qquad
K_m = z_m W_{K_m}, \qquad
V_m = z_m W_{V_m},
$$
$$
\mathrm{CrossAttn}(z_i, z_m) =
\mathrm{softmax}\!\left(\frac{Q_i K_m^\top}{\sqrt{d_{K_m}}}\right)V_m.
$$
The same cross-attention parameters are shared across the two directions, encouraging symmetric alignment between scales rather than independent one-way fusion [2507.18444].

After \(L\) encoder layers, the fused dual-scale features are aggregated with GeM pooling and projected to a 512-dimensional descriptor, followed by \(L_2\) normalization:
$$
p = \mathrm{GeM}(H), \qquad
z = \mathrm{proj}(p) \in \mathbb{R}^{512}.
$$
In the ResNet-50 setting, DSFormer uses \(L=3\) encoder layers and 16 heads; in the DINOv2 setting, it uses \(L=1\) encoder layer with 12 shared heads, fine-tuning the final two layers of DINOv2 (ViT-B) at input resolution \(322 \times 322\) [2507.18444].

The paper gives the per-layer attention complexity as
$$
O(N_1^2 + N_2^2 + 2N_1N_2),
$$
combining two intra-scale self-attentions and two directional cross-attentions. The stated mitigation is that attention operates on CNN feature maps rather than raw pixels, so token counts remain practical [2507.18444].

## 4. Block clustering strategy and optimization

The block clustering strategy restructures SF-XL into density-based, geographically coherent classes using UTM coordinates. For east-coordinate grouping, the initial assignment is
$$
j = \left\lfloor \frac{((x_e^i - \min(\{x_e\})) \bmod (M \cdot N))}{M} \right\rfloor + 1,
$$
with block width \(M=10\) m and \(N=5\) groups per direction [2507.18444]. HDBSCAN is then applied within each group to form density-based clusters, after which KNN retains only samples within radius \(r=7.5\) m of the peak-density location. Classes whose peak density locations are closer than \(l=40\) m are excluded to prevent overlaps. The same procedure is repeated for northing.

The east and north groups are merged as
$$
\{G_j\}_{j=1}^N = \{G^e_j \cup G^n_j\}_{j=1}^N.
$$
Finally, viewpoint robustness is improved through perspective selection. For each retained class with coordinates \(X_k \in \mathbb{R}^{n \times 2}\), the centered coordinates are decomposed by SVD, and focal points are defined as
$$
c_k = E[X_k] + F \cdot V,
$$
with \(F=15\) m. Splitting classes by the two principal directions doubles the groups to \(2N\) [2507.18444]. The resulting processed SF-XL training set contains approximately \(4.2\) million images across 10 groups, compared with approximately \(5.6\) million for CosPlace and approximately \(6.2\) million for EigenPlaces partitions [2507.18444].

Training follows EigenPlaces and uses Large Margin Cosine Loss (LMCL, CosFace):
$$
L_{\mathrm{LMCL}}
=
-
\sum_i
\log
\frac{
\exp\!\left(s(\cos\theta_{i,y}-m)\right)
}{
\sum_j \exp(s\cos\theta_{i,j})
}.
$$
The optimizer is Adam with learning rate \(1 \times 10^{-5}\) for the main model and a separate Adam optimizer with learning rate \(0.01\) for the classifier. Training runs for up to 40 epochs on an NVIDIA RTX 4080 (16 GB), with batch size 32 and 10,000 iterations per group per epoch. Inputs are resized to \(320 \times 320\) with random scaling and color jittering for the ResNet-50 configuration [2507.18444].

## 5. Empirical performance and efficiency

The evaluation protocol uses single-stage global retrieval with 512-dimensional descriptors and no reranking. Success for Recall@\(N\) is defined by a match within 25 m on MSLS Val, MSLS Challenge, Pittsburgh30k, Tokyo 24/7, Nordland, and large-scale SF-XL test subsets [2507.18444]. In the ResNet-50 setting, DSFormer reports average Recall@1 of 83.8 across five benchmarks, compared with 83.3 for R2Former, 78.1 for TransVPR, 76.7 for DELG, and 72.8 for Patch-NetVLAD. Its per-dataset Recall@1 values are 88.9 on MSLS Val, 68.1 on MSLS Challenge, 91.9 on Pitts30k, 88.6 on Tokyo24/7, and 81.5 on Nordland [2507.18444].

Against single-stage retrieval baselines using ResNet-50 and 512-dimensional descriptors, the reported averages are 77.5 for CosPlace, 77.9 for EigenPlaces, 76.8 for MixVPR, 78.6 for BoQ+PCA, 80.6 for GeM† trained on the block-clustered partition, and 83.8 for DSFormer [2507.18444]. The paper explicitly states that DSFormer improves over EigenPlaces by \(+5.9\) Recall@1 points on average and over GeM† by \(+3.2\), with especially strong gains on Tokyo24/7 and Nordland.

The DINOv2 configuration raises the average Recall@1 to 89.3, compared with 86.3 for SALAD+PCA, 86.3 for BoQ+PCA, 84.8 for CricaVPR+PCA, and 88.4 for GeM† with DINOv2 [2507.18444]. On the large-scale SF-XL test subsets, the ResNet-50 model reports Recall@1 values of 85.7 on v1, 91.3 on v2, 31.5 on Night, and 42.1 on Occlusion, averaging 62.7, while the DINOv2 model reaches 93.7, 94.8, 54.1, and 52.6 respectively, averaging 73.8 [2507.18444].

Efficiency is a central empirical claim. On MSLS Val using an RTX 3060, DSFormer uses a descriptor of size \(1 \times 512\), memory of 0.039 GB, and extraction time of 0.024 s/query, with no reranking [2507.18444]. Under the same comparison, R2Former uses 5.2 GB and 0.423 s/query for reranking, TransVPR uses 24.2 GB and 2.118 s/query for reranking, Patch-NetVLAD uses 908.3 GB and 16.88 s/query for reranking, and DELG uses 7.8 GB and 35.8 s/query for reranking. The paper characterizes this as substantially lower memory and latency than two-stage reranking pipelines [2507.18444].

Ablation studies attribute the gains to both the architecture and the data partition. With a ResNet-50 backbone, the number of DSFormer layers was varied from 0 to 4; three layers gave the best overall trade-off across MSLS Val, Pitts30k, Tokyo24/7, and Nordland. Removing IRPE, self-attention, cross-attention, or block clustering consistently reduced Recall@1, with block clustering showing a notable effect on Nordland [2507.18444].

## 6. Limitations, interpretations, and related models

The paper notes that DSFormer still faces difficulty under extreme domain shifts, especially the Night and Occlusion subsets, although the DINOv2-based configuration reduces these gaps substantially [2507.18444]. It also states that the method assumes dual-scale CNN features are sufficient; for ultra-wide viewpoint changes or sparse visual environments, additional cues such as depth or multi-view temporal context could help. The block clustering procedure relies on heuristic parameters \(M\), \(r\), \(l\), and \(F\), suggesting a dependence on manual tuning even though the reported empirical results are strong [2507.18444].

Future directions proposed in the paper include cross-domain pretraining, integrating geometric priors, scaling DSFormer layers with dynamic token pruning for faster inference, and coupling global descriptors with lightweight local verification when needed [2507.18444]. This suggests that the model is best understood not as a rejection of reranking or local verification in principle, but as a strong global retrieval backbone that already attains high accuracy without them.

The broader literature also makes clear that “DSFormer” should not be treated as a single architecture family across domains. In multispectral object detection, the term refers to a Dual Sparse Transformer inside DSAFormer, with spatial sparse multi-head cross-attention, channel sparse multi-head cross-attention, and a multi-scale feature refinement layer; that paper explicitly distinguishes its terminology from “Dual-Scale-Former” [2606.31015]. In multivariate time-series forecasting, DSformer denotes a Double Sampling Transformer that uses down sampling, piecewise sampling, and temporal-variable attention for long-term prediction [2308.03274]. Consequently, in encyclopedic or bibliographic usage, “Dual-Scale-Former (DSFormer)” is most precisely reserved for the VPR model built around dual-scale cross-learning and block-clustered SF-XL training [2507.18444].

Source: https://www.emergentmind.com/topics/dual-scale-former-dsformer