Dual-Scale-Former for Visual Place Recognition

• The paper introduces DSFormer, a Transformer-based cross-learning module that fuses dual-scale CNN features into a compact 512-dimensional descriptor.
• It employs intra-scale self-attention and shared bidirectional cross-attention to integrate spatial detail with semantic richness.
• The block clustering strategy enhances training efficiency and robustness to viewpoint and illumination changes, achieving superior Recall@1 scores.

Searching arXiv for the relevant DSFormer variants and the specific "Dual-Scale-Former" paper to ground the article in the current literature. arXiv search query: (Jiang et al., 24 Jul 2025) DSFormer Dual-Scale Cross-Learning Transformer Visual Place Recognition

Dual-Scale-Former (DSFormer) is a Transformer-based cross-learning module for Visual Place Recognition (VPR) that integrates dual-scale CNN features, intra-scale self-attention, and shared bidirectional cross-attention to produce a compact 512-dimensional global descriptor for single-stage retrieval (Jiang et al., 24 Jul 2025). In the formulation reported on arXiv, DSFormer is paired with a block clustering strategy for the SF-XL training set, with the combined system targeting robustness to viewpoint changes, illumination and seasonal variability, occlusions, and perceptual aliasing while maintaining memory and latency characteristics suitable for large-scale deployment (Jiang et al., 24 Jul 2025). The name is potentially ambiguous across domains: other arXiv papers use closely related spellings for a Dual Sparse Transformer in multispectral object detection and a Double Sampling Transformer in multivariate time-series forecasting, but those are distinct models and do not define “Dual-Scale-Former” in the VPR sense (Wu et al., 30 Jun 2026, Yu et al., 2023).

1. Nomenclature and scope

Within VPR, DSFormer denotes “Dual-Scale-Former,” specifically expanded in the paper title as “A Dual-Scale Cross-Learning Transformer for Visual Place Recognition” (Jiang et al., 24 Jul 2025). Its core idea is to fuse the final two CNN feature scales by combining self-attention for within-scale dependency modeling with shared cross-attention for cross-scale information transfer, yielding a compact global embedding for retrieval.

The acronym is not unique across arXiv. In multispectral detection, DSFormer denotes a “Dual Sparse Transformer,” the core block inside DSAFormer; that paper explicitly states that it does not use or define “Dual-Scale-Former” and instead decomposes DSFormer into a Spatial Sparse Transformer and a Channel Sparse Transformer (Wu et al., 30 Jun 2026). In long-term forecasting, DSformer denotes a “Double Sampling Transformer,” built from a double sampling block and a temporal variable attention block for multivariate time series (Yu et al., 2023). This naming overlap makes domain qualification important when citing “DSFormer.”

Term in paper	Expansion	Domain
DSFormer	Dual-Scale-Former / Dual-Scale Cross-Learning Transformer	Visual Place Recognition
DSFormer	Dual Sparse Transformer	Multispectral object detection
DSformer	Double Sampling Transformer	Multivariate time-series long-term prediction

2. Problem setting and design objectives

The VPR formulation addressed by DSFormer is global image retrieval for mobile robot localization under severe environmental and geometric variation. The paper identifies the central difficulties as viewpoint changes, illumination and seasonal variability, occlusions, and perceptual aliasing, together with practical constraints on memory and latency in large-scale systems (Jiang et al., 24 Jul 2025). The model is therefore designed to avoid dependence on costly local-feature storage or geometric reranking, instead producing a single global descriptor per image.

The architectural response is dual-scale feature integration. DSFormer extracts the final two CNN layers so that deeper features contribute semantic richness and shallower features contribute spatial detail, then performs bidirectional information transfer between them. This arrangement is meant to preserve long-range dependencies within each scale and complementary correlations between scales. A plausible implication is that the model treats scale interaction not as a late fusion heuristic but as a learned alignment problem embedded directly in the encoder.

The framework is complemented by a block clustering strategy for the SF-XL training dataset. According to the paper, this repartitioning improves class coherence for geographically proximate locations, reduces redundancy, increases robustness to viewpoint changes, and decreases training data by approximately $25\%$ to $32\%$ relative to prior SF-XL partitions, with “approximately $30\%$” emphasized in the abstract (Jiang et al., 24 Jul 2025).

3. Architectural formulation

The baseline backbone is a pretrained ResNet-50 truncated at the final two layers. All residual blocks except the last two are frozen during training, and linear projections standardize channel dimensions before attention (Jiang et al., 24 Jul 2025). The two resulting feature maps are denoted $f_1 \in \mathbb{R}^{C_1 \times H_1 \times W_1}$ and $f_2 \in \mathbb{R}^{C_2 \times H_2 \times W_2}$, with token counts $N_1 = H_1 W_1$ and $N_2 = H_2 W_2$ after flattening. Positional encodings are then added to preserve geometry.

Within each scale, DSFormer uses Transformer encoders with pre-normalization and residual connections:

$$z_t = \mathrm{MHA}(\mathrm{LN}(z_{l-1})) + z_{l-1}, \qquad z_l = \mathrm{FFN}(\mathrm{LN}(z_t)) + z_t.$$

The self-attention is augmented with IRPE (Improving Relative Position Encoding):

$$\mathrm{Attention}_{\mathrm{IRPE}}(q,k,v) = rpe_v\!\left( \mathrm{softmax}\!\left(\frac{qk^\top}{\sqrt{d_k}} + rpe_q(q) + rpe_k(k)\right) \right)v.$$

This adds relative-position terms to the logits and modulates outputs with learnable relative position functions, improving spatial coherence and long-range reasoning (Jiang et al., 24 Jul 2025).

The distinctive cross-scale mechanism is a shared-parameter multi-head cross-attention applied in both directions. Let $z_i \in \mathbb{R}^{N_i \times C}$ denote the token sequence of scale $32\%$0. Cross-attention from scale $32\%$1 to scale $32\%$2, with $32\%$3, is:

$32\%$4

$32\%$5

The same cross-attention parameters are shared across the two directions, encouraging symmetric alignment between scales rather than independent one-way fusion (Jiang et al., 24 Jul 2025).

After $32\%$6 encoder layers, the fused dual-scale features are aggregated with GeM pooling and projected to a 512-dimensional descriptor, followed by $32\%$7 normalization:

$32\%$8

In the ResNet-50 setting, DSFormer uses $32\%$9 encoder layers and 16 heads; in the DINOv2 setting, it uses $30\%$0 encoder layer with 12 shared heads, fine-tuning the final two layers of DINOv2 (ViT-B) at input resolution $30\%$1 (Jiang et al., 24 Jul 2025).

The paper gives the per-layer attention complexity as

$30\%$2

combining two intra-scale self-attentions and two directional cross-attentions. The stated mitigation is that attention operates on CNN feature maps rather than raw pixels, so token counts remain practical (Jiang et al., 24 Jul 2025).

4. Block clustering strategy and optimization

The block clustering strategy restructures SF-XL into density-based, geographically coherent classes using UTM coordinates. For east-coordinate grouping, the initial assignment is

$30\%$3

with block width $30\%$4 m and $30\%$5 groups per direction (Jiang et al., 24 Jul 2025). HDBSCAN is then applied within each group to form density-based clusters, after which KNN retains only samples within radius $30\%$6 m of the peak-density location. Classes whose peak density locations are closer than $30\%$7 m are excluded to prevent overlaps. The same procedure is repeated for northing.

The east and north groups are merged as

$30\%$8

Finally, viewpoint robustness is improved through perspective selection. For each retained class with coordinates $30\%$9, the centered coordinates are decomposed by SVD, and focal points are defined as

$f_1 \in \mathbb{R}^{C_1 \times H_1 \times W_1}$0

with $f_1 \in \mathbb{R}^{C_1 \times H_1 \times W_1}$1 m. Splitting classes by the two principal directions doubles the groups to $f_1 \in \mathbb{R}^{C_1 \times H_1 \times W_1}$2 (Jiang et al., 24 Jul 2025). The resulting processed SF-XL training set contains approximately $f_1 \in \mathbb{R}^{C_1 \times H_1 \times W_1}$3 million images across 10 groups, compared with approximately $f_1 \in \mathbb{R}^{C_1 \times H_1 \times W_1}$4 million for CosPlace and approximately $f_1 \in \mathbb{R}^{C_1 \times H_1 \times W_1}$5 million for EigenPlaces partitions (Jiang et al., 24 Jul 2025).

Training follows EigenPlaces and uses Large Margin Cosine Loss (LMCL, CosFace):

$f_1 \in \mathbb{R}^{C_1 \times H_1 \times W_1}$6

The optimizer is Adam with learning rate $f_1 \in \mathbb{R}^{C_1 \times H_1 \times W_1}$7 for the main model and a separate Adam optimizer with learning rate $f_1 \in \mathbb{R}^{C_1 \times H_1 \times W_1}$8 for the classifier. Training runs for up to 40 epochs on an NVIDIA RTX 4080 (16 GB), with batch size 32 and 10,000 iterations per group per epoch. Inputs are resized to $f_1 \in \mathbb{R}^{C_1 \times H_1 \times W_1}$9 with random scaling and color jittering for the ResNet-50 configuration (Jiang et al., 24 Jul 2025).

5. Empirical performance and efficiency

The evaluation protocol uses single-stage global retrieval with 512-dimensional descriptors and no reranking. Success for Recall@$f_2 \in \mathbb{R}^{C_2 \times H_2 \times W_2}$0 is defined by a match within 25 m on MSLS Val, MSLS Challenge, Pittsburgh30k, Tokyo 24/7, Nordland, and large-scale SF-XL test subsets (Jiang et al., 24 Jul 2025). In the ResNet-50 setting, DSFormer reports average Recall@1 of 83.8 across five benchmarks, compared with 83.3 for R2Former, 78.1 for TransVPR, 76.7 for DELG, and 72.8 for Patch-NetVLAD. Its per-dataset Recall@1 values are 88.9 on MSLS Val, 68.1 on MSLS Challenge, 91.9 on Pitts30k, 88.6 on Tokyo24/7, and 81.5 on Nordland (Jiang et al., 24 Jul 2025).

Against single-stage retrieval baselines using ResNet-50 and 512-dimensional descriptors, the reported averages are 77.5 for CosPlace, 77.9 for EigenPlaces, 76.8 for MixVPR, 78.6 for BoQ+PCA, 80.6 for GeM† trained on the block-clustered partition, and 83.8 for DSFormer (Jiang et al., 24 Jul 2025). The paper explicitly states that DSFormer improves over EigenPlaces by $f_2 \in \mathbb{R}^{C_2 \times H_2 \times W_2}$1 Recall@1 points on average and over GeM† by $f_2 \in \mathbb{R}^{C_2 \times H_2 \times W_2}$2, with especially strong gains on Tokyo24/7 and Nordland.

The DINOv2 configuration raises the average Recall@1 to 89.3, compared with 86.3 for SALAD+PCA, 86.3 for BoQ+PCA, 84.8 for CricaVPR+PCA, and 88.4 for GeM† with DINOv2 (Jiang et al., 24 Jul 2025). On the large-scale SF-XL test subsets, the ResNet-50 model reports Recall@1 values of 85.7 on v1, 91.3 on v2, 31.5 on Night, and 42.1 on Occlusion, averaging 62.7, while the DINOv2 model reaches 93.7, 94.8, 54.1, and 52.6 respectively, averaging 73.8 (Jiang et al., 24 Jul 2025).

Efficiency is a central empirical claim. On MSLS Val using an RTX 3060, DSFormer uses a descriptor of size $f_2 \in \mathbb{R}^{C_2 \times H_2 \times W_2}$3, memory of 0.039 GB, and extraction time of 0.024 s/query, with no reranking (Jiang et al., 24 Jul 2025). Under the same comparison, R2Former uses 5.2 GB and 0.423 s/query for reranking, TransVPR uses 24.2 GB and 2.118 s/query for reranking, Patch-NetVLAD uses 908.3 GB and 16.88 s/query for reranking, and DELG uses 7.8 GB and 35.8 s/query for reranking. The paper characterizes this as substantially lower memory and latency than two-stage reranking pipelines (Jiang et al., 24 Jul 2025).

Ablation studies attribute the gains to both the architecture and the data partition. With a ResNet-50 backbone, the number of DSFormer layers was varied from 0 to 4; three layers gave the best overall trade-off across MSLS Val, Pitts30k, Tokyo24/7, and Nordland. Removing IRPE, self-attention, cross-attention, or block clustering consistently reduced Recall@1, with block clustering showing a notable effect on Nordland (Jiang et al., 24 Jul 2025).

6. Limitations, interpretations, and related models

The paper notes that DSFormer still faces difficulty under extreme domain shifts, especially the Night and Occlusion subsets, although the DINOv2-based configuration reduces these gaps substantially (Jiang et al., 24 Jul 2025). It also states that the method assumes dual-scale CNN features are sufficient; for ultra-wide viewpoint changes or sparse visual environments, additional cues such as depth or multi-view temporal context could help. The block clustering procedure relies on heuristic parameters $f_2 \in \mathbb{R}^{C_2 \times H_2 \times W_2}$4, $f_2 \in \mathbb{R}^{C_2 \times H_2 \times W_2}$5, $f_2 \in \mathbb{R}^{C_2 \times H_2 \times W_2}$6, and $f_2 \in \mathbb{R}^{C_2 \times H_2 \times W_2}$7, suggesting a dependence on manual tuning even though the reported empirical results are strong (Jiang et al., 24 Jul 2025).

Future directions proposed in the paper include cross-domain pretraining, integrating geometric priors, scaling DSFormer layers with dynamic token pruning for faster inference, and coupling global descriptors with lightweight local verification when needed (Jiang et al., 24 Jul 2025). This suggests that the model is best understood not as a rejection of reranking or local verification in principle, but as a strong global retrieval backbone that already attains high accuracy without them.

The broader literature also makes clear that “DSFormer” should not be treated as a single architecture family across domains. In multispectral object detection, the term refers to a Dual Sparse Transformer inside DSAFormer, with spatial sparse multi-head cross-attention, channel sparse multi-head cross-attention, and a multi-scale feature refinement layer; that paper explicitly distinguishes its terminology from “Dual-Scale-Former” (Wu et al., 30 Jun 2026). In multivariate time-series forecasting, DSformer denotes a Double Sampling Transformer that uses down sampling, piecewise sampling, and temporal-variable attention for long-term prediction (Yu et al., 2023). Consequently, in encyclopedic or bibliographic usage, “Dual-Scale-Former (DSFormer)” is most precisely reserved for the VPR model built around dual-scale cross-learning and block-clustered SF-XL training (Jiang et al., 24 Jul 2025).