MVL-Loc: Multi-Scene 6-DoF Relocalization

• MVL-Loc is a multi-scene 6-DoF camera relocalization framework that integrates vision-language pretrained models with natural language directives.
• It employs transformer-based multimodal fusion to align visual features with scene-specific language cues for accurate pose regression.
• MVL-Loc demonstrates improved localization performance on benchmarks like 7Scenes and Cambridge, reducing median errors in both position and orientation.

Searching arXiv for the exact MVL-Loc paper and closely related localization papers to ground the article in current literature. MVL-Loc is a multi-scene, end-to-end $6$-DoF camera relocalization framework that uses pretrained vision-language world knowledge, multimodal fusion of image and text, and scene-specific natural-language directives to estimate camera position and orientation from a single RGB image across both indoor and outdoor scenes. It is positioned against the scene specificity of earlier deep pose regressors and against the storage and correspondence costs of classical geometric relocalization, and it reports average median errors of $0.16$ m and $6.98^\circ$ on 7Scenes and $0.93$ m and $2.90^\circ$ on Cambridge Landmarks (Xiao et al., 6 Jul 2025).

1. Problem setting and motivation

MVL-Loc addresses camera relocalization in the single-image setting, where the target output is a $6$-DoF pose composed of $3$D position and $3$D orientation. In the formulation used by the method, this is the familiar relocalization problem faced in AR, MR, autonomous driving, delivery drones, and robotic navigation. The paper frames the central deficiency of prior deep pose regression methods as a generalization problem: most such models are trained per scene, learn scene-specific appearance-to-pose mappings, and degrade sharply when transferred to new scenes or when viewpoint, illumination, seasonal conditions, or dynamic clutter change substantially (Xiao et al., 6 Jul 2025).

The method is motivated by a contrast between two established families of relocalizers. Traditional geometric systems recover pose through explicit 2D-3D correspondence, PnP, Kabsch, and RANSAC, but remain storage-heavy and computationally expensive because they require maps and explicit matching. By contrast, deep pose regression methods such as PoseNet, AtLoc, and EffLoc remove explicit map construction, yet often memorize a scene’s layout rather than learning a transferable spatial prior. MVL-Loc adopts the multi-scene setting as the more realistic operating regime: one model must localize across several distinct environments, each with its own geometry and appearance statistics. The paper further argues that prior multi-scene systems such as MSPN, MS-Trans, and C2f-MS-Transformer improve scalability but still lack a strong semantic prior for robust cross-scene generalization (Xiao et al., 6 Jul 2025).

2. Architecture and estimation pipeline

The architecture combines a pretrained vision-language encoder, transformer-based multimodal fusion, scene classification, and scene-specific pose regression. The paper identifies five constituent elements: a pretrained vision-LLM backbone, multimodal fusion of image and text, scene-specific natural-language descriptions that act as directives, multi-scene training with a scene classifier, and final pose regression from fused features. CLIP is the best-performing backbone in the reported experiments, while BLIP-2 and OpenFlamingo are included in encoder ablations (Xiao et al., 6 Jul 2025).

The pipeline has two major stages: language-guided text-to-scene correspondence generation and multi-scene pose regression. Given an image $I \in \mathbb{R}^{C \times H \times W}$, a 2D convolutional encoder extracts localized visual features, while token embeddings are extracted from a natural-language prompt $T$. The multimodal core is a stack of decoder-like Transformer layers with self-attention, multi-head attention, feedforward blocks, residual connections, and layer normalization. Fusion is expressed as a cross-modal encoder driven by dot-product interaction,

$0.16$0

so that the language representation functions as a query over the visual scene and amplifies semantically relevant and geometrically informative features (Xiao et al., 6 Jul 2025).

Because the model is trained across multiple scenes, relocalization is scene-indexed. The network first predicts scene membership, then applies scene-specific MLP heads to regress pose: $0.16$1 where $0.16$2 is position and $0.16$3 is a unit quaternion for orientation. The selected output corresponds to the scene with maximal predicted probability. The multi-scene objective combines classification and pose regression,

$0.16$4

with learned scale factors $0.16$5 and $0.16$6 balancing translation and rotation. The orientation term uses the logarithm of a unit quaternion, and the method restricts quaternions to one hemisphere using absolute values because $0.16$7 and $0.16$8 encode the same rotation (Xiao et al., 6 Jul 2025).

3. Natural language as a directive prior

A defining feature of MVL-Loc is the use of natural language not as an auxiliary caption but as a directive that shapes cross-modal attention. The paper explicitly distinguishes broad prompts from detailed prompts. A description such as “a chessboard in a classroom” is said to spread attention too widely, whereas “a chessboard on a small table surrounded by chairs” concentrates the model on salient objects and their spatial arrangement. The same logic is illustrated by prompts such as “two monitors side by side on a cluttered desk with a chair in front.” The intended effect is to bind linguistic relations such as “in front of,” “surrounded by,” “side by side,” and “on a cluttered desk” to stable visual structure (Xiao et al., 6 Jul 2025).

This mechanism is presented as a way to inject pretrained world knowledge into relocalization. In indoor scenes, objects such as chairs, monitors, desks, and chessboards serve as stable layout cues; in outdoor scenes, spires, arches, and facades serve an analogous role. The paper’s qualitative visualizations report that detailed language produces sharper attention maps and saliency concentrated on geometrically meaningful structures rather than dynamic or cluttered regions. It also reports that, in Cambridge, MVL-Loc focuses more on static architectural elements and less on pedestrians and cars than MS-Trans. The broader claim is that semantic understanding of object identity and spatial relations improves transfer across distinct scenes because the model is not relying on appearance alone (Xiao et al., 6 Jul 2025).

4. Training protocol and ablation structure

The training setup is fully specified in the paper. Images are resized or cropped to $0.16$9. Optimization is performed in PyTorch $6.98^\circ$0 on an Nvidia V100 for $6.98^\circ$1 epochs using AdamW with a cosine learning-rate schedule, initial learning rate $6.98^\circ$2, weight decay $6.98^\circ$3, batch size $6.98^\circ$4, and dropout $6.98^\circ$5. Data augmentation uses ColorJitter with brightness $6.98^\circ$6, contrast $6.98^\circ$7, saturation $6.98^\circ$8, and hue $6.98^\circ$9. The learned scale factors are initialized at $0.93$0 and $0.93$1 (Xiao et al., 6 Jul 2025).

The ablations are structured around three questions: the contribution of pretrained world knowledge, the incremental effect of language descriptions, and the role of multi-scene training. On 7Scenes, the sequence of results progresses from ImageNet pretraining only at $0.93$2 m and $0.93$3, to CLIP world knowledge only at $0.93$4 m and $0.93$5, to CLIP plus language descriptions at $0.93$6 m and $0.93$7, and finally to CLIP plus language plus multi-scene training at $0.93$8 m and $0.93$9. On Cambridge, the corresponding progression is $2.90^\circ$0 m and $2.90^\circ$1, $2.90^\circ$2 m and $2.90^\circ$3, $2.90^\circ$4 m and $2.90^\circ$5, and $2.90^\circ$6 m and $2.90^\circ$7. The encoder comparison reports BLIP-2 at $2.90^\circ$8 m and $2.90^\circ$9 on 7Scenes and $6$0 m and $6$1 on Cambridge, OpenFlamingo at $6$2 m and $6$3 and $6$4 m and $6$5, and CLIP at $6$6 m and $6$7 and $6$8 m and $6$9. Transformer-depth ablations show that $3$0 decoder layers are the best efficiency–performance tradeoff; $3$1 layers perform poorly, while deeper alternatives provide no better reported average accuracy and incur higher cost (Xiao et al., 6 Jul 2025).

5. Benchmarks and reported performance

The evaluation uses two standard relocalization benchmarks. The 7Scenes dataset is an indoor RGB-D benchmark with Kinect-captured sequences in small office-like environments, each around $3$2–$3$3 m$3$4, and includes Chess, Fire, Heads, Office, Pumpkin, Kitchen, and Stairs. Cambridge Landmarks is an outdoor benchmark with smartphone images and SfM-generated poses. The reported comparison uses four Cambridge scenes—King’s College, Old Hospital, Shop Façade, and St Mary’s Church—and explicitly excludes Great Court and Street because some baselines failed to converge there. The primary metrics are median position error in meters and median rotation error in degrees, averaged across scenes (Xiao et al., 6 Jul 2025).

Benchmark	Scenes used in the reported comparison	MVL-Loc average median error
7Scenes	Chess, Fire, Heads, Office, Pumpkin, Kitchen, Stairs	$3$5 m, $3$6
Cambridge Landmarks	King’s College, Old Hospital, Shop Façade, St Mary’s Church	$3$7 m, $3$8

On 7Scenes, the reported averages are PoseNet at $3$9 m and $3$0, AtLoc at $3$1 m and $3$2, MSPN at $3$3 m and $3$4, MS-Trans at $3$5 m and $3$6, C2f-MS-Trans at $3$7 m and $3$8, and MVL-Loc at $3$9 m and $I \in \mathbb{R}^{C \times H \times W}$0. Scene-level MVL-Loc results are reported as $I \in \mathbb{R}^{C \times H \times W}$1 m and $I \in \mathbb{R}^{C \times H \times W}$2 on Chess, $I \in \mathbb{R}^{C \times H \times W}$3 m and $I \in \mathbb{R}^{C \times H \times W}$4 on Fire, $I \in \mathbb{R}^{C \times H \times W}$5 m and $I \in \mathbb{R}^{C \times H \times W}$6 on Heads, $I \in \mathbb{R}^{C \times H \times W}$7 m and $I \in \mathbb{R}^{C \times H \times W}$8 on Office, $I \in \mathbb{R}^{C \times H \times W}$9 m and $T$0 on Pumpkin, $T$1 m and $T$2 on Kitchen, and $T$3 m and $T$4 on Stairs. The paper states that MVL-Loc improves over AtLoc by $T$5 in position and $T$6 in rotation on 7Scenes, and outperforms MS-Trans by about $T$7 in position and $T$8 in rotation (Xiao et al., 6 Jul 2025).

On Cambridge Landmarks, the reported averages are PoseNet at $T$9 m and $0.16$00, BayesianPoseNet at $0.16$01 m and $0.16$02, MapNet at $0.16$03 m and $0.16$04, PoseNet17 at $0.16$05 m and $0.16$06, IRPNet at $0.16$07 m and $0.16$08, PoseNet-Lstm at $0.16$09 m and $0.16$10, MSPN at $0.16$11 m and $0.16$12, MS-Trans at $0.16$13 m and $0.16$14, C2f-MS-Trans at $0.16$15 m and $0.16$16, and MVL-Loc at $0.16$17 m and $0.16$18. Per-scene results for MVL-Loc are $0.16$19 m and $0.16$20 on King’s College, $0.16$21 m and $0.16$22 on Old Hospital, $0.16$23 m and $0.16$24 on Shop Façade, and $0.16$25 m and $0.16$26 on St Mary’s Church. The paper highlights gains over PoseNet-LSTM on Shop Façade and St Mary’s Church of $0.16$27 and $0.16$28, respectively (Xiao et al., 6 Jul 2025).

6. Scope, limitations, and nomenclature

MVL-Loc belongs specifically to the literature on multi-scene camera relocalization with language-guided multimodal fusion. It should be distinguished from several similarly named or easily conflated localization systems in adjacent subareas. “LiteVLoc” is a hierarchical visual localization and image-goal navigation system built around a lightweight topo-metric map rather than dense $0.16$29D scene reconstruction (Jiao et al., 2024). “VMLoc” is a variational RGB-D / RGB-LiDAR pose regressor based on Product-of-Experts latent fusion and attention-based multimodal fusion (Zhou et al., 2020). “VLM-GLoc” is a hierarchical semantic Monte Carlo Localization system that uses open-vocabulary vision-language observations inside a particle filter for global localization in cluttered quasi-static environments (Agrawal et al., 28 May 2026). The overlap is therefore lexical rather than methodological.

The limitations reported or implied for MVL-Loc are specific to its formulation. The method still relies on scene-indexed prompts and multi-scene supervision, so it is not fully open-vocabulary localization. It uses a scene classifier, which means incorrect scene selection can affect the final pose. The paper does not report large-scale real-time efficiency or latency, and the transformer-depth ablation indicates nontrivial compute cost. Prompt quality appears consequential because the method contrasts broad and detailed language and attributes some gains to sharper directive descriptions. The evaluation is limited to 7Scenes and a subset of Cambridge Landmarks. At the same time, the paper’s central implication is that semantic grounding can function as a structural prior for geometry: localization can improve when object identity and spatial language are coupled to pose regression rather than treated as peripheral metadata (Xiao et al., 6 Jul 2025).