AsymLoc: Towards Asymmetric Feature Matching for Efficient Visual Localization

Abstract: Precise and real-time visual localization is critical for applications like AR/VR and robotics, especially on resource-constrained edge devices such as smart glasses, where battery life and heat dissipation can be a primary concerns. While many efficient models exist, further reducing compute without sacrificing accuracy is essential for practical deployment. To address this, we propose asymmetric visual localization: a large Teacher model processes pre-mapped database images offline, while a lightweight Student model processes the query image online. This creates a challenge in matching features from two different models without resorting to heavy, learned matchers. We introduce AsymLoc, a novel distillation framework that aligns a Student to its Teacher through a combination of a geometry-driven matching objective and a joint detector-descriptor distillation objective, enabling fast, parameter-less nearest-neighbor matching. Extensive experiments on HPatches, ScanNet, IMC2022, and Aachen show that AsymLoc achieves up to 95% of the teacher's localization accuracy using an order of magnitude smaller models, significantly outperforming existing baselines and establishing a new state-of-the-art efficiency-accuracy trade-off.

• The paper introduces an asymmetric training method that employs a high-capacity teacher for offline processing and a compact student for real-time localization.
• It utilizes geometric matching loss and joint detector–descriptor distillation to bridge feature compatibility, achieving near-teacher accuracy with 25× fewer parameters.
• Experimental benchmarks demonstrate up to 7× faster FPS on edge devices while maintaining robust performance across diverse datasets like HPatches and Aachen.

AsymLoc: Asymmetric Feature Matching for Efficient Visual Localization

Overview

AsymLoc is a novel visual localization framework that explicitly models the asymmetry between compute-rich offline and compute-constrained online localization contexts. By using a parameter-efficient student model for real-time query processing and a high-capacity teacher model for offline database pre-processing, AsymLoc bridges the feature compatibility gap between heterogeneous models through a joint geometry-driven and probabilistic distillation regime. The approach enables real-time, on-device localization performance that is near that of powerful teacher models, but with up to $25\times$ reduced model size and substantially lower inference cost.

Figure 1: The AsymLoc framework enables compact student models to perform high-accuracy localization on edge platforms by leveraging teacher-derived offline database features.

Problem Setting and Methodology

Motivation and Asymmetric Pipeline Formulation

Classical visual localization pipelines rely on symmetric feature extractors (same architecture for both database and query), which limits deployability on resource-constrained devices (smart glasses, small drones, embedded robotics). AsymLoc adopts an asymmetric architecture: the teacher model extracts robust keypoints and descriptors from database images in an unconstrained offline setting, while the student model extracts lightweight features from queries in real-time.

The central challenge is matching features across heterogeneous teacher and student representations without resorting to parameter-heavy learned matchers. Naive distillation strategies or pure feature regression yield poor cross-model compatibility. AsymLoc addresses this via two complementary objectives:

1. Geometric Matching Loss: Enforces student-teacher correspondence consistency using soft, detector-aware matches based on ground truth homographies.
2. Joint Detector–Descriptor Distillation Loss: Aligns the student’s joint detection-confidence-modulated descriptor similarity distribution with that of the teacher, using KL divergence over the induced matching distributions.

   Figure 2: AsymLoc training pipeline with geometric and probabilistic alignment in joint detector–descriptor space between student and teacher models.

Technical Contributions

Geometric Consistency Enforcement

Given pairs of images related by known homographies, AsymLoc computes mutual matching matrices weighted by both detector confidences and softmax-normalized descriptor similarities. The geometric loss is applied only to high-confidence teacher keypoints, focusing the learning signal where the teacher is most reliable.

Joint Detector–Descriptor Distillation

The innovation lies in distilling the probabilistic structure of joint detector-descriptor interactions. Rather than aligning raw features, the approach transfers the teacher’s soft matchability distribution (considering both detection reliability and descriptor similarity) to the student, using Kullback–Leibler divergence as the alignment term. This ensures that the student's feature interactions remain compatible with the teacher for matching, enabling parameterless mutual nearest-neighbor matching at inference.

Full Objective

The overall loss is a weighted sum of the geometric matching loss and joint detector–descriptor distillation, with the balance empirically tuned for optimal localization accuracy.

Experimental Results

Evaluation Benchmarks

AsymLoc is benchmarked on four diverse datasets: HPatches (planar homography), ScanNet (indoor relative pose), IMC2022/Aachen (large-scale outdoor visual localization), and MegaDepth (pose estimation). Both SiLK and SuperPoint are used as teacher models, covering a wide range of detector–descriptor architectures.

Figure 3: Evaluation includes planar, indoor, and outdoor benchmarks, capturing a broad spectrum of localization challenges.

Asymmetric Matching Performance

AsymLoc-trained student models, with as few as $0.04$M parameters (vs. $1$M for the teacher), achieve $95\%$ teacher-level accuracy across all benchmarks. The framework decisively outperforms symmetric training (standard or naive distillation), as well as asymmetric distillation strategies adapted from retrieval (AML, RKD, CSD, D3Still) in both matching accuracy and parameter efficiency.

Figure 4: Asymmetric student–teacher matching in AsymLoc preserves teacher-level correspondences, while symmetric student–student matching fails.

Figure 5: Across multiple benchmarks, AsymLoc yields a superior efficiency–accuracy trade-off, with flatter Pareto frontiers and higher parameter efficiency than symmetric training.

Notably, standard downsizing of feature extractors leads to rapid performance degradation, but AsymLoc's asymmetric training stabilizes this decline, enabling much smaller online models without substantial accuracy loss.

Figure 6: Homography estimation accuracy on HPatches reveals that AsymLoc models retain high accuracy at dramatically reduced student sizes.

Latency and Edge Deployment

Detailed latency-vs-accuracy analyses demonstrate AsymLoc achieves up to $7\times$ higher FPS on GPU for equivalent accuracy bands, exceeding the real-time requirements for edge devices.

Figure 7: AsymLoc delivers strong FPS/accuracy trade-offs for real-time edge deployment on HPatches.

Ablation and Theoretical Implications

Ablation studies indicate that joint detector–descriptor distillation is critical, providing most of the cross-model compatibility gain, while geometric matching loss provides regularization for correspondence reliability. The asymmetric feature matching paradigm validates and extends findings from asymmetric retrieval, highlighting the unique importance of aligning local feature interaction distributions (rather than global descriptors or individual features).

Practical and Theoretical Implications

By decoupling database and query model complexity, AsymLoc establishes a scalable architectural template for edge-device localization. Resource-constrained platforms can leverage offline-generated, teacher-derived map features without sacrificing localization robustness. The methodology generalizes across detectors/descriptors (SiLK, SuperPoint, XFeat) and is robust to large parameter mismatches between teacher and student. Theoretically, AsymLoc’s probabilistic cross-model matching regularization could inspire broader asymmetric architectures in vision and multi-modal learning.

On-device AR systems, autonomous UAV navigation, and robotics can now deploy ultra-lightweight visual localization pipelines with minimal compute and energy footprint, removing a longstanding bottleneck in real-world scalable localization.

Conclusion

AsymLoc introduces a principled, effective solution for asymmetric feature matching that allows lightweight student models to maintain near-teacher localization accuracy with orders-of-magnitude reductions in parameters and flops. By combining geometry-aware objective design and joint detector–descriptor probabilistic distillation, AsymLoc advances the state of the art in efficient visual localization and sets a foundation for the next generation of deployable, edge-friendly computer vision systems.