Image-Based Localization in GNSS-Denied Areas

• Image-based localization is a method that fuses visual, depth, and radar data with geo-referenced maps to accurately determine vehicle or robot positions in challenging, GNSS-denied environments.
• It combines multi-sensor fusion, semantic segmentation, and learned embeddings to deliver robust, drift-corrected localization through techniques like particle filtering and graph optimization.
• Practical applications span UGVs, UAVs, maritime vessels, and indoor systems, providing reliable navigation in urban canyons, tunnels, and off-road scenarios.

Image-Based Localization in GNSS-Denied Environments

Image-based localization in GNSS-denied environments is a class of methodologies that leverage local visual, depth, or radar sensing to estimate the global pose of vehicles or robots by associating onboard perceptions with geo-referenced satellite or aerial maps. These approaches are necessary in scenarios where GNSS signals are unreliable, jammed, or unavailable, including dense urban canyons, tunnels, hazardous environments, and off-road or maritime locations. Recent research has converged on multistage pipelines combining learned representations, semantic segmentation, sensor fusion, and particle or graph-based filtering to achieve robust, drift-corrected localization under challenging operational conditions.

1. Architectural Principles and Sensing Modalities

Localization pipelines for GNSS-denied environments are structured around the fusion of onboard sensors—visual cameras (RGB, infrared, or thermal), LiDAR, radar, IMU, and occasionally UWB transceivers—with pre-existing geo-referenced map data. Architectures typically exploit the following:

• Multi-sensor fusion: LiDAR–camera fusion (e.g., BEV perception (Sun et al., 23 Apr 2025)), thermal–LiDAR fusion for tunnel scenarios (Schichler et al., 6 May 2025), or visual–IMU–UWB integration (Shi et al., 2019) enables cue redundancy and complements weaknesses of individual modalities.
• Bird’s Eye View (BEV) and semantic abstraction: Transforming local sensor data into a top-down BEV image mitigates cross-view discrepancies and enables template or feature matching with satellite/aerial imagery (Jin et al., 14 May 2024, Sun et al., 23 Apr 2025).
• Semantic segmentation: High-level classes (road, building, vegetation, water) abstract away seasonal and illumination changes, providing stable matching primitives (Yuan et al., 17 Sep 2025).
• Visual place recognition (VPR): Foundation model-based descriptors (e.g., DINO/ViT) and compact NetVLAD global descriptors anchor odometry (He et al., 2023, Zhang et al., 29 Nov 2024).

These sensor fusion strategies are modular and extendable depending on vehicle type—UGV, UAV, marine USV, or pedestrian systems.

2. Map Association and Feature Spaces

A central challenge is bridging the large domain gap between onboard sensor perspectives and global maps. Key representations and algorithms include:

• Semantic road similarity spaces: BEV/satellite images are embedded via encoder–decoder networks into per-pixel feature tensors, which are then aggregated and compared (max-cosine similarity, normalized cross-correlation) (Sun et al., 23 Apr 2025).
• Occupancy maps from overhead RGB: Attention U-nets predict spatial occupancy from satellite images, supporting ICP-based association with ground radar data (RaSCL) (Abdullai et al., 22 Apr 2025).
• Ratio-based descriptors: Building Ratio Map (BRM) localization computes rotation-invariant area ratios in concentric regions, matched globally to numerical cadastral maps (Choi et al., 2020).
• Learned cross-view embeddings: Siamese CNNs learn location-discriminative representations across ground/satellite domains, robust to viewpoint and appearance shifts (Kim et al., 2017, Kinnari et al., 2021).
• Monocular depth–semantic fusion: Visual Map Registration (VMR) leverages deep metric depth estimation, semantic filtering for static content, and generalized ICP for 2D–3D alignment (Elmaghraby et al., 24 Jun 2025).

These representations enable rapid, scalable global map queries necessary for correcting odometric drift.

3. Matching, Filtering, and Optimization Algorithms

Robust global localization is achieved by embedding map association in probabilistic filtering and optimization frameworks:

• Particle filters (Monte Carlo Localization): Particles represent hypotheses of vehicle pose and are propagated via motion/odometry models. Image–map match scores (NCC, Euclidean embedding distance, semantic-weighted likelihoods) update particle weights (Sun et al., 23 Apr 2025, Yuan et al., 17 Sep 2025, Kim et al., 2017, Jurevičius et al., 2019).
• Extended Kalman Filters and factor graphs: EKFs fuse continuous odometry (LiDAR, VIO, optical-flow) with discrete, intermittent absolute pose corrections from image-to-map matches (Schichler et al., 6 May 2025, Zhang et al., 29 Nov 2024, Elmaghraby et al., 24 Jun 2025). Factor graph optimization solves sliding-window pose graphs with both odometric and map measurement constraints (RaSCL (Abdullai et al., 22 Apr 2025), FoundLoc (He et al., 2023)).
• Discrete candidate pruning and continuous optimization: BRM methods maintain candidate sets over large map extents, pruned via matching error thresholds and refined via nonlinear least-squares (Choi et al., 2020).

Pseudomeasurement strategies and systematic resampling mitigate weight degeneracy and ensure convergence to the true pose.

4. Quantitative Evaluation and Performance Metrics

Performance is validated using ground truth (GNSS/INS, RTK-GPS, high-resolution SLAM) and standard error metrics:

Pipeline	Error (m)	Notable Conditions	Reference
Road similarity BEV–satellite	0.89 (lateral), 3.41 (planar)	10 km off-road, night robustness	(Sun et al., 23 Apr 2025)
Semantic-weighted particle PF	6.57 (RMSE), 97% @10m recall	4D (3D + yaw), multi-altitude	(Yuan et al., 17 Sep 2025)
Radar-to-satellite ICP FG	1.3–4.5 (trajectories)	Urban, suburban, marine, multi-modal	(Abdullai et al., 22 Apr 2025)
BEVRender	19–22 (APE), 57–63% match rate	Off-road, 3 Hz runtime	(Jin et al., 14 May 2024)
BRM (building ratio map)	7.53–12.01 (RMSE)	Full-trajectory UAV, unknown start	(Choi et al., 2020)
Monocular VMR + semantic	0.98 (RMSE), 92% <1m	Urban canyons/indoors, lane-level	(Elmaghraby et al., 24 Jun 2025)
Visual-UWB SLAM	0.036 (ATE)	Centimeter, metric scale	(Shi et al., 2019)

Localization pipelines robustly reduce odometric drift and maintain meter-level or sub-meter accuracy under appearance variance, viewpoint changes, and seasonal transitions.

5. Limitations, Failure Modes, and Domain-Specific Issues

Despite significant progress, several limitations constrain the applicability and accuracy of image-based localization systems:

• Feature sparsity and homogeneous terrain: Lack of roads, buildings, or distinctive vegetation in satellite imagery degrades semantic matching (Sun et al., 23 Apr 2025, Choi et al., 2020, Jin et al., 14 May 2024).
• Straight trails and longitudinal drift: Absence of discriminative features along extended straight paths increases uncertainty in the trajectory (Sun et al., 23 Apr 2025).
• Perspective and map currency mismatches: Oblique views, seasonal changes, and outdated or misaligned maps lower matching confidence and can degrade localization by 30–50% (Kinnari et al., 2021, Jurevičius et al., 2019).
• Planarity and altitude change assumptions: Orthorectification errors rise in undulating terrain, or under large altitude changes for UAVs/UGVs (Kinnari et al., 2021, Choi et al., 2020).
• Dynamic and occlusive environments: Urban canyons and tunnels challenge feature extraction and matching; fusion of thermal/optical/LiDAR modalities or opportunistic beacons can mitigate these conditions (Schichler et al., 6 May 2025, Zhang et al., 29 Nov 2024).
• Initialization and convergence requirements: Some pipelines require hundreds of meters of motion to converge if initial position is unknown or ambiguous (Choi et al., 2020, Kinnari et al., 2021).

These limitations motivate the use of multi-modal sensor fusion, semantic generalization, and adaptive matching strategies.

6. Practical Applications and Adaptation to Platform Domains

Image-based localization systems have been demonstrated for:

• UGVs in off-road, agricultural, and suburban environments: BEV similarity matching, LiDAR-camera fusion, global satellite map registration (Sun et al., 23 Apr 2025, Jin et al., 14 May 2024).
• UAVs in urban, rural, and maritime settings: Semantic-weighted matching, VPR, visual-inertial odometry, and multi-altitude datasets (Yuan et al., 17 Sep 2025, He et al., 2023, Choi et al., 2020).
• Autonomous surface vessels (USVs): Radar-to-satellite registration, factor graph smoothing, and occupancy mapping (Abdullai et al., 22 Apr 2025).
• Tunnels and perceptually degraded spaces: Thermal-LiDAR EKF fusion enables robust, sub-meter tracking against strong drift (Schichler et al., 6 May 2025).
• Indoor urban pedestrian navigation: Opportunistic visual beacon fusion with dead-reckoning in Kalman filters yields >40% accuracy improvement (Zhang et al., 29 Nov 2024).
• GNSS-denied parking structures and urban canyons: Monocular metric depth + semantic filtering with 3D digital maps delivers >80% sub-meter accuracy (Elmaghraby et al., 24 Jun 2025).

Memory-efficient implementations, GPU-accelerated semantic segmentation, and cross-view training ensure operational feasibility on embedded and mobile platforms.

7. Future Research Directions

Recent studies identify the following avenues for continued advancement:

• Beyond NCC: Learnable, attention-weighted cross-view matching to prioritize discriminative regions (Jin et al., 14 May 2024).
• Seasonal and appearance adaptation: Direct training on multi-season pairs and integration of multispectral/elevation data to mitigate domain shift (Sun et al., 23 Apr 2025, Kinnari et al., 2021).
• Multi-sensor and opportunistic fusion: Tight coupling of IMU, optical-flow, AprilTag beacons, and UWB ranging (Lee et al., 12 Oct 2024, Shi et al., 2019).
• Scalability and memory compression: Low-resolution global tiles, online semantic alignment, and efficient database management for rapid global queries (Yuan et al., 17 Sep 2025).
• Online adaptation and continual learning: Dynamic tuning of depth networks, 2D–3D descriptor adaptation, and loop-closure for extended deployments (Elmaghraby et al., 24 Jun 2025).
• Cross-domain generalization: Foundation model features, semantic map abstraction, and meta-learned embeddings for robust deployment in unseen scenes (He et al., 2023).

The field is moving toward fully modular, uncertainty-aware, and domain-adaptive architectures capable of real-time, global localization in the absence of GNSS.