egenioussBench: A New Dataset for Geospatial Visual Localisation

Abstract: We present egenioussBench, a visual localisation benchmark built on geospatial reference data: a city-scale airborne 3D mesh and a CityGML LoD2 model. This pairing reflects deployable mapping assets and supports true scalability beyond traditional SfM-based approaches. The query data comprise smartphone images with centimetre-accurate, map-independent ground truth obtained via PPK and GCP/CP-aided adjustment. From 2,709 images, we derive a non-co-visible subset by estimating the full co-visibility matrix from rendered depth and selecting a maximum independent set; the released data include a test split of 42 non-co-visible images with withheld ground truth and a validation split of 412 sequential images with poses, e.g. for training of pose regressors and self-validation. The benchmark features a public leaderboard evaluated with binning metrics at multiple pose-error thresholds alongside global statistics (median, RMSE, outlier ratio), ensuring fair, like-for-like comparison across mesh- and LoD2-based methods. Together, these design choices expose realistic cross-view and cross-domain challenges while providing a rigorous, scalable path for advancing large-scale visual localisation. We make the evaluation code and data availeable at https://github.com/fratopa/egenioussBench and https://www.egeniouss.eu/

• The paper introduces a novel dataset that pairs high-resolution aerial 3D textured meshes with CityGML LoD2 models for geospatial visual localisation.
• Methodologies employ centimetre-accurate pose estimation, non-co-visible query setups, and cold-start single-image localisation protocols.
• Baseline evaluations demonstrate realistic challenges with only 19.05% of images localised within strict pose-error thresholds, urging robust algorithm development.

egenioussBench: A City-Scale Benchmark for Geospatial Visual Localisation

Motivation and Context

Visual localisation—estimating a camera's 6-DoF pose from imagery relative to spatial reference data—is critical in autonomous driving, robotics, UAV navigation, and AR, especially in GNSS-denied urban canyons. Current dataset and system design often rely on SfM-based point clouds or ground-level photogrammetric meshes, which suffer from poor scalability, significant storage footprints, and over-optimistic evaluation due to ideal reference representations. Furthermore, many benchmarks are constrained by pose ground truth tightly coupled to the reference map, and sequences with spatially overlapping imagery enable multi-view geometry, reducing the true challenge of localisation.

egenioussBench, introduced in "egenioussBench: A New Dataset for Geospatial Visual Localisation" (2605.05351), addresses these methodological and practical limitations. It enables rigorous, scalable benchmarking of city-scale visual localisation by providing both an airborne-derived 3D mesh and an industry-standard CityGML LoD2 object model as geospatial baselines.

Figure 1: Comparison of egenioussBench to other state of the art mesh-based visual localisation datasets.

Dataset Design and Properties

The coverage area is a 0.4 km² region in Braunschweig, Germany, with high-standard map and query data. The dataset design follows several critical principles:

• Cross-domain, cross-view composition: Query images (2709 total, of which 42 are test, 412 for validation) were captured at street level using a tightly mounted smartphone+INS, whereas reference data comprises (i) a high-resolution, airborne 3D textured mesh (7.5 cm GSD) and (ii) a CityGML LoD2 model.
• Centimetre-accurate, map-independent ground truth: The query pose is computed via PPK, photogrammetric bundle adjustment using independent GCPs and CPs, and refined SfM. Image pose mean error is 8 mm (X,Y), 3 mm (Z), and $0.04^\circ$ mean orientation error.
• Non-co-visible test subset via co-visibility graph: Rather than allowing multi-view clues, the authors constructed a maximal independent set from a graph of image co-visibility based on rendered mesh depth images, yielding 42 strictly non-co-visible test queries.
• Public release and standardised evaluation: Data, evaluation code, and leaderboard are open-access, directly supporting rigorous and reproducible experiments.

  Figure 2: Overview of the area of interest in Braunschweig. Map source: basemap.de.

  

Figure 3: Geospatial mesh data; Data: Geofly, Processing: Skyline.

Figure 4: LoD2 model; Data: City of Braunschweig.

Benchmark Protocol and Tasks

The evaluation protocol enforces cold-start single-image localisation. Query splits are provided such that the test images have no spatial overlap and their ground truth is withheld for leaderboard submission. Methods are categorised by reference data:

• Mesh-based localisation: Methods localise against a high-fidelity, textured 3D mesh.
• LoD2/Object-based localisation: The CityGML model presents a more abstract, textureless and low-detail scene for evaluation.

Metrics are aligned with established conventions: the percentage of images localised within progressively stricter pose-error bins (e.g., 0.5m,2°, 2m,5°, 5m,10°), median/RMSE translation and rotation error, and outlier rate. Methods must submit 6-DoF pose estimates for the test queries.

Baseline Evaluation

The authors establish baselines with MeshLoc and their previous mesh-based pipeline [vultaggio_et_al_lc3d2024]. To ensure comparability, both pipelines use identical retrieval (CosPlace descriptors tailored to synthetic views) and pose estimation (SuperPoint features and modern RANSAC). Initialisation is reduced to mesh rendering at semantically meaningful street-level views with OpenStreetMap-derived road networks.

Summary of baseline accuracy on the challenging test split:

• % localised within 0.5 m,2°: 19.05 (both methods)
• Median translation error: 0.97 m for MeshLoc, 0.89 m for the improved method
• Runtime: Substantial speedup in the improved pipeline (41 s vs. 290 s per image)

These results are substantially lower than those reported for typical SfM-based or ground-level mesh benchmarks, highlighting the increased difficulty introduced by viewpoint discrepancy, abstract/textureless geometry, and outdoor city-scale context.

Figure 5: Left and right: query image and mesh-rendered reference view. Center: camera trajectory, GNSS, and sampled views.

Figure 6: Smartphone rigidly mounted to INS system for cm-level query pose estimation.

Practical and Theoretical Implications

egenioussBench fills a significant gap in city-scale visual localisation:

• Challenging cross-view, cross-domain matching: By pairing ground imagery with an aerial mesh and LoD2 object model, the task exposes the limitations in pose estimation when traditional feature-based or simple render-and-compare strategies are applied.
• Scalable to production assets: Both reference data types are obtained from production mapping workflows, avoiding the laborious, small-scale ground mapping required by previous benchmarks.
• Enables method development beyond fine-grained textured models: LoD2 supports development of robust matchers, domain adaptation, and neural implicit pose regressors effective with textureless or simplified models.
• Reproducible evaluation: Map-independent, professionally surveyed ground truth removes bias, and the test split structure neutralises multi-view shortcutting.

Limitations and Future Directions

Current limitations are acknowledged:

• Scene diversity: The release covers a single (albeit urban and structurally complex) region. Extension to different cities, landscapes, or acquisition platforms (UAV, bicycle) would improve coverage.
• Object-based baselines: The CityGML reference evaluation is mainly proposed; further work is needed to develop robust cross-domain feature extractors and pose regressors for textureless/abstract models.
• Multimodality and dynamic context: Future work should address localisation robustness with dynamic urban scenes as well as support for temporal and weather variation.

Conclusion

egenioussBench (2605.05351) provides a rigorously curated, scalable, and challenging benchmark for mesh- and object-based visual localisation. It enables fair, realistic, and reproducible evaluation across methods leveraging high-resolution aerial meshes and LoD2 object models under strict cold-start conditions, catalysing research in cross-domain correspondence, viewpoint selection, robust or learned localisation, and scalability in large-scale urban environments. The dataset and evaluation protocol are available for community participation and extension.

Figure 7: EU Horizon project funding acknowledgement.