TerraSky3D: European 3D Benchmark

• TerraSky3D is a high-resolution, real-world 3D reconstruction dataset featuring 50,000 4K images and mixed aerial-ground scenes to support cross-view localization.
• It offers pre-calibrated camera intrinsics, curated depth maps, and explicit registration of aerial and ground views to enhance structure-from-motion and multi-view stereo methods.
• The benchmark protocol evaluates tasks like relative pose estimation, end-to-end 3D reconstruction, and novel view synthesis using metrics such as AUC@5°, PSNR, and SSIM.

Searching arXiv for TerraSky3D and related work to ground the article in current papers. TerraSky3D is a high-resolution, real-world, multi-view 3D reconstruction dataset centered on European landmarks and designed to address limitations of existing public benchmarks for modern multi-view stereo, structure-from-motion, and neural reconstruction methods. It comprises approximately 50,000 RGB images divided into 150 ground, aerial, and mixed scenes, and provides curated calibration data, camera poses, and depth maps (D'Urso et al., 30 Mar 2026). Its distinguishing features are real capture rather than internet scraping, constant-aspect-ratio 4K imagery, pre-calibrated intrinsics, filtered dense depth maps, and explicit inclusion of mixed aerial-ground scenes registered in the same SfM model, making it directly relevant to cross-view reconstruction and localization (D'Urso et al., 30 Mar 2026).

1. Definition and motivation

TerraSky3D was introduced in response to a recurring deficiency in public 3D datasets: many are low resolution, contain few scenes, rely on internet photos with inconsistent quality, omit reliable calibration or intrinsics, or are restricted to either ground-only or aerial-only viewpoints (D'Urso et al., 30 Mar 2026). The dataset was therefore captured specifically for 3D reconstruction rather than assembled from web imagery. This yields cleaner imagery, controlled capture conditions, pre-calibrated camera parameters, and improved viewpoint coverage for geometry estimation (D'Urso et al., 30 Mar 2026).

A central motivation is support for cross-view reconstruction and localization. In particular, TerraSky3D targets settings in which methods must operate across aerial-to-ground and ground-to-aerial viewpoint changes, a regime that is typically underrepresented in prior public benchmarks (D'Urso et al., 30 Mar 2026). The explicit inclusion of mixed aerial-ground scenes is therefore not incidental; it is one of the dataset’s principal design objectives.

The dataset focuses on European landmarks, especially in Central and Southern Europe, and includes castles, churches, town halls, museums, bridges, dams, shrines, statues, fountains, piers, and lakeside villas (D'Urso et al., 30 Mar 2026). Example scenes shown in the paper include Villalta Castle, Italy; Natural History Museum, Vienna; Italian Charnel House, Kobarid, Slovenia; Erto e Casso, Italy; and Barcis Dam, Italy (D'Urso et al., 30 Mar 2026).

2. Composition, scene taxonomy, and geographic coverage

TerraSky3D contains approximately 50,000 RGB images, 150 scenes, over 2.5 million image pairs, and a global mean reprojection error of 0.8 pixels (D'Urso et al., 30 Mar 2026). The imagery was captured in 4K resolution at 30 fps, collected across 26 cities in 10 countries, while the map figure covers 11 European countries overall (D'Urso et al., 30 Mar 2026).

The scene taxonomy is one of the dataset’s defining attributes.

Scene type	Count	Description
Ground-only	115	Ground views only
Aerial-only	5	Aerial views only
Mixed	30	Ground and aerial views registered in the same SfM model

The mixed category is especially important because it provides real aerial-plus-ground pairs within a shared reconstruction, rather than synthetic or weakly aligned cross-view correspondences (D'Urso et al., 30 Mar 2026). This makes TerraSky3D unusual among public benchmarks.

The capture platform is likewise heterogeneous but controlled. Ground images were recorded with smartphones, aerial images with a DJI Mavic 3 drone, and between 1 and 4 cameras per scene were used (D'Urso et al., 30 Mar 2026). Videos were recorded at 4K and 30 fps, after which frames were uniformly sampled to produce the final image set (D'Urso et al., 30 Mar 2026). The use of controlled capture rather than internet scraping also reduces artifacts such as watermarks and timestamps (D'Urso et al., 30 Mar 2026).

3. Acquisition, calibration, and reconstruction pipeline

The geometric pipeline combines pre-calibration, SfM reconstruction, manual verification, and dense depth computation. Camera intrinsics are pre-calibrated using a ChArUco board and OpenCV, stored with the SIMPLE_RADIAL camera model, and then refined during reconstruction (D'Urso et al., 30 Mar 2026). The authors explicitly state that COLMAP remains the most robust solution for their setting, and manual inspection is used to verify correct camera registration (D'Urso et al., 30 Mar 2026).

Depth is computed with APD-MVS, chosen because it handles large textureless areas better than standard PatchMatch-style methods and regions where COLMAP-MVS may fail (D'Urso et al., 30 Mar 2026). Semantic filtering is then applied with Mask2Former, which segments sky, vegetation, and transient objects so that those regions can be removed from the depth maps (D'Urso et al., 30 Mar 2026). The dataset also releases APD-MVS confidence masks for downstream filtering (D'Urso et al., 30 Mar 2026).

The resulting annotation package is multimodal.

Modality	Provided content	Function
Geometry	Calibration data, camera poses, COLMAP sparse reconstructions	Metric camera geometry and pose supervision
Dense supervision	Filtered depth maps, APD-MVS confidence masks	Dense reconstruction and depth filtering
Imagery	RGB images	Reconstruction, matching, NVS, localization

These modalities jointly support both classical geometry pipelines and learning-based models (D'Urso et al., 30 Mar 2026). Metric camera geometry enables SfM and MVS training or evaluation; pose supervision supports relocalization and relative pose estimation; dense depth supervision supports reconstruction, depth completion, and novel view synthesis; confidence masks exclude unreliable regions; and the registered mixed-view scenes supply explicit cross-view alignment between aerial and ground imagery (D'Urso et al., 30 Mar 2026).

4. Metrics, benchmark protocol, and evaluation formalism

TerraSky3D defines an official test split for consistent benchmarking and evaluates multiple tasks: relative pose estimation, end-to-end 3D reconstruction, novel view synthesis, and bidirectional geometric consistency (D'Urso et al., 30 Mar 2026).

For end-to-end reconstruction, the paper defines relative distance error for an image pair $(i,j)$ as

$$E_{rel}(i, j) = \rho \left( (P_i^{gt})^{-1} P_j^{gt}, \; \hat{P}_i^{-1} \hat{P}_j \right),$$

where $P^{gt}$ denotes ground-truth absolute pose, $\hat{P}$ estimated absolute pose, and $\rho(\cdot,\cdot)$ the maximum of angular rotation and translation errors between the relative transforms (D'Urso et al., 30 Mar 2026). Reported summary performance uses AUC@5° (D'Urso et al., 30 Mar 2026).

For depth evaluation, geometric consistency is measured through a bidirectional reprojection check,

$$E_{cyc}(\mathbf{p}_i) = ||\mathbf{p}_i - \mathbf{p}_{i \to j \to i}||_2,$$

which tests how well a point reprojects from one image to another and back again (D'Urso et al., 30 Mar 2026). Relative pose estimation similarly uses pairwise pose error defined as the maximum of rotation error $\Delta R$ and translation error $\Delta t$, and reports AUC up to 5° (D'Urso et al., 30 Mar 2026).

For novel view synthesis, the benchmark reports PSNR, SSIM, and LPIPS, with higher PSNR and SSIM and lower LPIPS indicating better performance (D'Urso et al., 30 Mar 2026). In the 3D Gaussian Splatting protocol, every 8th image is used as test input and training is performed on the remaining images for 30,000 iterations (D'Urso et al., 30 Mar 2026).

The test-set pair distribution is itself informative: it contains over 43,000 image pairs, of which 74.9% are ground, 10.7% aerial, and 14.4% mixed (D'Urso et al., 30 Mar 2026). The benchmark is therefore strongly skewed toward ground pairs while still retaining a meaningful hard subset of mixed-view data (D'Urso et al., 30 Mar 2026).

5. Reported empirical findings

In relative pose estimation, RoMa is reported as the best overall method, while ALIKED + SANDesc performs best among sparse-feature-based pipelines (D'Urso et al., 30 Mar 2026). Retraining SANDesc on TerraSky3D improves mixed-view performance by 1.8 AUC points at 5°, indicating that the dataset is sufficiently distinct that in-domain adaptation yields measurable gains (D'Urso et al., 30 Mar 2026).

For end-to-end 3D reconstruction, the paper evaluates VGGT, MapAnything, and $\pi^3$. VGGT achieves the best mean AUC@5° at 49.7, followed by MapAnything at 45.0 and $\pi^3$ at 36.8 (D'Urso et al., 30 Mar 2026). The paper further notes that VGGT is the most consistent, whereas $$E_{rel}(i, j) = \rho \left( (P_i^{gt})^{-1} P_j^{gt}, \; \hat{P}_i^{-1} \hat{P}_j \right),$$0 can score well on individual scenes but also suffers catastrophic failures, especially in aerial-ground scenarios (D'Urso et al., 30 Mar 2026). This observation is important because it indicates that mixed-view robustness is not well captured by aggregate scores alone.

For bidirectional geometric consistency, TerraSky3D is compared with MegaDepth using cumulative inlier percentages at 1 px, 3 px, 5 px, and 10 px (D'Urso et al., 30 Mar 2026). TerraSky3D shows higher consistency overall and, crucially, includes mixed aerial-ground pairs that MegaDepth does not (D'Urso et al., 30 Mar 2026). This makes direct comparison only partially symmetrical: the newer benchmark is simultaneously more consistent and more cross-view diverse.

The dataset is presented as useful for SfM, multi-view stereo, depth estimation, relative pose estimation, end-to-end 3D reconstruction, novel view synthesis, cross-view localization, and aerial-ground matching (D'Urso et al., 30 Mar 2026). A plausible implication is that TerraSky3D is best understood not merely as an image collection, but as a benchmarked geometric supervision package for realistic large-baseline outdoor reconstruction.

6. Relation to terrain modeling, simulation, and adjacent work

TerraSky3D is a reconstruction dataset rather than a terrain-generation method. Nevertheless, several adjacent papers clarify the broader pipeline in which such data may be used. "An Automatic Digital Terrain Generation Technique for Terrestrial Sensing and Virtual Reality Applications" describes an automatic workflow that converts LiDAR-style point clouds into Digital Terrain Models through preprocessing, heightmap generation, and terrain modeling, with output packaged into Unreal Engine 4 Landscapes for real-time VR rendering (Easson et al., 2019). That work addresses DTM extraction and rendering from point clouds, whereas TerraSky3D contributes calibrated multi-view imagery, poses, and depth supervision (D'Urso et al., 30 Mar 2026). This suggests complementary rather than competing roles.

"Deep Generative Framework for Interactive 3D Terrain Authoring and Manipulation" addresses a different adjacent problem: interactive, example-based terrain authoring using a VAE latent space over topographic maps and a Pix2pix-like cGAN for DEM generation (Naik et al., 2022). Its emphasis is latent-space terrain editing, multiple variants from one input, and interpolation between terrains (Naik et al., 2022). A plausible implication is that TerraSky3D could inform geometric or visual components of broader outdoor scene pipelines, but it does not itself provide a sketch-to-terrain authoring interface.

"Fully Automated Photogrammetric Data Segmentation and Object Information Extraction Approach for Creating Simulation Terrain" adds semantic segmentation and object-information extraction to UAS-derived photogrammetric terrain for ATLAS-style simulation environments (Chen et al., 2020). Its reported outputs include top-level terrain segmentation, tree locations and attributes, and ground material classification for navigation mesh weighting (Chen et al., 2020). In contrast, TerraSky3D is presented as a benchmark for reconstruction and cross-view geometry, not as a semantic simulation-terrain framework (D'Urso et al., 30 Mar 2026).

A common misconception is to conflate TerraSky3D with later works named Terra. The paper "Terra: Explorable Native 3D World Model with Point Latents" does not use the name TerraSky3D explicitly; its method name is Terra and it concerns a native 3D world model for indoor scenes (Huang et al., 16 Oct 2025). Likewise, "Terra: Hierarchical Terrain-Aware 3D Scene Graph for Task-Agnostic Outdoor Mapping" introduces a terrain-aware outdoor 3DSG for robotic mapping and navigation rather than a reconstruction dataset (Samuelson et al., 23 Sep 2025). TerraSky3D therefore denotes a specific 2026 European-landmark reconstruction dataset, not a generic family of Terra-branded 3D systems.