MegaUnScene Benchmark Overview

• MegaUnScene Benchmark is a large-scale dataset designed to assess 3D foundation models on wild, unconstrained Internet scenes with minimal visual overlap.
• It supports two main tasks—relative pose estimation and dense 3D reconstruction—using strictly curated data that avoids any training data leakage.
• The benchmark’s detailed metrics and baseline results expose current generalization challenges, driving advancements in robust 3D scene understanding.

MegaUnScene is a large-scale, challenging benchmark for evaluating 3D foundation models (3DFMs) under unconstrained Internet conditions not seen during model training. It was introduced to systematically test both relative pose estimation and dense 3D reconstruction in Internet scenes characterized by minimal visual overlap, wild illumination, diverse camera parameters, and uncontrolled content. MegaUnScene explicitly excludes any overlap with existing 3DFM training data, thus serving as a critical resource for assessing generalization and emergent geometry reasoning under extreme conditions (Zhang et al., 27 Nov 2025).

1. Dataset Construction and Statistics

MegaUnScene is built entirely from Wikimedia Commons photo collections and linked Wikidata entries, employing the MegaScenes pipeline with explicit exclusions for all scenes or images used in existing 3DFM datasets (e.g., MegaDepth, MegaScenes, AerialMegaDepth, WikiScenes). Scene IDs and filenames are cross-checked to avoid data leakage—Wikimedia enforces unique filenames, which ensures precise curation.

• Total scenes: 476 unique Internet scenes.
• Reconstructions: 485 SfM+MVS reconstructions, each comprising at least 50 input images.
• Exclusion criteria: Scenes or images that appear in any prior dataset used for 3DFM training or with >10% image-level overlap to MegaScenes are removed. For UnScenePairs, both images in a pair must have zero overlap with any MegaScenes image.

Test split composition:

Subset	Scenes	Image Pairs	Overlap Regimes (pairs)
UnScenePairs	458	3,883	1,878 large, 1,227 small, 778 none
UnScenePairs-t	387	2,432	1,146 large, 523 small, 763 none
UnSceneRecon	96	100 dens.	Human-annotated metric reconstructions

No training or validation data is provided in MegaUnScene; all test splits are reserved for systematic evaluation only.

2. Benchmark Splits, Task Definitions, and Preprocessing

MegaUnScene provides distinct splits optimized for two primary tasks: relative pose estimation and dense 3D reconstruction.

• UnScenePairs: Focuses on rotation-only relative pose estimation. Pairs are selected as K=5 mutual nearest-neighbors in translation space, manually filtered for blur, occlusion, and texture. Pairs have negligible baseline, targeting pure orientation prediction.
• UnScenePairs-t: Evaluates pose estimation under nontrivial camera translation. K=50 mutual nearest-neighbor selection admits significant baseline. Focal length and image resolution are checked to avoid scale mismatch. Pairs without geometric overlap are verified by enforcing zero inlier feature matches (Doppelgangers++ and MASt3R-SfM).
• UnSceneRecon: Dense 3D reconstruction is performed for 96 scenes (~100 reconstructions). Scenes are processed using COLMAP, MASt3R-SfM, Doppelgangers++, and MVS stereo fusion. Human annotators prune failed reconstructions, assign a metric scale based on Google Maps satellite measurements, and label accordingly.

All images undergo undistortion with COLMAP and are resized (longest edge to 518 px for dense, 512 px for multiview pose), rounding to multiples of 14 for transformer model compatibility, and zero-padded as needed. No synthetic augmentation is used.

3. Evaluation Metrics and Mathematical Formulation

Performance is reported under both relative pose estimation and dense reconstruction, using the following metrics:

Relative Pose Estimation

Given predicted and ground-truth rotations ($$\mathbf{R}_{\mathrm{pred}}, \mathbf{R}_{\mathrm{gt}} \in \mathrm{SO}(3)$$), the geodesic rotation error is: $$\Delta R = \arccos\!\left(\frac{1}{2}\big(\operatorname{tr}(\mathbf{R}_{\mathrm{pred}}^\top \mathbf{R}_{\mathrm{gt}}) - 1\big)\right)$$

For translation direction (UnScenePairs-t): $$\Delta t = \arccos\!\left(\frac{|\mathbf{t}_{\mathrm{pred}}^\top \mathbf{t}_{\mathrm{gt}}|}{\|\mathbf{t}_{\mathrm{pred}}\|\;\|\mathbf{t}_{\mathrm{gt}}\|}\right)$$

Reported pose metrics:

Name	Description
MRE	Median of $\Delta R$
RA$_\theta$	% pairs with $\Delta R < \theta$ ($\theta=15^\circ,30^\circ$)
MTE	Median of $\Delta t$ (translation angle)
TA$_\theta$	% pairs with $\Delta t < \theta$

Dense 3D Reconstruction

Predicted point cloud ($\mathcal{P}_{\mathrm{pred}}$) is aligned to ground truth ($\mathcal{P}_{\mathrm{gt}}$) via Umeyama + ICP:

• Accuracy (ACC): Mean/median $\min_{q\in\mathcal{P}_{\mathrm{gt}}} \|p - q\|$ for all $p \in \mathcal{P}_{\mathrm{pred}}$.
• Completion (CMP): Mean/median $\min_{p\in\mathcal{P}_{\mathrm{pred}}} \|q - p\|$ for all $q \in \mathcal{P}_{\mathrm{gt}}$.

Lower ACC/CMP (in meters) indicates higher reconstruction fidelity. Some ablations report aggregate reconstruction score $$\mathrm{REC} = \frac{1}{2}(\mathrm{ACC}_{\mathrm{med}} + \mathrm{CMP}_{\mathrm{med}})$$.

4. Baseline Model Performance

MegaUnScene baseline tables report results for state-of-the-art 3DFMs (VGGT, WorldMirror, $\pi^3$); both pretrained and fine-tuned ("FT") variants are compared. A summary (rotation-only, non-overlap pairs):

Method	UnScenePairs MRE	RA$_{30}$	UnScenePairs-t MRE	RA$_{30}$
VGGT	31.64°	48.8%	46.65°	42.1%
VGGT$_{\mathrm{FT}}$	12.71°	67.9%	14.48°	62.1%
WorldMirror	19.25°	58.9%	21.52°	57.4%
WM$_{\mathrm{FT}}$	11.75°	68.1%	13.13°	64.5%
$\pi^3$	17.66°	59.4%	21.62°	56.8%
$\pi^3_{\mathrm{FT}}$	12.92°	69.2%	13.31°	65.5%

Fine-tuning on similar data yields drops of $20^\circ$–$30^\circ$ median rotation error and $+20$–$30$ percentage point gain in RA$_{30}$.

Dense Reconstruction (UnSceneRecon, 100 scenes):

Method	ACC$_{\mathrm{med}}$ (m)	CMP$_{\mathrm{med}}$ (m)
VGGT	1.049 → 0.908	0.729 → 0.650
WM	0.612 → 0.660	0.387 → 0.368
$\pi^3$	0.466 → 0.517	0.377 → 0.403

Reconstruction quality is generally preserved or slightly improved, even when only rotation supervision is applied.

5. Benchmark Significance and Application Scenarios

MegaUnScene is designed to stress-test 3DFMs beyond the scope of existing benchmarks. Its unique challenges are:

• Unconstrained Content: Scenes are "in the wild" Internet captures, exhibiting uncontrolled illumination, varying focal lengths, dynamic elements, and extreme intra-scene variations.
• Zero Overlap Regime: Large fractions of test pairs feature minimal or no field-of-view overlap, precluding classical feature-matching approaches. Models must leverage 3D priors and internal representations to infer relative pose and structure.
• Generalization Requirement: All scenes are strictly unseen in any prior public 3DFM training data, eliminating spurious memorization or domain leakage.
• Benchmark Breadth: Simultaneously supports evaluation of both pairwise pose estimation and dense point cloud reconstruction with metric scale.

Potential usage domains include:

• Robust pose estimation for historical archives and photo tourism collections with non-overlapping or temporally distant views.
• Sparse wide-baseline SLAM and scene digitization for AR/VR applications from highly sparse or crowd-sourced image sets.
• Cultural heritage preservation and modeling via large-scale Internet photo mining.
• Any downstream system requiring 3D scene understanding under visual occlusion, viewpoint mismatch, or scene ambiguity.

MegaUnScene is described as "476 Internet scenes unseen by existing 3DFMs, enabling systematic evaluation of 3DFMs under realistic, unconstrained conditions for both relative pose estimation and dense 3D reconstruction" (Zhang et al., 27 Nov 2025).

6. Contextual Placement and Relations to 3D Vision Benchmarks

MegaUnScene directly addresses limitations identified in prior large-scale 3D vision benchmarks, notably the lack of out-of-distribution evaluation material and reliance on overlapping/landmark-centric datasets. It builds on the MegaScenes curation pipeline but innovates by strictly separating test data from any model pretraining sets. The conception of MegaUnScene also responds to calls within the 3D-LLM literature for "ultra-diverse" benchmarks aggregating real scans and procedurals to test generalizable scene understanding (Zhang et al., 23 Apr 2024).

In contrast with benchmarks such as 3DBench (focused on multi-modal 3D-LLM evaluation across procedurally generated indoor scenes), MegaUnScene exclusively targets real, wild Internet scenes at scale, significant for grounding emerging 3DFMs in truly novel content distributions. It is complementary to large-scale scene reconstruction datasets like GauU-Scene, which target outdoor campus-scale settings with different representation (Gaussian splatting, LiDAR ground truth) (Xiong et al., 25 Jan 2024).

7. Prospects and Open Challenges

MegaUnScene exposes current deficits in 3DFM generalization, as evidenced by substantial performance drops under extreme viewpoint and overlap regimes. The dataset motivates future research directions:

• Improved point cloud and pose encoders that leverage global geometric priors.
• Enhanced alignment and finetuning protocols selectively targeting backbone bias terms without full decoder retraining.
• Larger-scale multi-source data curation for increasing scene and appearance diversity.
• New evaluation metrics better measuring generalization to wild, unconstrained Internet imagery.

Ongoing work emphasizes refining both model architecture and benchmark design to better model the nonstationarity and ambiguity intrinsic to user-generated Internet photo collections.