RealX3D: Real-World 3D Reconstruction Benchmark

• RealX3D is a benchmark that defines real-world, physically degraded multi-view 3D reconstruction with authentic degradations such as low-light, scattering, occlusion, and blur.
• It employs high-resolution RGB and laser-scanned geometry with precise calibration to deliver both supervised and blind evaluation protocols.
• The dataset underpins the NTIRE 2026 3D Restoration and Reconstruction Challenge, measuring performance across photometric and geometric metrics.

RealX3D is a real-world, physically degraded multi-view 3D reconstruction and visual restoration benchmark designed to rigorously test and advance methods for novel-view synthesis, depth estimation, and geometry recovery in challenging, non-ideal imaging conditions. Emphasizing authentic acquisition, the benchmark covers four principal families of degradations—illumination, scattering, occlusion, and blurring—each realized at multiple severity levels using controlled yet realistic protocols. RealX3D provides high-resolution RGB (including RAW) images, dense laser-scanned geometry, and precise extrinsics, enabling both supervised and blind evaluation under disruptive conditions such as extreme low-light or atmospheric smoke. It serves as the foundation for the NTIRE 2026 3D Restoration and Reconstruction (3DRR) Challenge and systematically measures performance of both optimization-based and feed-forward models across a broad range of metrics (Liu et al., 29 Dec 2025, Liu et al., 5 Apr 2026).

1. Benchmark Motivation and Scope

Current multi-view 3D pipelines—Neural Radiance Fields (NeRF), Structure-from-Motion (SfM), Multi-View Stereo (MVS), and 3D Gaussian Splatting—are conventionally developed and evaluated assuming photometrically pristine inputs with minimal scene complexity. In real-world applications, however, these methods are routinely confronted with sensor limitations, environmental degradations, and scene dynamics that violate key assumptions, leading to breakdowns in photometric consistency and geometric reliability. RealX3D addresses several key deficiencies in prior benchmarks:

• Synthetic perturbations (e.g., simple Gaussian blur, noise injection) fail to capture higher-order sensor effects and complex image-formation physics.
• Scope restrictions: existing datasets target single corruption families or lack tightly controlled, pixel-aligned low-quality (LQ) and ground-truth (GT) pairs.
• Absence of world-scale geometry: most benchmarks do not provide metric depth or mesh data for rigorous evaluation against GT surfaces.

Motivated by bridging the “lab-to-wild” gap, RealX3D captures authentic degradations—sensor noise, multiple-scattering haze, dynamic occlusion, reflective glass, and complex blur—under stringent protocols, enabling comprehensive benchmarking of both photometric and geometric performance (Liu et al., 29 Dec 2025).

2. Dataset Design, Acquisition Protocols, and Degradation Families

RealX3D employs a unified hardware and acquisition pipeline, centering on a Sony A7 IV camera mounted to a programmable, high-precision rail-dolly completing dense trajectories (~400 views/scene) stabilized with a gimbal and synchronized with a Leica BLK360 G2 laser scanner. For all images, precise calibration yields intrinsic and extrinsic parameters and alignment in unified world coordinates.

Degradation Families and Implementation

Corruption Family	Model (Mathematical)	Realization & Levels
Illumination	$I_{\rm illu}(x)=T_{\rm illu}(x)\,J(x)+n(x)$	Low-light (–2.7 EV), varying exposures, ISO/scenes
Scattering	Atmospheric path radiance	Multiple smoke density levels via in situ smoke generator
Occlusion	$$I_{\rm occ}(x)=T_{\rm occ}(x)\,J(x)+A_{\rm occ}(x)+n(x)$$	Dynamic objects, glass-plate reflections (5 levels each)
Blur	$I_{\rm blur}(x)=\mathcal B[J(x)]+n(x)$	Motion blur (by exposure/path), defocus (set lens)

For each acquisition, both processed sRGB and linear RAW streams are saved (up to 7008 × 4672 px/view). Laser scans (native precision ≈4 mm @ 10 m) are registered to camera trajectories, yielding metric ground-truth meshes and per-view depth maps.

All degradations are applied physically rather than synthetically. For example, smoke/haze is generated in-studio to achieve five density levels; motion blur is realized by capturing camera trajectories at controlled velocities and exposure times, then ground-truth mesh is used to compute physically accurate blur via integration over sampled poses (Liu et al., 29 Dec 2025).

3. Challenge Tasks, Data Splits, and Evaluation Protocol

NTIRE 2026 3DRR Challenge operationalizes RealX3D for rigorous competitive benchmarking (Liu et al., 5 Apr 2026). The dataset is split for each main track (low-light, smoke):

• Development Set: 1 fully paired “validation” scene (degraded + clean), 4 “blind” development scenes (only degraded images, camera poses).
• Test Set: 3 blind test scenes with degraded images only.

Each scene contains ~30 calibrated multi-view training images and ~5 held-out novel-view evaluation images. Degraded data modalities (low-light/smoke) are the exclusive input for most scenes; clean reference views exist only for validation.

Task formulation:

• Track 1: 3D Low-Light Restoration—predict 3D representation and synthesize novel (ideally clean) views from noisy, photon-starved multi-view input.
• Track 2: 3D Smoke Restoration—same, but input views suffer strong volumetric scattering.

Direct GT geometry is not released (except for validation) to enforce reliance on photometric supervision via novel-view synthesis. For both tracks, integrating 2D sub-modules (denoising, dehazing, illumination correction) and cross-view priors is explicitly permitted (Liu et al., 5 Apr 2026).

Evaluation Metrics:

• Photometric: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), learned perceptual image patch similarity (LPIPS).
• Geometric: When GT available (validation), Chamfer Distance (CD) and Intersection over Union (IoU) for mesh/voxel overlap.
• Rankings: Averaged PSNR over all test scenes determines challenge standings; SSIM breaks ties.

4. Baseline Methods and Quantitative Benchmarks

Organizers provide “Raw-3DGS” as a naïve reference, applying 3D Gaussian Splatting trained directly on degraded views:

• Low-light baseline: PSNR ≈16.5 dB, SSIM ≈0.61
• Smoke baseline: PSNR ≈15.8 dB, SSIM ≈0.58

State-of-the-art solutions for each track demonstrate substantial improvement:

Track & Top Methods	PSNR	SSIM
FuME-GS (low-light)	23.38	0.802
CISP-GS (low-light)	22.78	0.776
GenSmoke-GS (smoke)	20.21	0.726
Smoke-GS (smoke)	18.67	0.691

Feed-forward foundation models (e.g., VGGT, PI3, MapAnything, DepthAnything3) exhibit relative drops in pose, depth, and geometry F1 scores (e.g., F1 drops of –38% for VGGT, –41% for PI3) under strong degradations compared to clean baselines (Liu et al., 29 Dec 2025, Liu et al., 5 Apr 2026).

5. Algorithmic Innovations and Shared Design Principles

Top-performing 3DRR challenge teams adopt strategies that integrate both 2D and 3D reasoning, often exploiting ensemble and multi-branch designs:

• Multi-Model Restoration + Fusion: FuME-GS ensembles four restoration networks (Retinexformer, Zero-DCE, ReDDiT, HVI-CIDNet) with spatially learned α-blending for initializing 3DGS (Liu et al., 5 Apr 2026).
• Multi-Branch Supervision: CISP-GS trains three branches (analytical, ISP-guided, frequency-split), fusing outputs via weighted averaging to exploit complementary pseudo-ground-truth priors.
• Geometry–Illumination Disentanglement: IDEAL’s dual-MLP (geometry vs. appearance) allows the network to absorb photometric noise without baking artifacts into structural representation.
• Physics-Informed Scattering: Smoke-GS leverages a “Smoke Medium Module” with per-ray scattering parameter estimation via spherical harmonic representations.
• Progressive Hybrid Approaches: GenSmoke-GS employs staged (2D/3D) restoration and model ensembling to suppress uncertainty.
• Differentiable End-to-End Optimization: DarkIR-GS unifies 2D correction and 3D geometry via differentiable splatting.

Several novel components are introduced: region-fusion via learned α-masks (FuME-GS), physics-inspired chroma correction (NAKA-GS), global exposure regularization, dual-layer "Residual Gaussian Sets," and explicit modeling of classical haze via transmission and airlight prediction heads (Liu et al., 5 Apr 2026).

6. Performance Analysis, Limitations, and Failure Modes

Empirical results reveal:

• Restoration–Reconstruction Fusion: Explicit 2D restoration remains most robust for photometric fidelity but risks geometric distortion if not cross-view consistent. Hybrid approaches or physics-based priors yield substantial gains.
• Disentanglement: Networks that decouple appearance and geometry (e.g., via auxiliary MLPs for illumination) prevent degradation “bake-in” and achieve sharper, more reliable reconstructions.
• Failure modes: Generative/diffusion-based restoration stages risk hallucinating details, causing geometric drift; over-regularization in end-to-end training can introduce restoration-induced artifacts; pipelines lacking 3D-aware priors exhibit cross-view inconsistencies in deep shadow or dense smoke regions.
• Degradation-Specific Challenges:
  • Low-light: severe contrast collapse, color-shifts, inconsistent output brightness.
  • Scattering: multi-order scattering not addressed by single-scatter models, leading to geometry errors.
  • Blur: inaccurate PSF assumptions induce residual streaks/oversmoothing.
  • Occlusion/reflection: incomplete removal yields floating artifacts (Liu et al., 29 Dec 2025, Liu et al., 5 Apr 2026).

7. Open Challenges and Directions for Future Research

Persistent challenges include:

• Joint Photometric and Geometric Degradation: Existing approaches often treat 2D restoration and 3D inference independently, but severe physical corruption alters both feature matching and scene radiance in non-invertible ways.
• Robust Pose Estimation: SfM reliability collapses under extreme degradation; future pipelines must integrate pose uncertainty directly into reconstruction objectives.
• Generalization to Compound Degradations: Methods tailored to single corruption types seldom generalize across families or co-occurring degradations (e.g., low-light with smoke and motion blur).
• Physics-Aware Neural Fields: Embedding explicit models of multi-order scattering, sensor ISP, and continuous blur into neural reconstructions is recommended.
• Multi-modal and Self-supervised Pipelines: Incorporation of auxiliary sensors (depth, event cameras) offers a path towards more robust initialization; learning cross-view descriptors invariant to physical perturbations is essential.
• Simultaneous Optimization: End-to-end approaches optimizing radiance, depth, pose, and degradation parameters simultaneously are a recommended direction.
• Benchmark Extension: RealX3D may be expanded to cover additional phenomena (rain/snow, underwater, non-rigid, dynamic illumination scenes) and to drive foundation model adaptation for extreme restoration and reconstruction (Liu et al., 29 Dec 2025, Liu et al., 5 Apr 2026).

A plausible implication is that future benchmarks and methods leveraging RealX3D will need to explicitly couple restoration, geometry, and camera parameter estimation in a physically grounded optimization, likely including explicit scene, sensor, and medium models. This suggests interdisciplinary approaches integrating computer vision, computational imaging, and physics-based modeling will be increasingly necessary for robust, real-world 3D scene reconstruction.