RealX3D Benchmark for 3D Restoration & Reconstruction

• RealX3D is a large-scale, real-capture multi-view benchmark featuring pixel-aligned low-quality and ground-truth images from 55 indoor scenes for robust 3D restoration and reconstruction.
• It rigorously evaluates physics-aware methods across controlled degradations—including low-light, smoke, occlusion, and blur—using detailed photometric and geometric metrics.
• The benchmark underpins international challenges like NTIRE 2026, providing high-resolution data, precise calibration, and comprehensive evaluation protocols for advancing multi-view computer vision.

RealX3D is a large-scale, real-capture multi-view benchmark designed to rigorously evaluate and facilitate research on visual restoration and 3D reconstruction under physically realistic degradations, including adverse conditions that critically test the robustness of multi-view computer vision pipelines. By providing strictly pixel-aligned pairs of degraded (LQ) and clean (GT) views captured under a suite of controlled degradations, RealX3D enables detailed assessment and development of physics-aware, degradation-robust methods for structure-from-motion, novel-view synthesis, and dense geometry estimation (Liu et al., 29 Dec 2025). It further serves as the basis for international benchmarks such as the NTIRE 2026 3D Restoration and Reconstruction Challenge (Liu et al., 5 Apr 2026).

1. Dataset Composition and Organization

RealX3D consists of 55 distinct indoor scenes, each captured along a programmable rail-dolly trajectory comprising approximately 30 viewpoints. For each scene, a high-fidelity "clean" (ground truth, GT) reference sequence is acquired under studio lighting conditions, and a series of physically degraded sequences—covering four corruption families—are recaptured with intrinsics and extrinsics strictly preserved. Each family of degradations is applied at multiple discrete severity levels, yielding a total of 9 low-quality (LQ) settings per family. This results in, per scene, 30 GT and 270 LQ images, summing to a corpus of 1,650 GT and 14,850 LQ images across the dataset.

Every view is available as a high-resolution 7008×4672 px sRGB TIFF, a full-frame RAW sensor measurement, and is strictly pixel-aligned between LQ–GT pairs. Dense geometry is provided via laser scans (≈50 million points/scan at 4 mm accuracy, fused from 5 scans per scene), Poisson-reconstructed watertight meshes, and depth maps rendered from registered camera poses. The directory structure reflects the hierarchy of scenes and degradation types, enabling precise experimental control (Liu et al., 29 Dec 2025).

For benchmarking challenges, ablated versions (e.g., 14-scene subsets, 1920×1080 px) are used, providing paired or unpaired degraded/clean sequences, camera intrinsics/extrinsics (COLMAP format), and explicit test/validation splits, as detailed in (Liu et al., 5 Apr 2026).

2. Physical Degradation Families

The benchmark features four physically-motivated degradation families, each parameterized at several severity levels:

• Illumination: Encompasses consistent low-light (e.g., uniform under-exposure at 1/400 s vs. GT at 1/10 s) and varying exposure (shutter speeds 1/60 to 1/400 s, spanning ~0 to +2.7 EV). The degradation is modeled as scaling of the radiance map $J(x)$ by a transmission $T_\text{illu}(x)\in[0,1]$.
• Scattering (Smoke/Haze): Achieved by generating real smoke in a sealed studio via a 1,200 W fog machine, targeting a uniform 3D distribution. The observed image can be described (at first order) by the single-scatter atmospheric scattering model:

$$I(x) = J(x)\cdot\exp(-\beta z(x)) + B_\infty\cdot(1-\exp(-\beta z(x)))$$

In practice, complex multi-scattering and incident radiance attenuation from real smoke yield a more challenging, less tractable degradation (Liu et al., 29 Dec 2025).

• Occlusion: Includes both static occluders repositioned between views and dynamic (fast-moving) distractors, introducing motion streaks and ghosting. Additionally, a glass plate (92% transmittance) is used to create reflection-induced occlusion via view-dependent layers $A_\text{occ}(x)$.
• Blurring: Both defocus (lens misfocus from nominal 3–5 m down to 0.6 m or 0.4 m) and motion blur (path integration over 2–5 cm at each target pose using 64 sampled intermediate frames with an exposure-integration operator $\mathcal{B}_\text{motion}$).

All degradations are modeled under a unified forward model:

$$I_d(x) = \mathcal{B}_d[T_d(x)\cdot J(x) + A_d(x)] + n_d(x)$$

where $\mathcal{B}_d$ is a blur/integration operator, $T_d(x)$ is direct transmission, $A_d(x)$ is parasitic radiance (airlight, ghosting), and $n_d(x)$ is sensor noise (Liu et al., 29 Dec 2025).

3. Acquisition Protocol and Calibration

Scenes are captured using a Sony A7 IV (24–70 mm f/2.8 GM lens) mounted on a curved 1 m rail-dolly with constant-velocity automation and vibration-suppressed via gimbal. Intrinsics are calibrated per scene (4×9 CharUco board), and all views (both GT and LQ) are undistorted to the pinhole model. For scenes incompatible with the rail system, a tripod replicates the framing. All exposures are saved in RAW format; no video compression is used.

Laser-based geometry is acquired via a Leica BLK360 G2. COLMAP estimates camera poses on GT images, which are then registered to the world coordinate frame by rigidly aligning the sparse COLMAP point cloud to the laser scan (manual point correspondences + ICP refinement), achieving a typical pose-scan RMSE of 1.2 cm. Dense geometry is reconstructed by multi-scan fusion, denoising, subsampling, Poisson surface reconstruction, mesh decimation, and watertight mesh extraction (Liu et al., 29 Dec 2025).

4. Task Definitions and Evaluation Metrics

RealX3D supports multi-view restoration and 3D reconstruction or novel-view synthesis tasks under real physical degradations. The protocol for method evaluation consists of:

1. Ingest RAW or sRGB degraded images and GT pose files.
2. Restore or reconstruct per-view images, reporting photometric metrics:
   • MSE: Mean squared error
   • PSNR: $T_\text{illu}(x)\in[0,1]$0
   • SSIM: Structural similarity [Wang et al. 2004]
   • LPIPS: Learned perceptual metric
3. Fuse restored images into a 3D scene representation, reporting geometric metrics:
   • Chamfer distance, computed symmetrically between surfaces
   • For feed-forward models: pose AUC@5°/10°/20°, depth L1 (in cm), point-to-surface accuracy/completeness/F1 at tolerance (e.g., 5 cm)
4. (For challenge settings) Render novel views from 3D representation and compare photometrically to held-out GT frames (Liu et al., 29 Dec 2025, Liu et al., 5 Apr 2026).

Challenge leaderboards use average PSNR across test scenes as the principal ranking metric, with SSIM for tie-breaking. Geometry metrics (e.g., Chamfer distance, voxel IoU) are included where ground-truth meshes are available.

5. Baseline and State-of-the-Art Methods

RealX3D provides systematic benchmarks covering both optimization-based and feed-forward methods:

• Optimization-based: 3D Gaussian Splatting (3DGS) serves as a vanilla baseline. Ablations include:
  • Low-light: LITA-GS (best PSNR ≈ 15.63 dB, SSIM ≈ 0.542), AlethNeRF, I2-NeRF, Luminance-GS.
  • Exposure variation: LITA-GS (PSNR ≈ 16.06 dB).
  • Smoke: Watersplatting outperforms SeaThruNeRF (PSNR ≈ 10.74 dB).
  • Occlusion: SpotlessSplats and DeSplat lead (dynamic occlusion PSNR ≈ 28.68 dB, reflection PSNR ≈ 26.06 dB).
  • Blurring: DeBlurringGS (PSNR ≈ 20.64 dB) marginally improves over 3DGS.
• Feed-forward Foundation Models: For pose, VGGT and Pi3 maintain robustness (AUC@5° ≈ 82.7 %, 76.1 %). DepthAny.3 and MapAnything yield best completeness/accuracy (F1@5 cm ≈ 55.7 %), though performance degrades in heavy smoke/low light.

Challenge results (Liu et al., 5 Apr 2026) corroborate these findings, with top teams employing (i) elaborate 2D enhancement via multi-model fusion (e.g., Retinexformer, Zero-DCE, ReDDiT), (ii) depth-guided geometric regularization, (iii) analytic and learned degradation-invariant restoration, and (iv) photometric calibration as preconditions for successful 3DGS reconstruction. Example top-performers include FuME-GS (PSNR = 23.38 dB, SSIM = 0.802 on low-light) and GenSmoke-GS (PSNR = 20.21 dB, SSIM = 0.726 on smoke).

6. Analytical Insights and Design Principles

Empirical evaluations show that current pipelines for multi-view restoration and reconstruction are highly fragile under real physical corruptions. Significant qualitative failure modes include:

• Under-exposure: detail collapse, amplified noise, severe color shifts for EV ≤ –3.
• Smoke: non-uniform density occludes fine structure, colors become “washed out.”
• Occlusion: dynamic elements induce ghosting, semi-transparent artifacts.
• Blur: over-smoothing, geometry degradation, splat spreading.

Successful approaches commonly employ:

• Dual-stage pipelines that decouple 2D enhancement and multi-view 3D fusion.
• Physically-inspired priors (dark channel, atmospheric scattering, Naka-Rushton transform) for modeling degradations.
• Depth or multi-view geometric constraints to stabilize optimization.
• Ensembling and pixel-wise fusion to compensate for model variance or inherent noise.

For low-light, even a simple uniform gamma correction with mild denoising approaches the performance of complex 2D enhancement pipelines at lower compute cost, while for smoke it is beneficial to jointly learn the transmission ($T_\text{illu}(x)\in[0,1]$1) and airlight ($T_\text{illu}(x)\in[0,1]$2) coefficients within the 3D optimization (Liu et al., 29 Dec 2025, Liu et al., 5 Apr 2026).

7. Limitations and Directions for Extension

RealX3D in its current public release is limited to indoor scenes and four degradation types; the challenge subset encompasses only two adverse conditions (low-light, smoke) across 14 scenes, with no ground-truth meshes provided in the challenge splits (Liu et al., 5 Apr 2026). The benchmark does not yet address rain, snow, or underwater particulates, nor does it include dynamic or non-Lambertian surfaces. Scenes with severe specularities or ultra-dark regions remain failure points for both restoration and reconstruction.

Potential extensions include incorporating real HDR capture, multi-source/moving illumination, outdoor/dynamic environments, and releasing 3D scans to enable direct mesh/point-cloud metric evaluation (e.g., Chamfer, IoU). Research directions highlighted include end-to-end RAW-space restoration tied to pose/depth estimation, generative priors for unmodeled degradations, and robust learning of degradation-invariant features in SfM and multi-view synthesis (Liu et al., 29 Dec 2025).

The RealX3D benchmark thus establishes a reference platform for development and assessment of next-generation, physically-grounded 3D visual restoration and reconstruction methods under adverse, real-world conditions.