NTIRE 2026: 3D Restoration & Reconstruction Challenge

• The challenge is a competition evaluating advanced multi-view 3D restoration and reconstruction pipelines under real-world low-light and smoke conditions.
• It leverages the RealX3D benchmark to provide paired low-quality and ground truth images, ensuring precise photometric and geometric evaluation.
• Evaluation metrics like PSNR and SSIM demonstrate the impact of physics-informed, multi-stage methods on overcoming degradation-induced failures.

The NTIRE 2026 3D Restoration and Reconstruction Challenge (3DRR) is a competition designed to rigorously evaluate and advance multi-view 3D scene restoration and reconstruction methods under real-world adverse degradations. Utilizing the RealX3D benchmark of paired degraded and clean image captures with aligned geometry, the challenge focuses primarily on two extreme degradation tracks: low-light (illumination-limited) and smoke-scattering environments. Drawing 279 registrants and 33 final team submissions, the competition provides a comprehensive assessment of state-of-the-art pipelines under conditions known to destabilize conventional multi-view geometry and rendering techniques (Liu et al., 29 Dec 2025, Liu et al., 5 Apr 2026).

1. RealX3D Benchmark and Protocol

RealX3D underpins the NTIRE 2026 3DRR Challenge, offering pixel-aligned low-quality (LQ) and ground truth (GT) images for each scene, spanning multiple severe-realistic degradation families. The acquisition protocol standardizes capture and alignment:

• Camera Platform: Sony A7 IV with 24–70 mm f/2.8 GM, stabilized with a DJI RS4 gimbal and a programmable rail dolly (≈1 m radius, ~400 frames/trajectory) or matched tripod setup for stationary scenes.
• Geometry Ground Truth: High-resolution (up to 7008×4672 px), 16-bit RAW and sRGB outputs per view. Leica BLK360 G2 high-end laser scanner is used with ≥5 scans per scene in HDR; 5 mm subsampled fusions yield ground-truth meshes and metric depth.
• Alignment and Pose: GT captures precede degraded ones along identical paths to guarantee pixel-wise LQ/GT pairing; camera intrinsics/extrinsics are calibrated and registered with COLMAP, refined by ICP against laser scans (RMS error ≈1.2 cm).
• Correlation: This facilitates direct supervised learning and evaluation in both photometric and geometric domains (Liu et al., 29 Dec 2025).

2. Degradation Taxonomy and Dataset Structure

RealX3D structures degradations into four physically-derived families, each instantiated with continuous severity gradations:

• Illumination: Controlled low-light (shutter at 1/400 s) and varying exposure scenarios (multiple shutter speeds: 1/60 – 1/400 s, spanning ≈0–+2.7 EV), yielding noisy, color-shifted LQ images down to 1–5 lux.
• Scattering: Smoke is introduced using a 1200 W machine for multi-level atmospheric scattering in a controlled studio, capturing effects that exceed classical single-scattering models.
• Occlusion: Combination of static/dynamic blockers and glass-plate reflections renders standard view-synthesis and correspondence unstable.
• Blurring: Defocus (mild to strong, lens misfocused to 0.6 m/0.4 m vs 3–5 m) and in situ motion blur (exposure-integrated radiance, 2–5 cm).
• Composition: 55 scenes; each view provides both RAW and ISP-processed sRGB data, dense laser scans, and rendered metric depth (Liu et al., 29 Dec 2025). For the 3DRR Challenge, subsets are carefully selected to ensure paired clean GT and LQ in both training and evaluation splits, focusing on low-light and smoke tracks (Liu et al., 5 Apr 2026).

3. Evaluation Metrics and Ranking

The principal evaluation in the NTIRE 2026 3DRR Challenge is image-based, considering only the rendered novel-view synthesis (NVS) quality:

• Photometric Quality:
  • Peak Signal-to-Noise Ratio (PSNR):

  $$\mathrm{PSNR} = 10\log_{10}\left(\frac{L^2}{\mathrm{MSE}}\right), \quad L=1.0$$ - Structural Similarity Index (SSIM):

  $$\mathrm{SSIM}(x,y) = \frac{(2\mu_x\mu_y + C_1)(2\sigma_{xy} + C_2)} {(\mu_x^2 + \mu_y^2 + C_1)(\sigma_x^2 + \sigma_y^2 + C_2)}$$

• (Not used for leaderboard ranking)

  • Chamfer Distance (point clouds), Intersection over Union (IoU) for voxel/point data.
• Ranking: Teams are ranked by mean PSNR across NVS views (SSIM as tiebreaker), no composite metric.
• Ground Truth: For evaluation, only photometric metrics relative to pixel-aligned clean NVS ground truth are used—no direct geometry (e.g., mesh) supervision (Liu et al., 5 Apr 2026).

4. Challenge Tracks, Baselines, and Results

Competition Tracks

• Track 1: Low-Light Enhancement
  • Scenes: 7 per track (indoor, outdoor, mixed).
  • Degradation: 1–5 lux exposures, strong color and noise artifacts.
• Track 2: Smoke Restoration
  • Scenes: 7 per track, multi-level smoke densities.

Each track: ~30 degraded training views (known poses), 1–5 evaluation views per scene. “Development” split includes one fully supervised pair and four LQ-only scenes. Blind test split includes three scenes without GT (Liu et al., 5 Apr 2026).

Baselines

• Naïve approaches: Direct NeRF or 3D Gaussian Splatting (3DGS) on degraded images; or per-frame enhancement (Zero-DCE, DCP) pre-processing.
• Reference algorithms: Physics-based priors (Naka-Rushton for low-light, atmospheric scattering for smoke), Retinexformer enhancement.
• Standard 3DGS: Fails under extreme degradation—PSNR < 11 dB (low-light), ≈ 7–8 dB (smoke); SOTA methods required bespoke adaptation (Liu et al., 29 Dec 2025, Liu et al., 14 Apr 2026).

Results

Track	1st Place	PSNR↑	SSIM↑	2nd Place	PSNR↑	SSIM↑	3rd Place	PSNR↑	SSIM↑
Low-Light Enhancement	FuME-GS	23.38	0.80	CISP-GS	22.78	0.78	TCIDNet-IBGS	21.61	0.71
Smoke Restoration	GenSmoke-GS	20.21	0.73	Smoke-GS	18.67	0.69	Dehaze-then-Splat	18.38	0.66

• ELoG-GS (NTIRE submission): PSNR 18.66, SSIM 0.69, outperforming all previously published Luminance-GS and LITA-GS (Liu et al., 14 Apr 2026).

5. Methodological Advances and Pipeline Design

Top teams converged on multi-stage, restoration-first pipelines, harmonizing image enhancement (denoising, dehazing) with robust multi-view 3D reconstruction:

• Enhancement–Reconstruction Cascade: Two-stage or multi-branch first enhances per-frame degraded images, then reconstructs the 3D scene using modified 3DGS backbones.
• Physics-informed Models: Naka-Rushton transforms for low-light, deep atmospheric scattering modules (including MLP-predicted medium parameters and analytical scattering models) for smoke.
• Monocular Depth Anchors: Methods such as ELoG-GS use VGGT or ZoeDepth to initialize geometry and regularize fitting (Liu et al., 14 Apr 2026).
• Multi-model & Region-wise Fusion: E.g., FuME-GS implements patch-based adaptive fusion to combine strengths of deep/analytical restoration models, suppressing single-method failure modes.
• Frequency and Attention Mechanisms: Dual attention streams (e.g., in TCIDNet-IBGS), frequency-split branch fusion (e.g., CISP-GS in YCbCr), and luminance-guided loss weighting.

A representative workflow (ELoG-GS, (Liu et al., 14 Apr 2026)):

1. Zero-shot image restoration (Retinexformer).
2. Monocular depth estimation (VGGT); dense voxel fusion to produce $\mathcal{P}_0$.
3. Dual-branch 3DGS fitting: (A) random global initialization; (B) geometry-init with COLMAP/ZoeDepth splits; parallel optimization, with PSNR/SSIM-based selection.
4. Luminance-guided color enhancement: histogram matching, gamma/brightness scaling.

Loss:

$$\mathcal{L} = \mathcal{L}_{\mathrm{photo}} + \lambda_{\mathrm{geo}}\mathcal{L}_{\mathrm{geo}} + \lambda_{\mathrm{reg}}\mathcal{L}_{\mathrm{reg}}$$

with luminance-adaptive weightings and geometric depth constraints.

6. Shared Insights, Limitations, and Directions

Analysis of submissions and results reveals several key findings:

• Physics-driven Restoration is Essential: Domain-specific priors (scattering, response curves, RAW/ISP adaptation) undergird successful methods for severe degradations.
• Multi-model Fusion Reduces Failure: Ensemble and region-wise fusion mitigate the instability of any one restoration/synthesis process under non-uniform degradations.
• Geometric Anchoring via Depth Priors: Monocular or pseudo-depth anchors (VGGT, ZoeDepth, Depth-Anything V2) stabilize optimization, especially in few-view or degenerate regimes.
• Progressive/Adaptive Optimization: Schedulers for Gaussian splat densities, spherical harmonics ramp-up, and early stopping are critical for floaters and over-densification management.
• Remaining Challenges: Very dense smoke (multi-scattering), severe under-exposure, and limited-view scenarios remain open problems. Direct geometry supervision (e.g., using 3D scans) is absent, motivating benchmark augmentation.

This suggests future progress will require: differentiable, self-supervised integration of restoration and reconstruction; compact, learned physical degradation models; and runtime-efficient deployments for AR/robotics (Liu et al., 5 Apr 2026, Liu et al., 29 Dec 2025).

7. Comparison and Outlook

Compared to previous state-of-the-art, NTIRE 2026 methods deliver 3–6 dB PSNR and up to 0.15 SSIM improvement under real-world degradation. However, all methods still exhibit notable quality drops (2–3× in depth error, 20–40% in F-score) versus clean benchmarks.

A plausible implication is that achieving photorealistic, geometrically sound 3D reconstruction under authentic adverse conditions is fundamentally limited by current restoration and correspondence establishment under multi-factorial degradations. The NTIRE 2026 Challenge, built on RealX3D, thus constitutes a pivotal testbed for progressing degradation-robust multi-view 3D vision pipelines, with an explicit call for unified, physically-grounded, and end-to-end approaches incorporating both enhancement and geometry (Liu et al., 29 Dec 2025, Liu et al., 5 Apr 2026, Liu et al., 14 Apr 2026).