Dehaze-then-Splat: Generative Dehazing with Physics-Informed 3D Gaussian Splatting for Smoke-Free Novel View Synthesis

Abstract: We present Dehaze-then-Splat, a two-stage pipeline for multi-view smoke removal and novel view synthesis developed for Track~2 of the NTIRE 2026 3D Restoration and Reconstruction Challenge. In the first stage, we produce pseudo-clean training images via per-frame generative dehazing using Nano Banana Pro, followed by brightness normalization. In the second stage, we train 3D Gaussian Splatting (3DGS) with physics-informed auxiliary losses -- depth supervision via Pearson correlation with pseudo-depth, dark channel prior regularization, and dual-source gradient matching -- that compensate for cross-view inconsistencies inherent in frame-wise generative processing. We identify a fundamental tension in dehaze-then-reconstruct pipelines: per-image restoration quality does not guarantee multi-view consistency, and such inconsistency manifests as blurred renders and structural instability in downstream 3D reconstruction.Our analysis shows that MCMC-based densification with early stopping, combined with depth and haze-suppression priors, effectively mitigates these artifacts. On the Akikaze validation scene, our pipeline achieves 20.98\,dB PSNR and 0.683 SSIM for novel view synthesis, a +1.50\,dB improvement over the unregularized baseline.

• The paper introduces a two-stage pipeline that first restores smoky images using Nano Banana Pro and then applies physics-informed 3D Gaussian Splatting for novel view synthesis.
• It leverages auxiliary losses such as depth supervision, dark channel prior, and dual-source gradient matching to enforce cross-view photometric and geometric consistency.
• Empirical results on the Akikaze scene demonstrate improved PSNR and SSIM, underlining the method's effectiveness for AR/VR applications and digital twin reconstructions.

Dehaze-then-Splat: Generative Dehazing and Physics-Informed 3D Gaussian Splatting for Multi-View Smoke-Free Reconstruction

Problem Setting and Motivation

This paper addresses multi-view smoke and haze removal for downstream novel view synthesis (NVS) and 3D scene reconstruction. The task demands not only high-fidelity per-image restoration but strict multi-view photometric and geometric consistency to ensure physically plausible novel-view generation. Leveraging the RealX3D benchmark, which provides real-world scenarios with smoke, haze, and low-light, the authors develop a robust pipeline for the NTIRE 2026 3D Restoration and Reconstruction Challenge.

The central insight is the inherent tension between single-image generative restoration quality and multi-view consistency. While generative models such as Nano Banana Pro excel at per-view smoke removal, their lack of cross-view conditioning introduces stochastic photometric and structural variations across frames. These inconsistencies degrade the reconstruction capabilities of subsequent geometric pipelines such as 3D Gaussian Splatting (3DGS), leading to blurred renders and structural artifacts.

Pipeline Overview

The proposed Dehaze-then-Splat pipeline comprises two sequential stages:

1. Generative Dehazing: Each smoky input view undergoes restoration through Nano Banana Pro, a generative model with state-of-the-art dehazing ability. The process includes explicit prompt engineering for optimal removal and photorealistic preservation, followed by pixel-accurate resolution alignment and frame-wise brightness normalization, using ground truth statistics where available.
2. Physics-Informed 3DGS Training: Restored views are fed to a 3D Gaussian Splatting pipeline augmented with auxiliary losses: depth supervision (via scale-invariant Pearson correlation with pseudo-depth), dark channel prior (DCP) regularization, and dual-source gradient matching. These priors enforce geometric and photometric constraints, compensating for inconsistencies from generative restoration.

Multi-View Consistency Analysis

The pipeline’s novelty lies in the explicit identification and mitigation of the multi-view consistency gap induced by independent generative dehazing. Even after global brightness normalization, local color and texture variations persist across adjacent views.

Figure 1: Per-frame dehazing quality versus multi-view consistency. Red boxes denote regions of pronounced color shift and texture drift in Nano Banana Pro outputs, highlighting inconsistencies that degrade downstream 3DGS.

When 3DGS is trained on inconsistent pseudo-clean images, Gaussian kernels are broadened to average across conflicting signals, resulting in loss of detail and artifacts such as floaters and color bleeding. Early-stage model checkpoints empirically yield optimal NVS results, as later densification increases capacity for overfitting to per-view noise.

Auxiliary Physics-Informed Losses in 3DGS

The paper systematically introduces physics-informed regularizers into 3DGS:

• Depth Supervision (most impactful, +0.79 dB): Generated pseudo-depth maps (via Depth Anything V2) anchor geometric structure across views, preventing drift induced by local texture inconsistencies.
• Dark Channel Prior (DCP): Clean images exhibit near-zero dark channel values; regularization suppresses residual haze and improves perceptual fidelity.
• Dual-source Gradient Matching: Edge structure is supervised against MB-TaylorFormer output, leveraging its structural reliability.
• MCMC Densification and Early Stopping: MCMC injects stochasticity and improves sampling efficiency, while early stopping (DENSIFY_STOP_STEP=3k) prevents over-parameterization and floaters.

Empirical ablation confirms synergistic benefits, with depth supervision and MCMC yielding significant PSNR improvements. Early checkpoint selection remains critical for preserving cross-view coherence.

Figure 2: Training progression without early stopping reveals catastrophic degradation, with Gaussian count explosion and severe structural artifacts after extended training.

Quantitative and Qualitative Results

The paper reports strong numerical results: on the Akikaze validation scene, the full pipeline achieves 20.98 dB PSNR and 0.683 SSIM, outperforming the unregularized baseline by +1.50 dB. Per-frame dehazing with Nano Banana Pro (after GT normalization) achieves 20.07 dB, with 3DGS further compensating for residual noise. End-to-end scattering decomposition approaches fail to produce adequate restoration quality, confirming the necessity of hybrid regularization.

Figure 3: 2D dehazing comparison on Akikaze; Nano Banana Pro with GT normalization achieves highest per-frame PSNR, outperforming MB-TaylorFormer and DCP.

Qualitative comparisons reveal sharp, accurate renders from the full pipeline, with baseline approaches showing noticeable blur and color drift, and end-to-end methods unable to separate smoke from scene content.

Figure 4: Synthesis comparison for novel views; full pipeline yields sharp, color-accurate renders free of floater artifacts; baseline and route B suffer from blur and failed separation.

Implications and Future Directions

The research rigorously demonstrates that per-image restoration quality does not suffice for multi-view reconstruction when generative models operate independently per frame. Physics-informed priors and capacity control in the geometric stage are essential for suppressing artifacts and maintaining fidelity. Both practical and theoretical implications arise:

• Practical: The pipeline offers robust restoration and reconstruction in challenging real-world smoky scenes, applicable to digital twins, AR/VR, and film restoration.
• Theoretical: The study highlights the need for cross-view conditioning in generative restoration—future models should incorporate multi-view constraints, perhaps via video-based prompts or self-consistent optimization.

Potential future developments include consistency-aware multi-view generative models, test-time consistency optimization, adaptive scheduling of geometric capacity, and integration with advanced video restoration architectures.

Conclusion

The paper establishes Dehaze-then-Splat as a definitive pipeline for multi-view smoke removal and NVS, revealing an essential tension between generative single-view restoration and multi-view geometric consistency. Physics-informed regularization and early densification are shown to be effective in bridging this gap, yielding superior numerical results and artifact-free renders. Future work should focus on integrating direct cross-view constraints into generative restoration models to fully harmonize 2D restoration with 3D reconstruction.