GenSmoke-GS: Multi-Stage Smoke Restoration

• GenSmoke-GS is a multi-stage method that synthesizes clean novel views from smoke-degraded multi-view images using physics-inspired dehazing and constrained generative enhancement.
• It employs a pipeline of convolutional restoration, classical Dark Channel Prior dehazing, MLLM-based enhancement, and MCMC-augmented 3D Gaussian Splatting optimization.
• Results on NTIRE 2026 3DRR Track 2 show significant gains in PSNR, SSIM, and LPIPS, demonstrating robust performance over established baselines.

GenSmoke-GS denotes, in its primary published usage, a multi-stage method for novel view synthesis from smoke-degraded multi-view images, introduced for Track 2 of the NTIRE 2026 3D Restoration and Reconstruction Challenge. Its central design principle is to improve per-image visibility while minimally perturbing scene content across views, then optimize a 3D Gaussian Splatting representation with an MCMC-augmented optimizer, and finally average multiple independent optimization runs to reduce residual instability. On the NTIRE 2026 3DRR Track 2 benchmark, the method achieved rank 1 out of 14 participants, with reported gains over the provided baselines in both quantitative metrics and visual quality (Cao et al., 3 Apr 2026). In related literature, the name also appears as a broader point of reference for Gaussian-splatting-based smoke restoration systems, especially when contrasted with geometry-first or explicitly factorized smoke/surface formulations (Jain et al., 22 Sep 2025).

1. Problem setting and physical formulation

GenSmoke-GS addresses the task: given a set of multi-view images captured under smoke-degraded conditions and known camera parameters, synthesize clean, photorealistic novel views of the underlying scene (Cao et al., 3 Apr 2026). The formulation is motivated by three coupled failure modes induced by smoke: reduced visibility, weakened cross-view consistency, and domain shift relative to standard 3DGS or NeRF optimization.

The method adopts the classical single-scattering haze model, written as

$$I(x) = J(x)\,t(x) + A\,(1 - t(x)), \qquad t(x) = e^{-\beta\, d(x)}.$$

Here, $I(x)$ is the observed smoke-degraded intensity, $J(x)$ is the desired haze-free radiance, $A$ is airlight, $t(x)$ is transmission, $\beta$ is the attenuation coefficient, and $d(x)$ is scene depth along the camera ray. Under this model, smoke both attenuates direct radiance and adds scattered airlight, which degrades textures and edges required for reconstruction.

Within multi-view NVS, the principal difficulty is not only low contrast but the erosion of cross-view consistency. Because $t(x)$ varies with viewpoint through $d(x)$, the same scene point can appear with view-dependent attenuation and additive airlight. If preprocessing or generative enhancement introduces view-dependent structural changes, the optimizer receives contradictory supervision, which can yield unstable geometry, biased appearance, or poor local minima. GenSmoke-GS therefore treats visibility enhancement as a constrained preprocessing problem: visibility should improve, but geometry, layout, and object boundaries should remain consistent across views.

2. Multi-stage pipeline and ordering

The GenSmoke-GS pipeline comprises five stages applied in a fixed order (Cao et al., 3 Apr 2026). First, ConvIR-UDPNet is applied to each input image as an image restoration stage. In the described challenge setting, it is used in inference as an off-the-shelf restoration/dehazing model, without reported retraining, and is intended to recover coarse structures, stabilize colors, and suppress heavy artifacts and noise.

Second, the method applies classical Dark Channel Prior dehazing. DCP estimates airlight and transmission and inverts Koschmieder’s law in a physically motivated way. Standard DCP practices are followed to balance visibility and structural fidelity. The transmission estimate is written as

$$\hat{t}(x) = 1 - \omega \cdot \min_{c\in\{r,g,b\}} \left( \min_{y\in\Omega(x)} \frac{I_c(y)}{A_c} \right),$$

followed by edge-aware refinement, and the dehazed radiance is recovered as

$I(x)$0

The stated role of this stage is to reduce airlight and enhance contrast while introducing minimal scene-content changes.

Third, GPT-Image-1.5 is used for MLLM-based enhancement. Its input is the DCP-dehazed image, and its prompting strategy explicitly preserves global geometry, layout, object boundaries, and local structures, while allowing visibility enhancement, denoising, and moderate detail recovery. The prompts prohibit content insertion or removal, shape warping, and color shifts that would break multi-view consistency. The pipeline applies enhancement independently per image and relies on strict prompt constraints to minimize hallucination.

Fourth, the enhanced views supervise a 3DGS-MCMC optimization accelerated by FasterGS. This stage fits a 3D Gaussian Splatting scene representation to the enhanced multi-view set while interleaving gradient updates with MCMC proposals so that the optimizer can escape local minima induced by residual smoke artifacts or inconsistent supervision.

Fifth, the method averages outputs from repeated independent runs. The stated rationale is that stochasticity in the optimizer and in the MCMC procedure yields slightly different local artifacts across runs; averaging dampens these and improves stability and perceptual quality.

The ordering is itself part of the method. Restoration and dehazing are placed first because they remove heavy degradation components with minimal scene-content changes. MLLM enhancement is placed after dehazing so that the generative model operates on cleaner inputs under stricter structural constraints. MCMC is reserved for the scene-optimization stage, where residual inconsistencies are most likely to trap purely gradient-based procedures.

3. 3DGS representation, MCMC optimization, and implementation

GenSmoke-GS models the scene as a set of $I(x)$1 anisotropic 3D Gaussians,

$I(x)$2

where $I(x)$3 is the Gaussian mean, $I(x)$4 is the covariance, $I(x)$5 is opacity, $I(x)$6 is color, $I(x)$7 is per-Gaussian scale, and $I(x)$8 denotes optional SH appearances or view-dependent features (Cao et al., 3 Apr 2026). Each Gaussian projects to the image plane as a 2D elliptical Gaussian, and rendering uses front-to-back alpha compositing:

$I(x)$9

with

$J(x)$0

The optimization objective is defined over the enhanced images $J(x)$1:

$J(x)$2

with optional regularizers $J(x)$3, $J(x)$4, and $J(x)$5, giving

$J(x)$6

Cross-view consistency is not introduced as a separate explicit correspondence term; it is implied by the requirement that the same parameter set $J(x)$7 explain all views simultaneously.

The MCMC component uses Metropolis-Hastings proposals over positions, covariance or scale parameters, opacities, and color or appearance, with optional birth and death moves. Acceptance is governed by

$J(x)$8

Gradient descent or Adam steps are interleaved with periodic MCMC proposals. This design is intended to improve robustness in low-contrast regions and under smoke-induced ambiguities where residual artifacts can distort the local loss landscape.

For repeated-run aggregation, the rendered target view $J(x)$9 from run $A$0 is averaged as

$A$1

In the final submission, the method used $A$2 runs and an optimization length of 30,000 iterations per run. The challenge provides camera intrinsics and extrinsics, so no additional calibration is required. Learning rates, batch sizes, and the initial number of Gaussians follow standard 3DGS or FasterGS defaults, and initialization uses the challenge-provided inputs and camera parameters. The dataset is the RealX3D benchmark used in NTIRE 2026 3DRR Track 2, and code is publicly available at the stated repository.

4. Quantitative evaluation and challenge performance

Evaluation is reported with PSNR, SSIM, and LPIPS on the NTIRE 3DRR Track 2 test set built from RealX3D scenes (Cao et al., 3 Apr 2026). The average performance reported for GenSmoke-GS is PSNR 20.21, SSIM 0.729, and LPIPS 0.446. The provided baselines are reported as follows: 3DGS at PSNR 11.54, SSIM 0.597, LPIPS 0.705; I2-NeRF at PSNR 7.13, SSIM 0.257, LPIPS 0.852; SeaSplat at PSNR 9.00, SSIM 0.440, LPIPS 0.827; and SeaThru-NeRF at PSNR 9.14, SSIM 0.566, LPIPS 0.767. The official challenge ranking places the method first among 14 participants.

Per-scene PSNR and SSIM are reported as follows.

Scene	PSNR	SSIM
Futaba	20.97	0.807
Hinoki	19.12	0.607
Koharu	20.87	0.783
Midori	20.56	0.791
Natsume	20.76	0.745
Shirohana	17.50	0.569
Tsubaki	21.66	0.799
Average	20.21	0.729

The paper characterizes the qualitative improvements as increased visibility, improved contrast, and greater structural stability relative to the baselines. Representative views such as Futaba 0024 and Shirohana 0027 are described as showing crisper edges and fewer artifacts. It further states that large margins over baselines across all metrics indicate robust improvement.

The ablation narrative is qualitative rather than fully tabulated. The restoration and DCP stages are described as improving visibility and stabilizing optimization; MLLM enhancement as improving detail while requiring strict prompts for consistency; MCMC interleaving as improving robustness relative to pure gradient descent; and averaging across 91 runs as reducing local artifacts and improving perceptual metrics.

5. Relation to adjacent smoke-reconstruction paradigms

Within the immediate 3DGS literature, GenSmoke-GS is positioned as complementary to methods that model participating media directly in the renderer. The paper explicitly describes it as orthogonal to standard 3DGS improvements because it preprocesses the inputs to better satisfy 3DGS assumptions and then uses 3DGS-MCMC plus FasterGS for robust optimization under degradation (Cao et al., 3 Apr 2026). It is also described as complementary to methods such as SeaThru-NeRF and SeaSplat, which embed scattering models more directly into the rendering process.

A distinct comparison point is SmokeGS-R, which adopts a geometry-first pipeline rather than a generative enhancement stage. SmokeGS-R generates physics-guided pseudo-clean supervision with a refined dark channel prior and guided filtering, trains a sharp clean-only 3DGS source model on those pseudo-cleans, and harmonizes appearance afterward using donor renderings, geometric-mean aggregation, LAB-space Reinhard transfer, and light Gaussian smoothing (Fu et al., 7 Apr 2026). In that framing, GenSmoke-GS represents a visibility-oriented preprocessing and robust-optimization strategy, whereas SmokeGS-R represents a decouple-then-harmonize strategy.

A further line of related work is SmokeSeer, which uses synchronized or roughly co-located RGB and thermal videos and explicitly decomposes the scene into surface Gaussians and smoke Gaussians (Jain et al., 22 Sep 2025). That system models smoke as a dynamic, semi-transparent participating medium, uses thermal images to anchor geometry and transmittance, and introduces priors for smoke–surface disentanglement. Relative to such an explicit decomposition, GenSmoke-GS operates as an RGB-only preprocessing-plus-optimization pipeline rather than a joint inverse-rendering framework with separate smoke and non-smoke volumetric components.

Taken together, these comparisons place GenSmoke-GS within a broader design space. One end emphasizes preprocessing and optimizer robustness; another emphasizes geometry-first pseudo-clean supervision; another emphasizes explicit factorization of smoke and surfaces with multi-modal sensing. A plausible implication is that these approaches make different trade-offs between physical explicitness, susceptibility to hallucination, and computational burden.

6. Limitations, failure cases, and prospective extensions

The reported limitations of GenSmoke-GS follow directly from its stagewise design (Cao et al., 3 Apr 2026). Under heavy smoke, when $A$3 is large and $A$4 is near zero over wide regions, DCP may struggle to estimate $A$5 and $A$6 accurately; residual haze and low contrast can then persist and destabilize geometry. Parameter-estimation inaccuracies in DCP, such as incorrect $A$7 or overly aggressive $A$8, can produce color or contrast shifts. Refinement mitigates these effects but does not remove them in extreme conditions.

The MLLM stage introduces an additional failure mode. Even with constrained prompts, slight texture or color changes across views can occur; the optimizer may absorb some of these, but stronger hallucinations degrade cross-view consistency. In low-texture or heavily veiled regions, 3DGS itself can produce over-smoothed or floating splats. The paper identifies MCMC as helpful in such cases, but also states that averaging is critical to suppress residuals.

The method therefore exposes a central trade-off: aggressive visibility enhancement can over-sharpen or alter texture statistics, whereas conservative settings preserve consistency but may leave residual haze. This tension is fundamental to any pipeline that improves supervision before multi-view optimization rather than embedding a complete smoke model inside the renderer.

The paper lists several extensions. One is joint dehazing-rendering, in which $A$9, $t(x)$0, and $t(x)$1 would be learned jointly with 3DGS parameters under a scattering-aware renderer and physical priors. A second is the integration of learned priors, including denoising diffusion priors over textures or geometry. A third is uncertainty modeling, such as heteroscedastic photometric losses or Bayesian 3DGS, so that uncertainty from dehazing and MLLM enhancement can guide optimization and MCMC proposals. Related work suggests additional directions—most notably explicit smoke decomposition and multi-modal fusion—but those belong to adjacent systems rather than the published GenSmoke-GS pipeline itself (Jain et al., 22 Sep 2025).

In sum, GenSmoke-GS is best understood as a carefully ordered smoke-restoration and NVS pipeline: physics-inspired restoration and dehazing for stable supervision, constrained MLLM enhancement for visibility recovery with limited structural drift, robust 3DGS-MCMC optimization for degraded multi-view fitting, and repeated-run averaging for variance reduction. Its reported challenge performance establishes that this combination is effective on smoke-degraded RealX3D scenes, while its limitations delineate the boundary between preprocessing-based robustness and fully explicit modeling of participating media.