3DGS-Enhancer for Robust 3D Gaussian Splatting

• 3DGS-Enhancer is a class of pipelines that improve 3D Gaussian Splatting by integrating video diffusion, degradation-aware learning, and geometry-texture strategies.
• It enhances reconstruction fidelity in sparse-view, low-light, and noisy scenarios by mitigating artifacts and ensuring view consistency.
• Empirical improvements include increased PSNR, SSIM gains, and lower LPIPS errors while preserving real-time rendering performance.

3DGS-Enhancer

3DGS-Enhancer refers to a class of pipelines and frameworks designed to improve the reconstruction quality, fidelity, and robustness of 3D Gaussian Splatting (3DGS)-based scene representations, especially in challenging regimes such as sparse-view novel-view synthesis, low-light environments, or in the presence of significant sensor noise. While multiple works employ the term “3DGS-Enhancer” to denote specific architectures or enhancement strategies, all target the systematic mitigation of core 3DGS limitations—namely, under-sampling artifacts, view inconsistency, and poor robustness to real-world image degradations. State-of-the-art variants integrate explicit 2D video diffusion priors, geometry- and texture-aware densification, multi-view degradation learners, and noise-robust optimization within the 3DGS framework to establish new levels of realism and photorealistic performance, notably without sacrificing real-time capabilities.

1. Reconstruction Challenges in Sparse and Degraded Regimes

Unbounded 3DGS-based novel-view synthesis under sparse input views is inherently ill-posed: large regions may be never directly observed, causing standard 3DGS or NeRF reconstructions to converge to local minima, such as “hollow” or “ellipsoid” surface artifacts or hallucinated inconsistent textures. Moreover, 3DGS is particularly vulnerable to raw sensor noise and non-ideal imaging conditions (e.g., low light, motion blur, weather-induced degradations), compounding instability and visual artifacts (Liu et al., 2024, Li et al., 2024, Wu et al., 5 Aug 2025).

Empirical evaluations on large-scale, unbounded datasets (e.g. DL3DV, Mip-NeRF360) document severe under-sampling and loss of global coherency in baseline 3DGS under 3-6 view regimes, yielding PSNR values as low as 10–11 dB and high LPIPS errors (>0.5), even as 3DGS maintains fast rendering throughput (Liu et al., 2024).

2. Incorporating 2D Video Diffusion Priors for View-Consistent Enhancement

3DGS-Enhancer (Liu et al., 2024) introduces a modular pipeline for enhancing radiance field quality, formulated as:

• Stage 0: Low-quality 3DGS (LQ-3DGS) is first trained on sparse reference images to yield the initial (weak) set of Gaussians.
• Stage 1: For each pair of reference cameras, a temporally-interpolated sequence of views is rendered via LQ-3DGS, producing a low-quality “video sweep”. Latent codes for this video are encoded via a VAE, then enhanced using a video diffusion model (e.g., SVD) with additional cross-attention to CLIP embeddings. Only the T intermediate novel views are diffused; the two real reference images are used as “hard anchors” to impose temporal/view consistency.
• Stage 2: The spatial-temporal decoder (STD) is used to obtain high-quality, temporally-consistent video frames, via temporal convolutions and controllable feature warping against original renders. Each enhanced novel view is aligned in color statistics to the reference frames.

These enhanced views, now containing view-consistent synthetic content, are integrated as additional pseudo-observations in a subsequent fine-tuning phase of the 3DGS model. The photometric loss in this stage is both pixel- and image-confidence weighted; pixel-level confidence is derived from the splatted Gaussian size, and image-level confidence from pose novelty.

Empirical results document a PSNR increase across DL3DV (3-view) from 10.97 dB (baseline) to 14.33 dB using 3DGS-Enhancer, with corresponding SSIM and LPIPS gains. Similar improvements are recorded on Mip-NeRF360 (6-view), with PSNR increasing from 11.53 dB to 13.96 dB (Liu et al., 2024).

3. Multi-View Robustness and Degradation-Aware Feature Enhancement

RobustGS (Wu et al., 5 Aug 2025) (interpreted as a 3DGS-Enhancer for feedforward pipelines) approaches enhancement by inserting a lightweight dual branch:

• The Generalized Degradation Learner (GenDeg) encodes unknown per-image degradations into a compact vector, trained by reconstruction, contrastive, and classification objectives using synthetic degradations.
• The Multi-View State-Space Enhancement Module (MV-SSEM) performs feature-level restoration and semantic-aware multi-view fusion, including semantic reordering and state-space scanning.

This module is agnostic to the 3DGS backbone (e.g., works with PixelSplat, MVSplat) and is pretrained on synthetic degradations, then applied in a plug-and-play manner without retraining the 3DGS backbone. Under compound degradations (noise, rain, fog, low light), RobustGS-enhanced models realize up to 3–5 dB PSNR improvement and a 0.04–0.06 SSIM gain over two-stage “Restore→Reconstruct” baselines, with minimal overhead (∼50 ms per 2×64×64 input) (Wu et al., 5 Aug 2025).

Ablation studies establish that omitting GenDeg, semantic reordering, or multi-view fusion each causes systematic PSNR drops, affirming the significance of both global degradation guidance and cross-view semantic aggregation.

4. Geometry- and Texture-Aware Densification Enhancement Strategies

GeoTexDensifier (Jiang et al., 2024) (presented as a 3DGS-Enhancer mechanism) targets quality loss in weakly-textured or poorly-observed zones by combining:

• Texture-aware densification, where splitting operations are prioritized in high-gradient image regions, producing denser splat populations wherever rich texture is detected, while avoiding over-splitting on uninformative patches.
• Geometry-aware splitting, in which depth- and normal-guided candidate splats are pruned via a validation of depth ratio change (VDRC) check against monocular priors. Splats straying significantly from the surface are eliminated.

Quantitative benchmarks across indoor and outdoor scenes demonstrate consistent gains in SSIM, PSNR, and LPIPS compared to vanilla 3DGS. For example, on “Room” scenes: 3DGS (SSIM/PSNR/LPIPS) 0.919 / 31.41 dB / 0.219 vs. GeoTexDensifier 0.927 / 31.79 dB / 0.196 (Jiang et al., 2024).

5. Noise-Robust and Low-Light 3DGS-Enhancer Architectures

In the low-light and sensor noise regime, 3DGS-Enhancer (Li et al., 2024) integrates:

• A dedicated noise extractor branch ($F_n$), structured as a compact U-Net and pretrained on extreme low-light raw/noisy pairs, predicting a dense noise map consistent with a heteroscedastic sensor noise prior.
• A noise-robust 3DGS reconstruction, optimizing splats with a loss that jointly penalizes reconstruction error (weighted RawNeRF loss), noise model negative log likelihood, and spatial noise decorrelation.

This design suppresses the formation of “hairy” elongated Gaussians and maintains sharper geometry in HDR conditions. In full-view RawNeRF benchmarks, 3DGS-Enhancer achieves Raw PSNR 59.49 dB, RGB PSNR 23.53 dB, SSIM 0.535, and LPIPS 0.499, outperforming both LDR/HDR Scaffold-GS and all denoiser+3DGS baselines without sacrificing ∼80 FPS real-time inference (Li et al., 2024).

Qualitative ablations confirm that removing noise extraction or covariance regularization degrades PSNR by 1.5–3.1 dB and returns to baseline “hairy” artifacts.

6. Practical Considerations, Limitations, and Outlook

3DGS-Enhancer systems are modular, typically requiring no changes to underlying splatting/rasterization code: video diffusion priors are invoked only during enhanced supervision generation, while feature- and geometry-based enhancements operate in explicit pre/post-processing or intermediate feature domains. In practice, integration protocol involves:

• Pretraining or inference-time deployment of diffusion or degradation modules, with plug-and-play application to existing 3DGS pipelines.
• Confidence-controlled fine-tuning, which leverages both reference and denoised/synthesized views with pixel-level weighting.
• Recourse to monocular depth and normal priors for robust geometry validation.

Limitations include increased training time (due to diffusion, extra optimization, and/or noise estimation), modest increases in memory footprint (for densification), and residual over-smoothing or artifact suppression in high-frequency texture zones. Outlier regions where diffusion priors or monocular guidance fail (e.g., strong reflections, extreme occlusions) may still challenge the current frameworks (Liu et al., 2024, Li et al., 2024, Jiang et al., 2024).

Future research aims encompass end-to-end joint training of diffusion and 3DGS, unsupervised confidence weighting, scalable chunked reconstruction for very large scenes, and richer multi-scale priors for better discrimination between signal and noise.

7. Quantitative Summary of Enhancement Performance

Method	Domain	PSNR Gain	SSIM Gain	LPIPS Drop	Notable Properties
3DGS-Enhancer (Liu et al., 2024)	Sparse novel-view	+3.36 dB (DL3DV, 3-view)	+0.176	–0.103	Video diffusion + fine-tune
GeoTexDensifier (Jiang et al., 2024)	Multi-view	+0.38 dB	+0.008	–0.023	Texture + geometry densify
Noise-Robust (Li et al., 2024)	Low-light/HDR	+0.84 dB	+0.014	–0.014	Noise-aware denoise/fit
RobustGS (Wu et al., 5 Aug 2025)	Degraded images	+3–5 dB	+0.05	–	Degradation, semantic fusion

These results indicate that 3DGS-Enhancer frameworks systematically improve fidelity and robustness across a spectrum of adverse or under-constrained scenarios.

---

References

• "3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors" (Liu et al., 2024)
• "GeoTexDensifier: Geometry-Texture-Aware Densification for High-Quality Photorealistic 3D Gaussian Splatting" (Jiang et al., 2024)
• "From Chaos to Clarity: 3DGS in the Dark" (Li et al., 2024)
• "RobustGS: Unified Boosting of Feedforward 3D Gaussian Splatting under Low-Quality Conditions" (Wu et al., 5 Aug 2025)