Spatiotemporal Degradation-Aware 3D Gaussian Splatting for Realistic Underwater Scene Reconstruction

Abstract: Reconstructing realistic underwater scenes from underwater video remains a meaningful yet challenging task in the multimedia domain. The inherent spatiotemporal degradations in underwater imaging, including caustics, flickering, attenuation, and backscattering, frequently result in inaccurate geometry and appearance in existing 3D reconstruction methods. While a few recent works have explored underwater degradation-aware reconstruction, they often address either spatial or temporal degradation alone, falling short in more real-world underwater scenarios where both types of degradation occur. We propose MarineSTD-GS, a novel 3D Gaussian Splatting-based framework that explicitly models both temporal and spatial degradations for realistic underwater scene reconstruction. Specifically, we introduce two paired Gaussian primitives: Intrinsic Gaussians represent the true scene, while Degraded Gaussians render the degraded observations. The color of each Degraded Gaussian is physically derived from its paired Intrinsic Gaussian via a Spatiotemporal Degradation Modeling (SDM) module, enabling self-supervised disentanglement of realistic appearance from degraded images. To ensure stable training and accurate geometry, we further propose a Depth-Guided Geometry Loss and a Multi-Stage Optimization strategy. We also construct a simulated benchmark with diverse spatial and temporal degradations and ground-truth appearances for comprehensive evaluation. Experiments on both simulated and real-world datasets show that MarineSTD-GS robustly handles spatiotemporal degradations and outperforms existing methods in novel view synthesis with realistic, water-free scene appearances.

• The paper introduces a dual-Gaussian framework that disentangles intrinsic scene appearance from spatiotemporal underwater degradations.
• It employs a physically guided degradation module and depth-guided geometry loss to improve reconstruction fidelity under complex lighting and water conditions.
• Experimental results show significant improvements in PSNR, SSIM, and LPIPS, outperforming state-of-the-art methods on both simulated and real underwater benchmarks.

Spatiotemporal Degradation-Aware 3D Gaussian Splatting for Realistic Underwater Scene Reconstruction

Introduction

Underwater scene reconstruction poses substantial technical barriers due to complex, coupled spatiotemporal degradations inherent to aquatic imaging, such as attenuation, backscattering, caustics, and transient flickering—violating the multi-view color consistency assumption foundational to standard neural 3D reconstruction pipelines. Existing 3DGS and NeRF-based underwater techniques typically address only spatial or temporal degradations in isolation, resulting in suboptimal geometry and appearance recovery in environments where both types are prominent. The work "Spatiotemporal Degradation-Aware 3D Gaussian Splatting for Realistic Underwater Scene Reconstruction" (MarineSTD-GS) (2604.23551) directly targets these limitations by introducing a dual-primitives 3DGS framework with explicit spatiotemporal degradation modeling, robust self-supervised disentanglement, and geometry stabilization strategies.

Methodological Innovations

Dual-Gaussian Primitives and Degradation Modeling

MarineSTD-GS constructs two co-located sets of 3D Gaussian primitives: Intrinsic Gaussians encode the hypothesized true scene free of degradation, whereas Degraded Gaussians directly generate the input degraded underwater observations. Critically, the connection between these primitive sets is mediated by a learned, physically guided Spatiotemporal Degradation Modeling (SDM) module, which translates intrinsic appearance into the observed degraded colors for each view and timepoint, incorporating both global (spatial) and local instantaneous (temporal) degradation effects.

Figure 1: Pipeline of MarineSTD-GS, showing SDM's role in mapping intrinsic properties to per-frame degradations with depth-guided and multi-stage optimization.

The SDM module is defined around a physically motivated color degradation equation, incorporating water parameters (attenuation and backscatter coefficients, ambient light) and a localized, Gaussian-specific illumination perturbation term that models temporal variation (e.g., due to caustics or flickering). Water parameters are estimated scene-wide per frame (SD branch), leveraging degraded images enhanced via estimated depth. The temporal/nodal perturbation (TD branch) is inferred for each Gaussian, utilizing local and global instantaneous brightness features.

Figure 2: Architectures for spatial (SD) and temporal (TD) degradation prediction branches within SDM, showing dedicated pathways and feature encodings.

Supervision and Optimization

Supervision is fully self-supervised on the degraded input, combining:

• Photometric Loss: $L_1$ and D-SSIM between rendered and input degraded image for appearance fidelity.
• Depth-Guided Geometry Loss (DGGL): Leverages robust monocular depth priors for global and local geometry alignment, using both Pearson correlation (scale/shift invariance) and adaptive edge-aware smoothness; crucial for overcoming photometric cue degradation.

Depth priors are incorporated using recent robust monocular predictors (e.g., Depth-Anything-V2), providing stability even under strong caustics.

• Transient Regularization: A penalty ($L_2$) on TD branch outputs prevents overfitting to transient artifacts.

Optimization proceeds in three stages: (1) TD branch frozen for SD-only steady-state, (2) alternating SD/TD parameter updates for disentanglement, (3) joint finetuning for maximum appearance realism and stability.

Ablation and Architecture Analysis

Ablations show significant degradations in appearance and geometry when omitting either SD or TD components, depth guidance, transient regularization, or multi-stage optimization.

Figure 3: Visual comparisons of novel view synthesis quality under various ablation settings; detachment from depth guidance or regularization reduces realism and fidelity.

Figure 4: Effectiveness of depth-guided supervision with pseudo-depth, improving geometric consistency under spatiotemporal degradation.

Experimental Protocol and Results

Simulated Benchmarks

A synthetic benchmark with controllable spatial and temporal degradations (three scene types, multi-level attenuation, multiple caustics patterns) is introduced, providing paired ground truth and degraded renderings.

The proposed method achieves dominant scores in PSNR/SSIM/LPIPS across all scene types, outperforming recent baselines: e.g., +3.66 dB PSNR over SeaSplat on S1 (detailed textures), with even greater margins under severe spatial degradations (+8.98 dB on S3).

Figure 5: Qualitative novel view synthesis on simulated scenes, with MarineSTD-GS removing color distortions and overexposures, approaching ground truth.

Figure 6: Robustness of MarineSTD-GS under different spatial degradation levels—appearance stability and color fidelity persist even for severe degradation.

Real-World Generalization

On public real-world benchmarks (BVICoral, Flsea_VI, SeaThru), MarineSTD-GS achieves the lowest $\Delta E_{00}$ color difference and angular error, leading all prior art, including those optimized for underwater scenarios. The method is effective in both clear and highly degraded scenes, consistently suppressing haze, correcting color casts, and mitigating illumination artifacts.

Figure 7: Qualitative synthesis results on real underwater datasets, with clear suppression of haze and illumination artifacts.

Temporal Pattern Robustness

Quantitative evaluation under diverse temporal patterns (different caustics/flickering) indicates no significant drop in reconstruction accuracy, demonstrating the efficacy of the TD branch's explicit temporal modeling.

Disentangled Rendering and Applications

Because the framework disentangles scene and water effects, it supports novel rendering paradigms—injecting arbitrary water parameter sets for controllable water effect transfer across scenes, enabling both stable underwater view synthesis and data generation for simulation.

Figure 8: Disentangled scene and water representation applications—novel view synthesis and cross-scene water condition transfer.

Theoretical and Practical Implications

The explicit modeling and disentanglement of spatiotemporal degradations within 3DGS present a paradigm shift for physically grounded scene reconstruction in challenging domains. The proposed dual-Gaussian and SDM-based architecture demonstrates that complex, nonstationary degradations can be robustly factored out from true surface appearance and geometry by leveraging modern depth priors and carefully staged optimization. Moreover, the decoupled models facilitate further scientific and practical applications: e.g., data augmentation for marine robotics, virtual reality with environment reconfigurability, and generative simulation for vision research.

Future research directions include expanding such factorized degradation-aware representations to in vivo dynamic aquatic scenes, multi-modal input (e.g., sonar fusion), real-time operation, and cross-media domains such as atmospheric fog, where similar degradation couplings confound standard multi-view learning methods. Integration with generative scene priors or physics-based differentiable simulators may provide further gains in cases where geometry or lighting cues are even more severely degenerated.

Conclusion

MarineSTD-GS directly addresses the crucial problem of coupled spatiotemporal degradation in underwater scene reconstruction, establishing a new state-of-the-art in both simulated and real-world settings. Its dual-primitives 3DGS architecture, physically grounded spatiotemporal degradation modeling, depth-guided geometric supervision, and multi-stage optimization enable robust disentanglement of scene and water parameters. The method provides highly accurate, degradation-agnostic reconstructions and supports flexible water effect simulation, thus expanding the practical and theoretical capacity of 3DGS frameworks in adverse visual domains (2604.23551).