SSD-GS: Scattering and Shadow Decomposition for Relightable 3D Gaussian Splatting

Abstract: We present SSD-GS, a physically-based relighting framework built upon 3D Gaussian Splatting (3DGS) that achieves high-quality reconstruction and photorealistic relighting under novel lighting conditions. In physically-based relighting, accurately modeling light-material interactions is essential for faithful appearance reproduction. However, existing 3DGS-based relighting methods adopt coarse shading decompositions, either modeling only diffuse and specular reflections or relying on neural networks to approximate shadows and scattering. This leads to limited fidelity and poor physical interpretability, particularly for anisotropic metals and translucent materials. To address these limitations, SSD-GS decomposes reflectance into four components: diffuse, specular, shadow, and subsurface scattering. We introduce a learnable dipole-based scattering module for subsurface transport, an occlusion-aware shadow formulation that integrates visibility estimates with a refinement network, and an enhanced specular component with an anisotropic Fresnel-based model. Through progressive integration of all components during training, SSD-GS effectively disentangles lighting and material properties, even for unseen illumination conditions, as demonstrated on the challenging OLAT dataset. Experiments demonstrate superior quantitative and perceptual relighting quality compared to prior methods and pave the way for downstream tasks, including controllable light source editing and interactive scene relighting. The source code is available at: https://github.com/irisfreesiri/SSD-GS.

• The paper introduces a four-term reflectance decomposition (diffuse, specular, SSS, shadow) that enhances relighting accuracy in 3D Gaussian splatting.
• The progressive training strategy and online parameter refinement achieve robust disentanglement and superior PSNR/SSIM performance on both real and synthetic datasets.
• The method’s physically-grounded design improves material editability and interactivity, promising advances for AR/VR and digital human applications.

Physically-Based Relightable 3D Gaussian Splatting via Scattering and Shadow Decomposition

Introduction

The paper "SSD-GS: Scattering and Shadow Decomposition for Relightable 3D Gaussian Splatting" (2604.13333) introduces a physically-driven relighting framework based on 3D Gaussian Splatting (3DGS). The proposed method, SSD-GS, addresses the inherent limitations of prior 3DGS-based relighting techniques, particularly their inability to disentangle material and illumination effects in scenes involving complex light transport (e.g., anisotropic metals, translucent media). The framework builds a decomposable reflectance model—comprising diffuse, specular, subsurface scattering, and shadow components—thereby enabling accurate, interpretable, and relightable reconstructions under novel illumination conditions.

Methodology

Reflectance Model Decomposition

SSD-GS explicitly decomposes per-Gaussian appearance into four physically-motivated terms:

• Diffuse: Modeled with a standard Lambertian BRDF for stable, low-frequency reflectance.
• Specular: Realized via a Fresnel-modulated anisotropic Spherical Gaussian (ASG) basis, supporting high-frequency, directional reflectance and anisotropy.
• Subsurface Scattering (SSS): Integrated using a learnable neural module grounded in the classical dipole diffusion approximation. The network predicts per-Gaussian scattering parameters, allowing robust, geometry-aware simulation of multiple-scattering in translucent media.
• Shadow: Decomposed into a two-stage process: a continuous volumetric transmittance computation along light rays for shadow cues, followed by neural refinement of the per-Gaussian visibility based on local geometry and material features.

This physically-structured decomposition replaces the less interpretable, low-frequency spherical harmonics (SH) color model used in prior 3DGS variants, substantially increasing fidelity under relighting.

Progressive Training Strategy

To ensure disentanglement and convergence, SSD-GS introduces the reflectance components progressively through a coarse-to-fine curriculum. Components are enabled sequentially (diffuse → shadow → SSS → specular), aligning gradient dynamics and stabilization of low- and high-frequency terms during training. Additionally, the approach supports online refinement of camera and lighting parameters, enhancing generalization and robustness, especially under calibration noise in real-world OLAT datasets.

Experimental Analysis

Benchmarks

SSD-GS is systematically evaluated on both OLAT-captured real datasets (from NRHints) and synthetic benchmarks (from GS3 and SSS-GS). The test protocol assesses reconstruction quality on training views as well as relighting under unseen illumination.

Quantitative Results

Strong numerical superiority (in PSNR, SSIM, and LPIPS) is demonstrated over contemporary 3DGS baselines (3DGS, GI-GS, GS3, RNG) and on SSS-oriented datasets relative to KiloOSF and SSS-GS. Notably, SSD-GS achieves:

• On SSS-GS synthetic test sets: $\mathbf{41.87}$ dB PSNR and $\mathbf{0.9907}$ SSIM (with camera/light optimization), exceeding SSS-GS by over 6 dB PSNR.
• On GS3 OLAT datasets, SSD-GS consistently yields the highest fidelity relighting, with minimal loss across illumination changes and improved material detail reproduction.
• Ablation shows the full four-term decomposition yields marked SSIM/PSNR improvements over reduced or joint models, highlighting the necessity of both SSS and explicit shadow modeling.

Qualitative Outcomes

SSD-GS achieves accurate separation of soft and sharp shadows, preservation of fine-grained geometric and material details (e.g., anisotropic highlights, internal scattering), and robust relighting under high-frequency, spatially localized illuminations. In contrast, prior works often produce noisy, over-smooth shadows or fail to simulate back-lit translucency.

Component and Training Curriculum Analysis

Ablation confirms that both the explicit SSS term and shadow refinement are critical: removing either causes numerical degradation and visual artifacts (e.g., leaking of scattering into shadow or diffuse terms, loss of shadow crispness, and incorrect translucency). Furthermore, simultaneous training of all terms or non-progressive schedules produces suboptimal decompositions due to gradient interference.

Implications and Future Directions

SSD-GS advances the state of the art in editable, physically-motivated relightable 3D reconstruction, crucial for downstream applications in AR/VR content creation, digital humans, and material-aware editing. Its design enables both interpretable component supervision and generalization to diverse and challenging materials. Key implications and future prospects include:

• Physically-Grounded Generalization: By disentangling reflectance and illumination in a modular physically-plausible way, the approach allows for downstream tasks such as material transfer, fine-grained source editing, and high-quality relightable asset generation.
• Efficiency and Interactivity: SSD-GS exploits the efficiency of rasterization-based 3DGS pipelines, retaining real-time or near-interactive performance suitable for practical deployment.
• Extensibility: The framework can be extended with global illumination, more complex BSSRDFs, and material segmentation, further closing the gap with “full” physical transport simulation if computational budgets allow.
• Ray/Path Traced Integration: Future work might embed ray/path traced transport for additional indirect illumination, or integrate structured supervision to minimize leakage between decomposed terms.
• Robustness to Calibration: The joint optimization of cameras and lighting within the training loop improves robustness to calibration noise, making practical deployment more feasible in uncontrolled capture setups.

Conclusion

SSD-GS presents a systematic and interpretable advance for relightable neural rendering, equipping 3DGS frameworks with explicit and efficient modeling of complex reflectance behavior, including scattering and soft shadowing. The deliberate four-term decomposition with progressive training leads to higher generalization, improved visual fidelity under challenging lighting, and maintains a computational profile suitable for interactive applications. This work establishes a robust technical foundation for physically-sound, relightable scene synthesis and sets the agenda for further exploration into editable, interpretable neural assets.