SSD-GS: Physical Relighting for 3D Gaussian Splatting

• The paper introduces SSD-GS, a framework that replaces coarse shading with an explicit four-way reflectance model (diffuse, specular, shadow, subsurface scattering) to boost relighting fidelity.
• SSD-GS leverages analytic shading and learnable modules to accurately model occlusion-aware shadows and anisotropic specular highlights for various materials.
• A progressive optimization schedule disentangles appearance components, enabling state-of-the-art performance in photorealistic scene relighting on OLAT benchmarks.

Searching arXiv for SSD-GS and closely related relightable 3DGS papers to ground the article with current citations. SSD-GS is a physically-based relighting framework for 3D Gaussian Splatting (3DGS) that targets high-quality reconstruction and photorealistic relighting under novel lighting conditions. It extends relightable Gaussian-splatting pipelines by replacing coarse shading decompositions with an explicit four-way reflectance model comprising diffuse, specular, shadow, and subsurface scattering. The method is motivated by the observation that existing 3DGS-based relighting methods either model only diffuse and specular reflections or rely on neural networks to approximate shadows and scattering, which limits fidelity and physical interpretability, particularly for anisotropic metals and translucent materials. SSD-GS embeds analytic physically based shading, a learnable dipole-based scattering module, and an occlusion-aware shadow formulation directly into the Gaussian splatting pipeline, while preserving the rasterization-based efficiency of 3DGS (Zheng et al., 14 Apr 2026).

1. Definition and problem setting

SSD-GS, short for “Scattering and Shadow Decomposition for Relightable 3D Gaussian Splatting,” is designed to move 3DGS from view synthesis toward editable relighting under novel point-light conditions (Zheng et al., 14 Apr 2026). Its central objective is to improve both physical interpretability and relighting fidelity by explicitly modeling surface reflection, angular specular response, occlusion-driven shadows, and subsurface transport.

The framework is positioned against two limitations in prior relightable 3DGS methods. First, earlier methods use simplified reflectance modeling; the data specifically notes that methods like GS3 model diffuse and specular components while leaving shadows or residual effects to a deferred pixel-space module. Second, prior systems show weak handling of translucent and anisotropic materials, because neural residuals or low-frequency environment-map models often fail to reproduce soft shadows and subsurface scattering faithfully (Zheng et al., 14 Apr 2026).

This suggests that SSD-GS should be understood not merely as a renderer with additional learned appearance terms, but as a structured relighting model in which appearance decomposition is treated as a first-class design principle. A plausible implication is that its contribution lies as much in disentanglement and editability as in raw rendering quality.

2. Four-component reflectance decomposition

SSD-GS decomposes each Gaussian’s appearance into four physically meaningful components (Zheng et al., 14 Apr 2026):

1. Diffuse: a Lambertian term for low-frequency, view-independent reflection.
2. Specular: a Fresnel-weighted anisotropic spherical Gaussian (ASG) model for high-frequency, view-dependent highlights.
3. Shadow: an occlusion-aware soft shadow factor derived from volumetric visibility and refined by a learned network.
4. Subsurface scattering (SSS): a dipole-based diffusion approximation for translucent materials such as skin, wax, jade, or marble.

The decomposition is intended to prevent a neural network from absorbing all appearance variation implicitly. Instead, each term is assigned a distinct physical role. The paper states that this enables better separation of lighting from material properties and yields more robust generalization to unseen illumination conditions, including on the challenging OLAT dataset (Zheng et al., 14 Apr 2026).

A key design choice is that shadow modulates only the direct surface terms, not the scattering term. The per-Gaussian shading equation is given as

$$C_i = \big(c_d f_a + c_s f_s\big)\, S(x) + c_{sss} f_{sss}$$

where $c_d$, $c_s$, and $c_{sss}$ are learned base colors for diffuse, specular, and scattering; $f_a$ is the diffuse intensity; $f_s$ is the specular intensity; $f_{sss}$ is the subsurface scattering intensity; and $S(x)$ is the refined shadow factor at position $x$ (Zheng et al., 14 Apr 2026). The authors explicitly ablate the alternative of applying shadow to SSS and report that doing so harms translucency.

This decomposition distinguishes SSD-GS from coarse BRDF splits and from systems that model shadows or scattering as unstructured residuals. In encyclopedic terms, the method’s identity is inseparable from this four-way decomposition.

3. Rendering formulation and learnable modules

SSD-GS is built on the standard 3D Gaussian Splatting representation. Each Gaussian has a center $X_i$, opacity $c_d$0, covariance $c_d$1 parameterized by rotation and scaling, and learnable appearance or material features (Zheng et al., 14 Apr 2026). Pixel compositing follows the standard 3DGS front-to-back alpha blending rule:

$c_d$2

where $c_d$3 is the accumulated transmittance (Zheng et al., 14 Apr 2026). SSD-GS replaces the original spherical-harmonic appearance model with the physically based decomposition above.

The diffuse term is Lambertian:

$c_d$4

where $c_d$5 is the surface normal and $c_d$6 is the incident light direction (Zheng et al., 14 Apr 2026). The paper presents this term as a stable low-frequency baseline that helps establish coarse appearance and geometry.

The specular term combines Fresnel reflectance with an anisotropic spherical Gaussian basis:

$c_d$7

with $c_d$8 the half-vector, $c_d$9 the Fresnel term using Schlick’s approximation, and $c_s$0 an anisotropic spherical Gaussian reflectance function (Zheng et al., 14 Apr 2026). The ASG expansion is

$c_s$1

with basis functions

$c_s$2

According to the paper, this formulation is intended to represent anisotropic highlights such as brushed metal or textured reflective surfaces while remaining compact because the ASG bases are global and shared (Zheng et al., 14 Apr 2026).

The subsurface scattering module is based on the classical standard dipole diffusion approximation. A neural field $c_s$3 predicts scattering parameters:

$c_s$4

where $c_s$5 is the Gaussian center, $c_s$6 the view direction, $c_s$7 the light direction, $c_s$8 the normal, and $c_s$9 a learnable material embedding (Zheng et al., 14 Apr 2026). The dipole diffusion profile is expressed as

$c_{sss}$0

with $c_{sss}$1, and $c_{sss}$2, $c_{sss}$3, $c_{sss}$4, $c_{sss}$5, and $c_{sss}$6 defined in the paper’s dipole model (Zheng et al., 14 Apr 2026). The description emphasizes that the model follows the closed-form dipole BSSRDF idea rather than a purely learned residual formulation.

Two compact MLPs support the framework. The shadow refinement network $c_{sss}$7 has 3 layers with 32 hidden units per layer and ReLU activations, taking $c_{sss}$8, $c_{sss}$9, $f_a$0, and $f_a$1 as inputs with positional encoding on $f_a$2 and $f_a$3. The scattering predictor $f_a$4 has 6 layers with hidden size 256, ReLU activations, positional encoding for $f_a$5, $f_a$6, and $f_a$7, and outputs constrained to plausible ranges with sigmoids and rescaling (Zheng et al., 14 Apr 2026).

4. Occlusion-aware shadows and volumetric visibility

Visibility is central to SSD-GS’s shadow model. Rather than relying on post-projection screen-space opacity accumulation, the method computes continuous volumetric transmittance along light rays (Zheng et al., 14 Apr 2026). The paper argues that this yields smoother penumbrae, better handling of concavities and overlap, and greater geometric consistency than screen-space methods.

The shadow formulation has three stages. First, for each Gaussian and each relevant pixel or light ray, the method traces a shadow ray and accumulates opacity of intersecting Gaussians:

$f_a$8

where $f_a$9 is the depth-ordered set of Gaussians intersected by the shadow ray (Zheng et al., 14 Apr 2026). This produces a continuous ray-wise transmittance.

Second, those transmittance values are aggregated using projected Gaussian density weights $f_s$0:

$f_s$1

which yields a coarse, density-weighted visibility estimate for Gaussian $f_s$2 (Zheng et al., 14 Apr 2026).

Third, a lightweight MLP refines the coarse estimate:

$f_s$3

where $f_s$4 is the Gaussian center, $f_s$5 is the coarse visibility, $f_s$6 is the incident light direction, and $f_s$7 is the per-Gaussian material embedding (Zheng et al., 14 Apr 2026). The paper attributes contact shadows, fine attenuation, and material-dependent shadow behavior to this refinement stage.

An important interpretive point is that SSD-GS treats shadow as an occlusion-grounded term rather than as a purely image-space effect. This suggests that its shadow component is meant to encode visibility structure with some degree of physical meaning, while still allowing learned refinement where analytic visibility alone is insufficient.

5. Progressive optimization and disentanglement strategy

SSD-GS uses a coarse-to-fine progressive training schedule that the paper presents as essential for disentangling reflectance components (Zheng et al., 14 Apr 2026). The schedule is reported as follows:

Training stage	Iterations	Components active
Stage 1	0–5K iters	diffuse only
Stage 2	5K–9K	add shadow
Stage 3	9K–16K	add scattering
Stage 4	16K onward	add specular

The rationale given in the data is hierarchical. Diffuse is easy to optimize and stabilizes geometry and coarse color. Shadow introduces global visibility structure. Scattering is low-frequency but harder to fit. Specular is high-frequency and can dominate optimization if introduced too early (Zheng et al., 14 Apr 2026).

The authors also temporarily freeze some gradients: shadow gradients are held fixed while scattering converges, and ASG lobe parameters are delayed so that specular learning initially focuses on Fresnel intensity (Zheng et al., 14 Apr 2026). The paper reports that joint training from the beginning or non-physical schedules cause interference between terms and worsen disentanglement.

This suggests that SSD-GS is not only defined by its reflectance decomposition but also by a training curriculum that orders those components according to optimization stability. A plausible implication is that the method’s reported relighting quality depends substantially on this schedule rather than on the component formulas alone.

6. Evaluation protocol, comparative results, and practical capabilities

SSD-GS is evaluated on multiple OLAT-style benchmarks in which scenes are lit by one light at a time and test lighting directions are held out (Zheng et al., 14 Apr 2026). The paper reports three principal dataset groups.

Real OLAT datasets from NRHints include seven scenes: Cat, CatSmall, CupFabric, Fish, FurScene, Pikachu, and Pixiu. Synthetic GS3 datasets include six scenes: Translucent, AnisoMetal, Drums, FurBall, Hotdog, and Lego. The SSS-GS synthetic dataset includes five scenes: Bunny, Candle, Dragon, Soap, and Statue (Zheng et al., 14 Apr 2026). The appendix additionally describes a synthetic relighting validation setup using camera-light trajectories with dense angular sampling.

The reported evaluation metrics are PSNR, SSIM, and LPIPS. Training is typically 100K iterations on NRHints and GS3, and 60K iterations on SSS-GS (Zheng et al., 14 Apr 2026).

The paper compares SSD-GS against 3DGS, GI-GS, GS3, and RNG, and states that SSD-GS generally shows higher PSNR, lower LPIPS, and visually better preservation of material-specific effects (Zheng et al., 14 Apr 2026). The strongest gains are said to appear in translucent or SSS-heavy scenes, anisotropic or reflective scenes, shadow-rich scenes, and real captured scenes under unseen lighting. On the SSS-GS synthetic benchmark, SSD-GS is also reported to outperform methods that focus specifically on scattering, including SSS-GS and KiloOSF (Zheng et al., 14 Apr 2026).

Qualitatively, the paper states that prior methods often blur or oversmooth specular textures, fail to preserve subtle color or curvature cues, generate noisy or unstable shadows, and lose translucency in back-lit materials. SSD-GS is reported to better preserve sharp-to-soft shadow transitions, anisotropic reflections, translucent edge glow, and appearance consistency under novel point lights (Zheng et al., 14 Apr 2026).

Because the model separates reflectance components explicitly, the paper identifies practical downstream uses including controllable light source editing and interactive scene relighting (Zheng et al., 14 Apr 2026). This suggests that the framework is intended not only for reconstruction fidelity benchmarks but also for structured appearance manipulation.

7. Relation to prior Gaussian-splatting relighting and stated limitations

Within the broader 3DGS literature, SSD-GS is described as a drop-in shading extension of 3D Gaussian Splatting rather than a separate volumetric renderer (Zheng et al., 14 Apr 2026). The pipeline remains: represent the scene as anisotropic 3D Gaussians, project Gaussians to the image plane, compute per-Gaussian shading, and alpha-blend contributions front to back. The paper explicitly contrasts this with NeRF-style approaches that require expensive ray marching and with previous 3DGS relighting systems that use less explicit transport decomposition (Zheng et al., 14 Apr 2026).

The method’s novelty is therefore not in abandoning Gaussian splatting, but in redefining how shading is parameterized within that framework. Relative to earlier relightable 3DGS methods, SSD-GS emphasizes four points: explicit scattering, occlusion-aware shadows, anisotropic Fresnel-based specularity, and progressive component integration during training (Zheng et al., 14 Apr 2026).

The authors also identify several limitations. SSD-GS does not provide explicit multi-bounce global illumination and does not fully simulate physical light transport as ray tracing or path tracing would. Because it remains rasterization-based and depends on learned Gaussian geometry, performance is tied to the quality of the Gaussian representation and visibility estimation. The paper further notes that some component leakage can still occur despite the progressive schedule, and it suggests that stronger multi-term supervision could help. Finally, grouping Gaussians by learned material codes is described as an open direction for improving controllability and editing (Zheng et al., 14 Apr 2026).

These limitations address a potential misconception. SSD-GS is physically inspired and more interpretable than coarse relightable 3DGS baselines, but it is not a full path-transport simulator. Its significance lies in advancing Gaussian-splatting relighting toward a more structured and physically meaningful decomposition while retaining the speed and efficiency that make 3DGS attractive (Zheng et al., 14 Apr 2026).