DIAMOND-SSS: Diffusion-Augmented Multi-View Optimization for Data-efficient SubSurface Scattering

Abstract: Subsurface scattering (SSS) gives translucent materials -- such as wax, jade, marble, and skin -- their characteristic soft shadows, color bleeding, and diffuse glow. Modeling these effects in neural rendering remains challenging due to complex light transport and the need for densely captured multi-view, multi-light datasets (often more than 100 views and 112 OLATs). We present DIAMOND-SSS, a data-efficient framework for high-fidelity translucent reconstruction from extremely sparse supervision -- even as few as ten images. We fine-tune diffusion models for novel-view synthesis and relighting, conditioned on estimated geometry and trained on less than 7 percent of the dataset, producing photorealistic augmentations that can replace up to 95 percent of missing captures. To stabilize reconstruction under sparse or synthetic supervision, we introduce illumination-independent geometric priors: a multi-view silhouette consistency loss and a multi-view depth consistency loss. Across all sparsity regimes, DIAMOND-SSS achieves state-of-the-art quality in relightable Gaussian rendering, reducing real capture requirements by up to 90 percent compared to SSS-3DGS.

• The paper introduces a novel framework that leverages diffusion-based augmentation to reduce dense capture needs for relightable 3D subsurface scattering models.
• It employs geometric consistency losses, including silhouette and depth regularization, to enforce robust multi-view reconstruction under sparse conditions.
• Experiments demonstrate up to a 90-95% reduction in real measurements while maintaining high photorealism and accurate subsurface scattering effects.

DIAMOND-SSS: Diffusion-Augmented Multi-View Optimization for Data-Efficient Subsurface Scattering

Introduction and Motivation

Reconstructing relightable 3D models of translucent materials, exhibiting subsurface scattering (SSS), remains a fundamental challenge in neural rendering. SSS governs the visual complexity of materials like skin, wax, or jade by allowing light to penetrate and scatter within the medium, producing color bleeding, soft shadows, and global glow effects. Accurate modeling of SSS in learned volumetric representations demands controlled, dense multi-view, multi-illumination datasets (often exceeding 100 views and OLATs), constraining practical capture and limiting applicability.

DIAMOND-SSS directly addresses this bottleneck by proposing a data-efficient framework that dramatically reduces real data requirements for SSS-enabled 3D reconstruction. Leveraging domain-adapted, geometry-conditioned diffusion models for novel-view synthesis and relighting, the system augments sparse image sets to replace up to 95% of dense measurements with photorealistic, geometrically consistent synthetic supervision. This approach is stabilized with illumination-invariant geometric priors enforcing multi-view silhouette and depth consistency, enabling robustness to both real and synthetic augmentation error.

Figure 1: A data-efficient pipeline for relightable 3D reconstruction with subsurface scattering, using diffusion models to synthesize missing views and illuminations from sparse observations.

Methodology

DIAMOND-SSS builds upon the SSS-3DGS framework, which embeds a learned neural residual to 3D Gaussian Splatting for volumetric SSS modeling. The key enhancements responsible for its data efficiency are:

1. Diffusion-Based Data Augmentation: Fine-tuned diffusion models, adapted from Free3D for multi-view synthesis and ControlNet-style architectures for relighting, are trained on less than 7% of the original OLAT dataset. These models generate geometrically consistent, photometrically plausible novel views and synthetic OLAT conditions, ensuring broad generalizability and obviating per-object retraining. Conditioning is performed via depth and surface normals estimated from off-the-shelf predictors.

Figure 2: Overview of the data augmentation pipeline: additional viewpoints via novel-view diffusion (green), synthetic OLAT variants via relighting diffusion (yellow); all used to supervise SSS-3DGS with added multi-view geometric losses.

2. Multi-View Geometric Consistency Losses: Two illumination-agnostic priors are introduced: (a) silhouette consistency, penalizing misaligned object boundaries across views, and (b) depth consistency, enforcing cross-view agreement in scene geometry. Both losses rely on back-projected correspondences filtered for visibility, utilizing predicted depth and opacity maps.

Figure 3: Illustration of multi-view geometric consistency losses; silhouette loss sharpens contours, depth loss enforces global geometric stability across illumination.

3. Training Procedure: For each real view, several synthetic views and relit images are generated; losses are applied over both sets, with reduced weight on synthetic imagery to mitigate diffusion artifacts. The photometric loss aggregates L1, SSIM, and LPIPS terms, while regularization and geometric consistency terms enforce stability under sparsity. Optimization is performed over the 3DGS parameters and the SSS residual MLP.

Experimental Evaluation

Extensive experiments validate both qualitative and quantitative performance across varying capture regimes. The evaluation protocol systematically prunes the high-resolution OLAT datasets to sparsity levels as low as 3% of views and lights, while maintaining uniform pose-light distribution. Metrics are averaged on held-out views/lights using PSNR, SSIM, and LPIPS.

Under full supervision, DIAMOND-SSS offers marginally improved fidelity over SSS-3DGS but, as supervision is reduced, its advantages are pronounced. For “all views, one light per view” settings—where SSS-3DGS fails to generalize in illumination—DIAMOND-SSS maintains plausible relighting and subsurface effects by virtue of synthetic relighting augmentation. At 3–5% supervision (e.g., 10 views, 3 lights), the system continues to reproduce high-quality translucency, robust silhouettes, and depth stability.

Figure 4: Qualitative comparison under diverse supervision: DIAMOND-SSS retains SSS appearance and shape under extreme sparsity, where baselines collapse.

On ablation, removal of geometric consistency leads to geometric drift and contour bleeding, while disabling diffusion augmentation impairs photometric diversity and relighting quality. The best reconstructions combine both—enabling up to 90% reduction in real capture with strong numerical results (e.g., PSNR ↑ 21.3 vs. 18.1, SSIM ↑ 0.86 vs. 0.58, LPIPS ↓ 0.039 vs. 0.319 at extreme sparsity).

Diffusion Model Fine-Tuning and Relighting Performance

The system’s backbone is the adaptation of diffusion models to the OLAT regime. Free3D, off-the-shelf, is prone to geometric distortions; fine-tuning on a small translucent subset yields significantly improved silhouette and shape coherence across views. ControlNet-style relighting, conditioned on predicted depth and normals, delivers volumetric global illumination and soft shadow stability, outperforming unconditioned and standard NVS-based methods. This validates that, for physically plausible data augmentation in SSS, domain-specific fine-tuning and geometric conditioning are critical.

Figure 5: Multi-view diffusion on a synthetic object; fine-tuned Free3D outperforms general NVS baselines in geometric and silhouette fidelity.

Figure 6: Qualitative relighting comparison: DIAMOND-SSS (full pipeline) preserves SSS cues and appearance under new lighting, outperforming NeuralGaffer and IC-Light.

Multi-View Geometric Losses and Ablation

Isolating the geometric consistency losses, silhouette regularization most strongly sharpens contours and suppresses haloing, while depth consistency aligns global metric structure, preventing positional drift—especially relevant when training includes generative synthetic images susceptible to spatial noise. Their combination is necessary for stable, high-fidelity output.

Figure 7: Synthetic-data ablation of multi-view geometric losses; silhouette and depth consistency are both essential for boundary stability and global alignment.

Limitations and Future Work

While DIAMOND-SSS achieves superior data efficiency, generated synthetic observations via diffusion models may not be strictly physically correct and sometimes induce subtle color or SSS bias. Extending the pipeline with physics-based priors or joint physical–generative augmentation could raise fidelity further. Computational cost is increased through model fine-tuning and diffusion inference, but this is amortized by large reductions in acquisition effort and broad model reusability.

Conclusion

DIAMOND-SSS establishes a new paradigm for efficient neural reconstruction of relightable, SSS-capable 3D models under extreme capture sparsity. By unifying geometry-conditioned, diffusion-based augmentation and illumination-invariant multi-view geometric priors, it achieves consistent, photorealistic output with 90–95% reduction in ground truth data requirements. This enables scalable acquisition of volumetric appearance for translucent materials, with implications for digital content creation, novel-view relighting, and data-driven materials research. As physical–generative model integration and cross-domain adaptation mature, further advances in efficiency and realism are anticipated.