- The paper introduces a dual-Gaussian representation that separates dense surface geometry from interior scattering, preventing low-opacity cavities during translucent reconstruction.
- GTSR combines Fresnel-modulated opacity with a deferred Disney BSDF renderer to model view-dependent appearance without relying on invalid photometric consistency.
- The method achieves the best average Chamfer distance on the NeuralTO Syn benchmark, reaches 45.28 dB PSNR, and trains in about 2.5 hours, but requires co-located flash lighting and simplifies refraction.
Motivation and problem statement
Surface reconstruction of translucent objects from multi-view images remains substantially harder than reconstruction of opaque surfaces. For opaque materials, classical lighting models suffice to explain per-view appearance, so multi-view photometric consistency is a valid geometric prior. For translucent materials, light penetrates the surface, producing refraction and subsurface scattering; the appearance of a surface point varies strongly across viewpoints, invalidating the color-consistency assumption underlying most multi-view stereo pipelines. Prior approaches either rely on differentiable path tracing [recon_trans_obj_diff_rendering] or on NeuS-style implicit fields — NeuralTO being the state of the art for translucent reconstruction [cai2024neuralto] — but both families incur heavy computational cost. Meanwhile, 3DGS-based surface reconstruction methods such as 2DGS [2dgs] and PGSR [chen2024pgsr] offer fast training and real-time rendering but lack any rendering model that accounts for scattering, causing them to fail on translucent inputs.
GTSR identifies two specific failure modes when applying a standard 3DGS pipeline to translucent objects. First, optimization drives many Gaussian kernels inside the object to fit scattered light while depressing surface-kernel opacity below roughly 0.2 to keep interior kernels visible, destroying the dense, high-opacity surface layer that reliable depth rendering requires. Second, multi-view photometric consistency cannot refine geometry because surface color is not view-invariant. The method addresses each with a dedicated mechanism.
Dual Gaussian representation with Fresnel-modulated blending
The core representational change is a decomposition into two sets of Gaussians. Surface Gaussians are flattened (PGSR-style) primitives constrained to be dense and high-opacity; they carry the object's geometry, view-dependent color via spherical harmonics, and PBR material attributes. Interior Gaussians reside within the volume and model subsurface scattering color. Because the two roles are separated, surface Gaussians are no longer forced to become transparent to expose interior radiance.
Blending the two sets naively fails in both obvious ways: direct concatenation reintroduces the opacity conflict, and independent rendering followed by weighted merging places all interior Gaussians behind every surface Gaussian along the ray, which is an incorrect ordering. GTSR instead keeps both sets in a single sorted 3DGS pass but modulates the opacity of each surface kernel by a Fresnel weight computed with Schlick's approximation:
αi′=αi⋅Fr,i,Fr,i=F0+(1−F0)(1−ni⋅ωo)5
with F0=0.04 and ni taken from the shortest axis of the flattened kernel. This makes surface kernels more transparent at grazing-independent near-perpendicular viewing angles, simulating attenuation as light enters and exits the translucent body, while preserving correct depth ordering. Qualitatively, this yields visibly denser, opaque surface layers than 3DGS or PGSR, whose reconstructions exhibit cavities from sparse low-opacity surface kernels.
Disney BSDF with deferred rendering
To recover non-contour detail without photometric consistency, the paper adds a physically based rendering module built on the Disney BSDF [disney_bsdf], which decomposes translucent appearance into a specular BSDF, a dielectric BRDF, and a subsurface diffusion term. Specular and diffuse components are computed in screen space from normal, depth, base color, and roughness maps rendered from the surface Gaussians; the scattering component is rendered from the interior Gaussians' spherical harmonics and attenuated by (1−Fschlick) upon exiting the surface. The final image combines them as
CPBR=Cs+γCd+(1−Fschlick)Cscatter
with γ=0.3 controlling effective transparency.
Two assumptions deserve note. The module adopts NeuralTO's co-located light assumption: a single point light at the camera position with no ambient illumination, i.e., flash photography in darkness. This simplifies the shading denominator considerably but restricts applicability to controlled capture conditions. Additionally, full subsurface diffusion is deemed too complex for the splatting pipeline and is replaced by a simplified formulation — the paper does not quantify the error introduced by this approximation. Deferred rather than forward rendering is chosen deliberately: pixel-level normal maps stabilize gradients, screen-space Fresnel evaluation is convenient once normal/depth maps exist, and per-Gaussian mixtures are avoided, reducing compute.
Training combines L1 + SSIM supervision on both CSH and CPBR, PGSR's single-view and multi-view geometric consistency losses, a containment loss that pulls stray interior Gaussians toward the surface using projected-depth discrepancies, and edge-aware smoothness regularizers on roughness and base color. Initialization uses random point clouds because COLMAP fails on textureless translucent surfaces.
Experimental results
On the NeuralTO Syn dataset (six Blender-rendered translucent objects, 120 views at 800×800), evaluated by Chamfer distance, GTSR achieves the best result on four of six scenes and the lowest average CD, e.g., 0.7×10⁻³ mm on "Bear" versus 1.1 for NeuralTO, and 0.6 on "Nail" versus 2.2. It ties NeuralTO's 1.2 on "Yuanbao" and trails slightly on the two dragon scenes (3.7 vs. 3.1; 2.2 vs. 1.0). Notably, NeuS fails outright ("N/A") on one scene, and generic 3DGS methods degrade severely (e.g., PGSR at 19.4–39.5 on several scenes). One caveat: the authors could not reproduce NeuralTO's result on "Nail" and report the number from the original paper instead.
For novel view synthesis, using CSH as output, GTSR attains PSNR 45.28 dB, SSIM 0.9961, and LPIPS 0.0126 — exceeding even vanilla 3DGS (43.76 dB), which is notable given that the added geometric constraints do not sacrifice rendering quality. Training takes approximately 2.5 hours with under 8 GB of GPU memory on an RTX A6000, versus the significantly longer two-stage NeuS-based pipelines.
Ablations isolate each component's contribution: removing the PBR module raises average CD from 0.0020 to 0.0062 and drops PSNR by ~6.5 dB; removing the Fresnel term raises CD to 0.0075; removing interior Gaussians entirely raises CD to 0.0086. These confirm that the dual-Gaussian decomposition, Fresnel blending, and deferred PBR are individually necessary, with PBR contributing the largest share of geometric accuracy in non-contour regions.
The authors also extend the dataset with new translucent materials and demonstrate stable behavior across varying material properties. On three real-world scenes captured with flash in darkness (~30 images, background masked via SAM3 segmentation), qualitative results show complete surface recovery, though no quantitative ground-truth evaluation is possible there.
Limitations and open questions
The paper concedes several constraints explicitly. Fine geometric details such as dragon scales remain blurred, attributed to insufficient material decoupling. Light transport is simplified: refraction and back-side illumination are omitted, so performance degrades on highly transparent objects such as glass. The co-located lighting assumption excludes environmental illumination, which the authors suggest could be relaxed with learnable environment maps. Whether incorporating learned monocular depth or normal priors would extend the method to transparent media is left as an open question rather than demonstrated.
Conclusion
GTSR demonstrates that translucent surface reconstruction can be brought into the efficient 3DGS regime by separating surface and interior Gaussians, blending them through a Fresnel-modulated opacity, and imposing geometry-coupled constraints via a Disney BSDF deferred-rendering module. It achieves state-of-the-art Chamfer distance on the NeuralTO Syn benchmark with real-time-capable rendering and materially lower training cost than NeuS-based alternatives, while its dependence on controlled co-located lighting and simplified transport bounds its current scope to strongly scattering rather than refractive materials.