SGS-Intrinsic: Indoor Inverse Rendering

• SGS-Intrinsic is a 3D Gaussian Splatting framework designed for sparse-view indoor inverse rendering, enabling recovery of detailed scene geometry, material properties, and lighting.
• It employs a two-stage pipeline that first builds a dense, semantically-coherent Gaussian field, then applies physically based inverse rendering with a hybrid illumination model.
• The framework integrates geometric and semantic priors along with a learned deshadowing module to disentangle lighting from material effects for robust novel view synthesis.

SGS-Intrinsic is a 3D Gaussian Splatting based framework for sparse-view indoor inverse rendering whose objective is to recover a scene-level, editable decomposition of an indoor scene from a small set of posed RGB images, including geometry, normals, semantics, material properties such as albedo and roughness, and illumination, so that the scene can be re-rendered under novel views and new lighting using physically based rendering. Its central design is a two-stage pipeline: first, construction of a dense and geometry-consistent Gaussian semantic field guided by semantic and geometric priors; second, inverse rendering on fixed geometry using a hybrid illumination model, material priors, illumination-invariant constraints, and a learned visibility-based deshadowing module (Niu et al., 29 Mar 2026).

1. Definition, scope, and problem setting

SGS-Intrinsic is formulated for multi-view inverse rendering in indoor scenes under sparse views. The method is motivated by three coupled difficulties. First, sparse-view Gaussian reconstruction is unstable: vanilla 3DGS can overfit observed viewpoints and produce unreliable geometry. Second, indoor illumination is spatially complex and often near-field, so simple distant-light assumptions are inadequate. Third, material and illumination are tightly entangled, making it easy for optimization to absorb highlights or cast shadows into albedo rather than explaining them as lighting or visibility effects (Niu et al., 29 Mar 2026).

Within this formulation, the target outputs are scene geometry, surface normals, a semantic field, material parameters such as albedo and roughness, scene illumination, and relightable rendering. The framework is explicitly scene-level rather than object-centric, and it is designed for sparse-view indoor conditions where prior 3DGS inverse rendering methods are reported to be less reliable. The paper situates SGS-Intrinsic against GSIR, R3DG, IRGS, SVGIR, and GeoSplat, and attributes its advantages to stronger geometry initialization, semantic consistency, a hybrid indoor lighting model, and explicit treatment of shadow-material ambiguity (Niu et al., 29 Mar 2026).

The phrase “semantic-invariant” refers to using semantics as a stable cue across views and optimization conditions. In Stage I, semantic supervision is used to make the Gaussian field semantically coherent. In Stage II, semantic grouping becomes a cue for material consistency, so that regions with similar semantic content are encouraged to exhibit similar recovered material properties. This suggests that semantics function as an anchor for both geometry regularization and intrinsic decomposition, rather than only as a segmentation signal.

2. Dense Gaussian semantic field and Stage I reconstruction

The first stage constructs what the paper calls a dense and geometry-consistent Gaussian semantic field. Instead of initializing from sparse SfM points, SGS-Intrinsic uses VGGT to obtain a dense scene layout as point clouds. The Gaussian representation itself extends vanilla 3DGS. In the supplement, vanilla geometry and appearance parameters are written as

$T = \{p, s, q\}, \qquad C = \{\alpha, f_c\},$

with position $p$, scaling vector $s$, quaternion $q$, opacity $\alpha$, and spherical harmonics coefficients $f_c$. SGS-Intrinsic augments the Gaussian state to

$C = \{\alpha, f_c, f_s, n, a, r\},$

where $f_s$ is a semantic feature, $n$ is a normal, $a$ is albedo, and $p$0 is roughness (Niu et al., 29 Mar 2026).

Geometric supervision is provided by StableNormal through the normal loss

$p$1

where $p$2 is the rendered normal from the Gaussian field and $p$3 is the predicted normal prior. Semantic supervision uses dense LSeg features $p$4, object categories identified by Qwen3-VL, and CLIP text embeddings $p$5. The semantic label map is defined by

$p$6

and the semantic loss is

$p$7

To reduce overfitting to observed views, SGS-Intrinsic samples pseudo-views and enforces semantic consistency across corresponding regions. The paper writes

$p$8

where $p$9 and $s$0 are SAM-generated masks in training and pseudo-views, $s$1 denotes semantic uniformity over a region, and $s$2 reshapes tensors to match mask dimensions. The semantic uniformity operator is defined as

$s$3

These ingredients are combined in the Stage I objective

$s$4

with $s$5 and $s$6. The role of this stage is foundational: it supplies the reliable geometry and semantic field that subsequent inverse rendering assumes.

3. Material, BRDF, and hybrid illumination model

Once geometry is fixed, SGS-Intrinsic performs inverse rendering with physically based rendering. The material model is Cook-Torrance: $s$7 where $s$8, $s$9, and $q$0 denote albedo, roughness, and metallicity. The supplement further specifies

$q$1

$q$2

$q$3

$q$4

$q$5

This is a standard microfacet decomposition, but here it is embedded into a Gaussian scene representation and jointly optimized with lighting and visibility (Niu et al., 29 Mar 2026).

A principal contribution of SGS-Intrinsic is the hybrid illumination model

$q$6

which combines a learnable environment component with localized spherical Gaussian mixture lighting. The point-light representation is written as

$q$7

with axis $q$8, sharpness $q$9, amplitude $\alpha$0, and blend weight $\alpha$1. In the supplement, local-light rendering is given by

$\alpha$2

where $\alpha$3 is the light-to-surface distance and $\alpha$4 is visibility.

The environment component is represented as a learnable cubemap of resolution

$\alpha$5

Using split-sum approximation, the supplement writes

$\alpha$6

The purpose of the hybrid model is to represent both broad ambient room lighting and high-frequency local emitters. This is particularly important in indoor scenes, where environment maps alone are too smooth to capture local luminaires and local lights alone are insufficient for diffuse ambient transport.

4. Illumination-invariant material recovery and deshadowing

SGS-Intrinsic explicitly addresses the tendency of sparse-view inverse rendering to bake cast shadows or highlights into albedo. To mitigate this, it introduces illumination-invariant material constraints together with a learned deshadowing module (Niu et al., 29 Mar 2026).

The deshadowing module is effectively a learned visibility field. Because sparse-view geometry can be inaccurate, purely geometric visibility to local lights may be unreliable. SGS-Intrinsic encodes a 3D position $\alpha$7 using a multi-resolution hash encoder $\alpha$8,

$\alpha$9

then predicts visibility from the encoded feature, relative direction, and distance to a point light: $f_c$0

$f_c$1

This provides a lightweight learned approximation to local-light occlusion, and its practical role is to explain cast shadows as visibility effects rather than as reflectance variation.

Material invariance is enforced by perturbing illumination while keeping viewpoint fixed. Let $f_c$2 be the rendered albedo and $f_c$3 the RGBX material prior under the $f_c$4-th perturbed illumination. The relighting consistency loss is

$f_c$5

with

$f_c$6

$f_c$7

Here $f_c$8 is an object-level SAM mask, and the multi-scale gradient term discourages shadow or highlight structure from leaking into albedo.

A second regularizer uses semantic similarity as a cue for material consistency across views. With region-averaged semantic feature $f_c$9 and region-averaged albedo $C = \{\alpha, f_c, f_s, n, a, r\},$0, plus pseudo-view counterparts $C = \{\alpha, f_c, f_s, n, a, r\},$1 and $C = \{\alpha, f_c, f_s, n, a, r\},$2, the consistency loss is

$C = \{\alpha, f_c, f_s, n, a, r\},$3

The implication is not that semantic identity determines material exactly, but that semantic correspondence provides a stable regularization signal for intrinsic decomposition under sparse observation.

5. Optimization pipeline, implementation, and inputs

The method is trained in two stages over 10,000 iterations: Stage I for 7,000 iterations and Stage II for 3,000 iterations. Virtual view sampling starts after 2,000 iterations, and illumination invariance is introduced after 8,000 iterations. Self-view and multi-view invariance are applied every three iterations (Niu et al., 29 Mar 2026).

The Stage II objective is

$C = \{\alpha, f_c, f_s, n, a, r\},$4

with

$C = \{\alpha, f_c, f_s, n, a, r\},$5

The supplement additionally defines lighting regularizers

$C = \{\alpha, f_c, f_s, n, a, r\},$6

$C = \{\alpha, f_c, f_s, n, a, r\},$7

These stabilize compact local-light representations.

Pseudo-views are generated using a pose distance

$C = \{\alpha, f_c, f_s, n, a, r\},$8

followed by spline interpolation and SAM2 mask tracking. The implementation uses a single NVIDIA RTX 4090, about 40 minutes of training time, and about 3 minutes of preprocessing. Preprocessing includes VGGT, LSeg, StableNormal, and RGBX. The visibility network uses a 16-level hash encoder with max entries per level $C = \{\alpha, f_c, f_s, n, a, r\},$9, feature dimension 2, coarsest resolution 16, finest resolution 512, and a $f_s$0 MLP with Kaiming-uniform initialization and Sigmoid output.

The method assumes sparse but posed indoor RGB images and relies on pretrained models for geometry, normals, semantics, masks, and material priors. This suggests that SGS-Intrinsic is not annotation-heavy, but it is strongly prior-driven.

6. Empirical results, ablations, and relation to prior methods

SGS-Intrinsic is evaluated on Interiorverse, TensoIR, MipNeRF 360, FIPT, and DL3DV. The sparse-view protocols are 12 training views for Interiorverse, 8 for TensoIR, and 24 for MipNeRF 360, FIPT, and DL3DV. The baseline set includes GSIR, GIGS, R3DG, IRGS, SVGIR, and GeoSplat (Niu et al., 29 Mar 2026).

On real-world novel-view synthesis, SGS-Intrinsic reports the strongest numbers among the listed methods. For DL3DV, it achieves $f_s$1 in PSNR/SSIM/LPIPS, compared with $f_s$2 for GeoSplat. For FIPT, it achieves $f_s$3, compared with $f_s$4 for GeoSplat. For MipNeRF, it achieves $f_s$5, compared with $f_s$6 for GeoSplat. On TensoIR, SGS-Intrinsic reports albedo $f_s$7 and PBR $f_s$8, improving over the compared Gaussian baselines.

The ablations are especially informative. In Table 4, the full model reports albedo $f_s$9 and PBR $n$0. Removing the normal prior reduces albedo PSNR to $n$1 and PBR PSNR to $n$2. Removing semantic correspondence reduces albedo PSNR to $n$3 and PBR PSNR to $n$4. Removing $n$5 reduces albedo PSNR sharply to $n$6 and PBR PSNR to $n$7. Removing deshadowing reduces albedo PSNR to $n$8. Removing $n$9 reduces albedo PSNR to $a$0, and removing $a$1 reduces it to $a$2. These results isolate the functional role of each component: semantic and geometric priors stabilize geometry; SGM lighting is essential for indoor light expressiveness; deshadowing prevents material corruption; and invariance constraints improve disentanglement.

A robustness study reports that geometry, normal, and semantic noise produce only mild degradation, and performance with “GT cameras” from 200-view VGGT is nearly identical to the ordinary setting. This suggests that the method is relatively tolerant of imperfect priors. A plausible implication is that the dense semantic-geometric Stage I representation is doing more than supplying a good initialization; it is also regularizing the optimization landscape of Stage II.

Taken together, SGS-Intrinsic can be characterized as a sparse-view indoor inverse rendering system in which reliable Gaussian geometry, semantic coherence, hybrid indoor lighting, and illumination-invariant material supervision are treated as mutually dependent components rather than separate add-ons.