---
title: Decomposed Gaussian Splatting in 3D Reconstruction
url: https://www.emergentmind.com/topics/decomposed-gaussian-splatting
type: topic
---

# Decomposed Gaussian Splatting in 3D Reconstruction

Searching arXiv for recent papers on decomposed Gaussian splatting and closely related formulations.
Decomposed Gaussian splatting is a family of 3D Gaussian Splatting formulations in which the representation, rendering process, or optimization pipeline is partitioned into explicit components instead of being treated as a single entangled set of splats. In recent work, those components have included static and dynamic instances, distractor and background layers, geometry and appearance opacities, albedo–shading–residual fields, low- and high-frequency bands, space–time encodings, and modular training operators. Across these formulations, the decomposition is used to improve scene separation, physically grounded motion, geometric fidelity, relighting, editability, dynamic level-of-detail control, or training efficiency while preserving the explicit rasterization and alpha-compositing structure of Gaussian splatting [2511.19235] [2606.05124] [2503.21226].

## 1. Formal basis and decomposition axes

Most decomposed variants retain the standard 3DGS primitive: a set of anisotropic Gaussians with mean $\mu_i \in \mathbb{R}^3$, covariance $\Sigma_i \in \mathbb{R}^{3\times3}$, opacity, and appearance parameters. A common parameterization is
$$
\Sigma_i = R_i \operatorname{diag}(s_i^2) R_i^\top,
$$
with Gaussian density
$$
G_i(x) = \exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right),
$$
followed by projection to a 2D elliptical footprint and front-to-back alpha compositing such as
$$
C(p) = \sum_i \alpha_i(p)c_i(p)\prod_{j<i}(1-\alpha_j(p)).
$$
Decomposed methods usually intervene after this common base, either by splitting the Gaussian set itself, by splitting the per-Gaussian attributes used in different rendering passes, or by splitting the optimization into explicit operators or subproblems [2606.05124].

A concise way to organize the literature is to distinguish decomposition by **scene partition**, **attribute factorization**, **frequency or space–time factorization**, and **pipeline modularization**. Independent Gaussian sets for albedo, shading, and residual, per-instance rigid transforms in canonical coordinates, per-band Gaussian groups aligned with Laplacian pyramids, and operator-level training decomposition all instantiate different meanings of the same term [2606.31637] [2511.19235] [2503.21226] [2503.01199].

| Decomposition axis | Representative papers | Defining mechanism |
|---|---|---|
| Scene or instance partition | IDSplat, Inst4DGS, DeSplat | Static background plus dynamic instances, canonical IDs, or per-view distractor layers |
| Attribute factorization | Geometry Gaussians, intrinsic decomposition, BiGS, Discretized SDF | Separate opacities, independent Gaussian fields, or relighting factors |
| Frequency or space–time factorization | Wavelet-GS, Frequency-Aware GS Decomposition, 4D-GS, Grid4D | LF/HF branches, Laplacian levels, decomposed planes, tri-axial grids |
| Pipeline modularization | LiteGS | Clustering, culling, compaction, projection, binning, rasterization, and backward reduction |

A common misconception is that decomposed Gaussian splatting refers only to semantic or instance-level separation. The literature shows a broader usage: decomposition may be semantic, physical, spectral, spatiotemporal, or purely algorithmic.

## 2. Instance and scene-layer decomposition

Instance-decomposed formulations treat dynamic objects as coherent entities rather than as time-varying primitive attributes. IDSplat is a self-supervised driving-scene formulation that partitions the scene into static background and dynamic object instances, keeps each dynamic object fixed in a canonical frame, and optimizes per-instance rigid $SE(3)$ trajectories. For a Gaussian with instance label $z_i$, IDSplat uses
$$
\mu_{i,t} = T_{z_i,t}\mu_i, \qquad
\Sigma_{i,t} = R_{z_i,t}\Sigma_i R_{z_i,t}^\top.
$$
Its pipeline combines Grounded-SAM-2 masks with prompts such as “car,” “truck,” “van,” and “bus,” LiDAR anchoring, DBSCAN filtering, DINOv3 feature correspondences, RANSAC with Umeyama’s estimator, coordinated-turn smoothing over states $\{T_t,v_t,\kappa_t\}$, and joint optimization against camera and LiDAR renderings. On Waymo Open Dataset, it reports PSNR 30.61, SSIM 0.897, LPIPS 0.163, and DPSNR 28.49 on Dynamic NOTR under the DeSiRe-GS protocol, and at 25% training views it reports DPSNR 26.19 versus AD-GS 21.38 and DeSiRe-GS 19.59 [2511.19235].

Inst4DGS addresses a different but related problem: inconsistent instance labels across independently segmented multi-view videos. It introduces per-video label-permutation latents normalized through a differentiable Sinkhorn layer so that canonical logits can be mapped into local per-view label spaces. The method couples this explicit label alignment with long-horizon per-Gaussian trajectories and instance-decomposed motion scaffolds using low-dimensional motion bases and dual-quaternion blending. On Panoptic Studio, it improves PSNR from 26.10 to 28.36 and instance mIoU from 0.6310 to 0.9129 over the strongest baseline; on Neural3DV it reports PSNR 30.88, SSIM 0.9384, LPIPS 0.0492, and instance mIoU 0.9420 [2603.18402].

DeSplat decomposes a scene into a global static layer and per-view distractor layers rather than persistent object identities. Static Gaussians are shared across views, while view-specific distractor Gaussians are initialized on a plane in front of each camera and optimized only for that view. The final image is composed as
$$
C_v(x) = C^{(D)}_v(x) + (1-A^{(D)}_v(x))\,C^{(S)}_v(x),
$$
with regularization
$$
L_{\text{tot}} = L_{\text{GS}} + \lambda_s |1-\alpha_s| + \lambda_d |\alpha_d|.
$$
This formulation avoids external semantic preprocessing and yields explicit separation of static scene content and distractors. On RobustNeRF and On-the-go, it is frequently best or near-best on highly occluded scenes while maintaining rendering speed close to Splatfacto; for example, on Statue it reports 23.40 PSNR at 108.92 FPS versus Splatfacto 106.34 FPS [2411.19756].

These methods collectively show that object-level decomposition is not unique. Some formulations model persistent actors with canonical geometry and learned motion; others model view-specific, nonpersistent occluders as separate rendering layers.

## 3. Geometry, appearance, and intrinsic factorization

A second major interpretation of decomposed Gaussian splatting separates **what contributes to appearance** from **what defines geometry**. Geometry Gaussians argues that vanilla 3DGS is “inheritedly unsuited” to represent texture and geometry at the same time because a single opacity must simultaneously govern visibility, photometric compositing, and geometry accumulation. It introduces a separate geometry opacity $\beta_i$, while retaining appearance opacity $\alpha_i$, and performs two transmittance passes:
- appearance pass with $\alpha_i$ for RGB and $D_{\text{photo}}$,
- geometry pass with $\beta_i$ for $D_{\text{geo}}$ and normals.

This explicit decoupling alleviates the conflict between photorealistic rendering and surface localization, especially for transparent objects. On NeRF Synthetic with PGSR, adding GT depth without $\beta_i$ changes PSNR/CD from 31.76/0.016 to 29.78/0.010, while “+ opacity_geo + GT depth” yields 33.06/0.010. On TransLab, the method reports PSNR $\approx 39.95$, CD $\approx 1.665$, and F1 $\approx 0.960$, while remaining $\approx 3\times$ faster than TSGS [2606.05124].

“Intrinsic decomposition and editing of 3D Gaussian splats” extends the classical intrinsic model
$$
I = A \times S + R
$$
to Gaussian splatting by representing albedo, shading, and view-dependent residual as three independent Gaussian fields,
$$
\mathcal{G}^{A},\quad \mathcal{G}^{S},\quad \mathcal{G}^{R}.
$$
Each field has its own positions, covariances, opacities, and color parameters, and the full image formation becomes
$$
\mathbf{I}(r) = \sum_i T_i\,\alpha_i\,\left(\mathbf{A}_i \odot \mathbf{S}_i + \mathbf{R}_i(r)\right).
$$
The method optimizes the albedo field using DiffusionRenderer predictions and Depth Anything V2 regularization, then optimizes shading under the diffuse model, and finally adds a residual field with spherical harmonics. It also provides an editing workflow in which a user edits only the albedo of a planar surface in one image, re-optimizes only the albedo field, and then re-renders the edited scene under arbitrary viewpoints. On a synthetic scene it reports albedo PSNR/SSIM/LPIPS of 29.419/0.932/0.095, versus 13.675/0.769/0.310 for GI-GS and 11.680/0.683/0.445 for R3GS [2606.31637].

BiGS decomposes relightable appearance into diffuse scattering $\rho$, directional scattering $s(\omega_i,\omega_o)$, direct transport $T_{\mathrm{dir}}$, and indirect transport $\mathbf{T}_{\mathrm{ind}}$, using bidirectional spherical harmonics rather than normal-dependent BRDF terms. Its outgoing radiance is represented as a contraction of lighting SH coefficients with per-primitive SH and BSH parameters, and the result is injected directly into standard 3DGS alpha compositing. This gives a normal-free relighting formulation compatible with volumetric splats and materials such as fur, hair, and translucent objects. BiGS reports 40–50 fps for relight+render on scenes of about 40k primitives, with about 1,089 appearance parameters per primitive and about 200 MB appearance memory at that scale [2408.13370].

“Gaussian Splatting with Discretized SDF for Relightable Assets” decomposes inverse rendering differently: it replaces per-Gaussian learned opacity with a discretized SDF sample $s_i$ and computes opacity through
$$
o_i = T_\gamma(s_i) = \frac{4e^{-\gamma s_i}}{(1+e^{-\gamma s_i})^2}, \qquad \alpha_i = p_i\,o_i.
$$
The method then enforces projection-based consistency between the alpha-blended surface and the zero-level set implied by $(s_i,n_i)$, thereby avoiding continuous SDF networks, Eikonal losses, and ray marching. It reports the same per-Gaussian memory as GS, 4 GB training memory versus 22 GB for GS-ROR, and on Glossy Blender it reports mean PSNR/SSIM/LPIPS of 24.52/0.9229/0.0762 with mean normal MAE 6.48° on Shiny Blender and mean CD 0.0107 on Glossy Blender [2507.15629].

These geometry- and intrinsic-factorized formulations share an important principle: decomposition is used to reduce interference between incompatible objectives, such as photometric fit versus geometry, or texture editing versus illumination preservation.

## 4. Frequency and spectral decomposition

Frequency-decomposed Gaussian splatting partitions either the geometry, the images, or the motion model into explicit bands. Wavelet-GS performs 3D wavelet decomposition of the input point cloud into low-frequency and high-frequency components, then optimizes them with different strategies. The low-frequency branch stabilizes global scene structure through voxel-guided Gaussian generation and grow-and-prune updates, while the high-frequency branch restores geometry and texture detail, adds a relight module, and is guided by a Laplacian–Wavelet loss computed from Laplacian pyramids and 2D DWTs of rendered and target images. The overall representation remains additive, with $P = P_{\text{low}} + P_{\text{high}}$. On Waymo, JHU-Drone, Tanks and Temples, and Mip-NeRF360, it reports SSIM/PSNR/LPIPS of 0.853/28.34/0.274, 0.892/29.92/0.082, 0.863/24.40/0.124, and 0.870/29.68/0.170, respectively [2507.12498].

Frequency-Aware Gaussian Splatting Decomposition aligns Gaussian groups with Laplacian-pyramid subbands of the input images. Each Gaussian is assigned an integer level $l_i \in \{1,\dots,L\}$, and rendering all Gaussians with $l_i \le k$ is trained to match a low-pass target
$$
I_k = (I \downarrow_2^{L-k}) \uparrow_2^{L-k}.
$$
Higher-frequency levels use residual colors in $[-1,1]$ through
$$
C_i = 2\cdot SH(G_i,d)-1,
$$
followed by final clamping to $[0,1]$. Band coherence is enforced with a spatial image loss and a DFT magnitude loss, and new levels are introduced every $K=2500$ steps by duplicating the current Gaussians and reinitializing the optimizers. The method enables explicit 3D frequency editing, stylization, progressive rendering, streaming, and foveated rendering, but it also reports a modest reconstruction trade-off as the number of levels increases; on the Kitchen scene, PSNR changes from 33.683 at one level to 33.289 at four levels [2503.21226].

“Laplacian Analysis Meets Dynamics Modelling: Gaussian Splatting for 4D Reconstruction” transfers spectral decomposition into motion modeling. It represents per-primitive motion as a low-frequency Laplacian/Fourier component plus a high-frequency hash residual,
$$
\hat u_i(t) = u_{\text{low},i}(t) + u_{\text{high},i}(t),
$$
and fuses them with spectral-aware attention. The low-frequency branch is regularized by graph-Laplacian penalties, while the high-frequency branch receives only mild temporal Tikhonov smoothing. The method explicitly frames prior low-rank dynamic encodings as suffering from spectral conflict between deformation consistency and detail preservation. On HyperNeRF-vrig it reports average SSIM/PSNR/LPIPS of 0.720/25.83/0.253, improving over Grid4D 0.715/25.46/0.259 and 4DGaussians 0.681/25.05/0.346; on D-NeRF it reports 0.994/42.17/0.007 [2508.04966].

A common misconception is that frequency decomposition is only a rendering convenience. In these papers it is also a control mechanism for optimization: low bands enforce global coherence, while high bands or residual branches prevent over-smoothing.

## 5. Decomposed space–time encodings and task-specific fields

Dynamic Gaussian splatting has also used decomposition at the level of space–time encoding. 4D-GS represents a scene with a single canonical set of 3D Gaussians and a time-varying deformation field built from six learned 2D planes over $(x,y)$, $(x,z)$, $(y,z)$, $(x,t)$, $(y,t)$, and $(z,t)$. Plane features are bilinearly sampled, multiplied within each resolution, concatenated across resolutions, and decoded into $\Delta X$, $\Delta r$, and $\Delta s$ for each Gaussian. This factorization avoids storing a full per-frame Gaussian set and yields real-time dynamic rendering: 82 FPS at $800\times800$ on an RTX 3090 with about 18 MB storage, versus 418 MB for K-Planes and 440 MB for FFDNeRF in the reported synthetic setting [2310.08528].

Grid4D argues that plane-based 4D decompositions impose an inappropriate low-rank assumption and create excessive feature overlap. It replaces six 2D planes with one spatial 3D hash encoder $G_{xyz}$ and three temporal 3D encoders $G_{xyt}$, $G_{yzt}$, and $G_{xzt}$, then aggregates them through a directional attention mechanism
$$
\mathbf a = 2\cdot \Phi(\mathbf h_{xyz}) - 1, \qquad \mathbf h = \mathbf a \odot \mathbf v_t.
$$
The directional range $(-1,1)$ is used so that temporal features can be amplified or inverted depending on spatial context. Grid4D further adds smooth regularization directly in encoder feature space. On D-NeRF it reports mean PSNR/SSIM/LPIPS of 42.00/0.994/0.008, with “Grid4D + SC” reaching 42.41/0.994/0.008; on HyperNeRF it reports 25.50/0.856 on vrig and 28.56/0.933 on interp [2410.20815].

DEGSTalk shows that decomposed Gaussian splatting is not restricted to novel-view synthesis or autonomous driving. It uses field-level decomposition into separate face and mouth Gaussian fields, a preserved hair layer, and per-primitive embedding decomposition in which each Gaussian carries a learnable embedding $e_i \in \mathbb{R}^{32}$ that conditions a deformation MLP. Hair is preserved by a dedicated composition stage,
$$
\mathcal C(u)=\mathcal C_{\text{hair}}(u)+\mathcal C_{\text{face}}(u)\,\mathcal O_{\text{face}}(u)+\mathcal C_{\text{mouth}}(u)\,(1-\mathcal O_{\text{face}}(u)).
$$
The method reports PSNR 37.30, LPIPS 0.0140, LMD 2.349, SSIM 0.9664, 114 FPS, and about 0.5h training time [2412.20148].

These examples broaden the scope of the term. Decomposition may refer to canonical-time encodings, motion bases, field separation, or compositional rendering specific to a domain such as talking-face synthesis.

## 6. Optimization patterns, evaluation, applications, and open issues

Despite their diversity, decomposed methods share several optimization patterns. They usually keep the standard splatting rasterizer and jointly optimize Gaussian geometry, opacity-like parameters, appearance features, and decomposition-specific variables such as instance poses, permutation matrices, geometry opacities, band levels, spectral coefficients, or auxiliary fields. Photometric supervision is commonly combined with SSIM-like terms, geometry or depth regularization, motion smoothness, opacity or scale priors, and density-control procedures such as cloning, splitting, pruning, or densification. At the framework level, LiteGS shows that decomposition can also be applied to the **training pipeline itself**, factorizing the computation into clustering, cluster culling, cluster compaction, projection, binning, rasterization, compact valid gradients, shared-memory multibatch reduction, sparse gradient assembly, and fused Adam. Without changing the core 3DGS projection or blending, it reports about 3.4× training speedup and approximately 30% lower GPU memory on Mip-NeRF 360 [2503.01199].

The applications of decomposed Gaussian splatting are correspondingly broad. IDSplat targets autonomous driving reconstruction, multi-sensor rendering, scene editing, and simulation [2511.19235]. DeSplat targets distractor-free novel view synthesis in real scenes [2411.19756]. Intrinsic Gaussian decomposition targets multi-view-consistent texture editing while preserving lighting [2606.31637]. Frequency-aware methods target stylization, progressive rendering, streaming, foveated rendering, and faster geometry interaction [2503.21226]. DEGSTalk targets long-hair-preserving talking-face synthesis [2412.20148].

The limitations are equally recurrent. Instance-level driving methods inherit the weaknesses of rigid-body assumptions, LiDAR field-of-view constraints, zero-shot masks, and missing track management; IDSplat reports lower cyclist DPSNR due to the rigid-body assumption and notes that camera-visible objects outside LiDAR FOV are not instantiated as dynamic [2511.19235]. Intrinsic decomposition with independent fields is slower than vanilla 3DGS because it requires multiple render calls and separate optimization stages, and its albedo quality depends on the quality of diffusion-based intrinsic predictions [2606.31637]. BiGS avoids normals but pays a substantial appearance-parameter cost, and its SH/BSH basis is limited for very sharp shadows or highlights [2408.13370]. Frequency-aware decomposition improves control but can slightly reduce final reconstruction metrics as the number of levels grows [2503.21226]. Dynamic space–time decompositions continue to negotiate a tension between low-rank smoothness, feature collision, and high-frequency motion fidelity, which later spectral methods explicitly identify as a central failure mode [2410.20815] [2508.04966].

Taken together, the literature does not support a single canonical definition of decomposed Gaussian splatting. Instead, it identifies a design pattern: explicit partitioning of a splat-based model into components whose roles are easier to constrain, optimize, or edit than those of a single entangled Gaussian field. That pattern has already produced distinct technical lineages—instance-aware, intrinsic, geometry-aware, spectral, dynamic, and pipeline-modular—and the current papers suggest that future work will continue to mix these axes rather than choose only one.

Source: https://www.emergentmind.com/topics/decomposed-gaussian-splatting