Decomposed Gaussian Splatting in 3D Reconstruction

• Decomposed Gaussian splatting is a 3D rendering technique that partitions the representation into discrete components to improve scene separation and dynamic modeling.
• It decomposes factors such as static versus dynamic layers, intrinsic attribute fields, and frequency bands to optimize both rendering quality and training efficiency.
• This approach enhances geometric fidelity, relighting, editability, and supports specialized applications in autonomous driving, texture editing, and dynamic reconstruction.

Searching arXiv for papers on decomposed Gaussian splatting and closely related formulations. Decomposed Gaussian splatting is a family of 3D Gaussian Splatting formulations in which the representation, rendering process, or optimization pipeline is partitioned into explicit components instead of being treated as a single entangled set of splats. In recent work, those components have included static and dynamic instances, distractor and background layers, geometry and appearance opacities, albedo–shading–residual fields, low- and high-frequency bands, space–time encodings, and modular training operators. Across these formulations, the decomposition is used to improve scene separation, physically grounded motion, geometric fidelity, relighting, editability, dynamic level-of-detail control, or training efficiency while preserving the explicit rasterization and alpha-compositing structure of Gaussian splatting (Lindström et al., 24 Nov 2025, Zhou et al., 3 Jun 2026, Lavi et al., 27 Mar 2025).

1. Formal basis and decomposition axes

Most decomposed variants retain the standard 3DGS primitive: a set of anisotropic Gaussians with mean $\mu_i \in \mathbb{R}^3$, covariance $\Sigma_i \in \mathbb{R}^{3\times3}$, opacity, and appearance parameters. A common parameterization is

$$\Sigma_i = R_i \operatorname{diag}(s_i^2) R_i^\top,$$

with Gaussian density

$$G_i(x) = \exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right),$$

followed by projection to a 2D elliptical footprint and front-to-back alpha compositing such as

$$C(p) = \sum_i \alpha_i(p)c_i(p)\prod_{j<i}(1-\alpha_j(p)).$$

Decomposed methods usually intervene after this common base, either by splitting the Gaussian set itself, by splitting the per-Gaussian attributes used in different rendering passes, or by splitting the optimization into explicit operators or subproblems (Zhou et al., 3 Jun 2026).

A concise way to organize the literature is to distinguish decomposition by scene partition, attribute factorization, frequency or space–time factorization, and pipeline modularization. Independent Gaussian sets for albedo, shading, and residual, per-instance rigid transforms in canonical coordinates, per-band Gaussian groups aligned with Laplacian pyramids, and operator-level training decomposition all instantiate different meanings of the same term (Lanvin et al., 30 Jun 2026, Lindström et al., 24 Nov 2025, Lavi et al., 27 Mar 2025, Liao, 3 Mar 2025).

Decomposition axis	Representative papers	Defining mechanism
Scene or instance partition	IDSplat, Inst4DGS, DeSplat	Static background plus dynamic instances, canonical IDs, or per-view distractor layers
Attribute factorization	Geometry Gaussians, intrinsic decomposition, BiGS, Discretized SDF	Separate opacities, independent Gaussian fields, or relighting factors
Frequency or space–time factorization	Wavelet-GS, Frequency-Aware GS Decomposition, 4D-GS, Grid4D	LF/HF branches, Laplacian levels, decomposed planes, tri-axial grids
Pipeline modularization	LiteGS	Clustering, culling, compaction, projection, binning, rasterization, and backward reduction

A common misconception is that decomposed Gaussian splatting refers only to semantic or instance-level separation. The literature shows a broader usage: decomposition may be semantic, physical, spectral, spatiotemporal, or purely algorithmic.

2. Instance and scene-layer decomposition

Instance-decomposed formulations treat dynamic objects as coherent entities rather than as time-varying primitive attributes. IDSplat is a self-supervised driving-scene formulation that partitions the scene into static background and dynamic object instances, keeps each dynamic object fixed in a canonical frame, and optimizes per-instance rigid $SE(3)$ trajectories. For a Gaussian with instance label $z_i$, IDSplat uses

$$\mu_{i,t} = T_{z_i,t}\mu_i, \qquad \Sigma_{i,t} = R_{z_i,t}\Sigma_i R_{z_i,t}^\top.$$

Its pipeline combines Grounded-SAM-2 masks with prompts such as “car,” “truck,” “van,” and “bus,” LiDAR anchoring, DBSCAN filtering, DINOv3 feature correspondences, RANSAC with Umeyama’s estimator, coordinated-turn smoothing over states $\{T_t,v_t,\kappa_t\}$, and joint optimization against camera and LiDAR renderings. On Waymo Open Dataset, it reports PSNR 30.61, SSIM 0.897, LPIPS 0.163, and DPSNR 28.49 on Dynamic NOTR under the DeSiRe-GS protocol, and at 25% training views it reports DPSNR 26.19 versus AD-GS 21.38 and DeSiRe-GS 19.59 (Lindström et al., 24 Nov 2025).

Inst4DGS addresses a different but related problem: inconsistent instance labels across independently segmented multi-view videos. It introduces per-video label-permutation latents normalized through a differentiable Sinkhorn layer so that canonical logits can be mapped into local per-view label spaces. The method couples this explicit label alignment with long-horizon per-Gaussian trajectories and instance-decomposed motion scaffolds using low-dimensional motion bases and dual-quaternion blending. On Panoptic Studio, it improves PSNR from 26.10 to 28.36 and instance mIoU from 0.6310 to 0.9129 over the strongest baseline; on Neural3DV it reports PSNR 30.88, SSIM 0.9384, LPIPS 0.0492, and instance mIoU 0.9420 (Lee et al., 19 Mar 2026).

DeSplat decomposes a scene into a global static layer and per-view distractor layers rather than persistent object identities. Static Gaussians are shared across views, while view-specific distractor Gaussians are initialized on a plane in front of each camera and optimized only for that view. The final image is composed as

$$C_v(x) = C^{(D)}_v(x) + (1-A^{(D)}_v(x))\,C^{(S)}_v(x),$$

with regularization

$\Sigma_i \in \mathbb{R}^{3\times3}$0

This formulation avoids external semantic preprocessing and yields explicit separation of static scene content and distractors. On RobustNeRF and On-the-go, it is frequently best or near-best on highly occluded scenes while maintaining rendering speed close to Splatfacto; for example, on Statue it reports 23.40 PSNR at 108.92 FPS versus Splatfacto 106.34 FPS (Wang et al., 2024).

These methods collectively show that object-level decomposition is not unique. Some formulations model persistent actors with canonical geometry and learned motion; others model view-specific, nonpersistent occluders as separate rendering layers.

3. Geometry, appearance, and intrinsic factorization

A second major interpretation of decomposed Gaussian splatting separates what contributes to appearance from what defines geometry. Geometry Gaussians argues that vanilla 3DGS is “inheritedly unsuited” to represent texture and geometry at the same time because a single opacity must simultaneously govern visibility, photometric compositing, and geometry accumulation. It introduces a separate geometry opacity $\Sigma_i \in \mathbb{R}^{3\times3}$1, while retaining appearance opacity $\Sigma_i \in \mathbb{R}^{3\times3}$2, and performs two transmittance passes:

• appearance pass with $\Sigma_i \in \mathbb{R}^{3\times3}$3 for RGB and $\Sigma_i \in \mathbb{R}^{3\times3}$4,
• geometry pass with $\Sigma_i \in \mathbb{R}^{3\times3}$5 for $\Sigma_i \in \mathbb{R}^{3\times3}$6 and normals.

This explicit decoupling alleviates the conflict between photorealistic rendering and surface localization, especially for transparent objects. On NeRF Synthetic with PGSR, adding GT depth without $\Sigma_i \in \mathbb{R}^{3\times3}$7 changes PSNR/CD from 31.76/0.016 to 29.78/0.010, while “+ opacity_geo + GT depth” yields 33.06/0.010. On TransLab, the method reports PSNR $\Sigma_i \in \mathbb{R}^{3\times3}$8, CD $\Sigma_i \in \mathbb{R}^{3\times3}$9, and F1 $$\Sigma_i = R_i \operatorname{diag}(s_i^2) R_i^\top,$$0, while remaining $$\Sigma_i = R_i \operatorname{diag}(s_i^2) R_i^\top,$$1 faster than TSGS (Zhou et al., 3 Jun 2026).

“Intrinsic decomposition and editing of 3D Gaussian splats” extends the classical intrinsic model

$$\Sigma_i = R_i \operatorname{diag}(s_i^2) R_i^\top,$$2

to Gaussian splatting by representing albedo, shading, and view-dependent residual as three independent Gaussian fields,

$$\Sigma_i = R_i \operatorname{diag}(s_i^2) R_i^\top,$$3

Each field has its own positions, covariances, opacities, and color parameters, and the full image formation becomes

$$\Sigma_i = R_i \operatorname{diag}(s_i^2) R_i^\top,$$4

The method optimizes the albedo field using DiffusionRenderer predictions and Depth Anything V2 regularization, then optimizes shading under the diffuse model, and finally adds a residual field with spherical harmonics. It also provides an editing workflow in which a user edits only the albedo of a planar surface in one image, re-optimizes only the albedo field, and then re-renders the edited scene under arbitrary viewpoints. On a synthetic scene it reports albedo PSNR/SSIM/LPIPS of 29.419/0.932/0.095, versus 13.675/0.769/0.310 for GI-GS and 11.680/0.683/0.445 for R3GS (Lanvin et al., 30 Jun 2026).

BiGS decomposes relightable appearance into diffuse scattering $$\Sigma_i = R_i \operatorname{diag}(s_i^2) R_i^\top,$$5, directional scattering $$\Sigma_i = R_i \operatorname{diag}(s_i^2) R_i^\top,$$6, direct transport $$\Sigma_i = R_i \operatorname{diag}(s_i^2) R_i^\top,$$7, and indirect transport $$\Sigma_i = R_i \operatorname{diag}(s_i^2) R_i^\top,$$8, using bidirectional spherical harmonics rather than normal-dependent BRDF terms. Its outgoing radiance is represented as a contraction of lighting SH coefficients with per-primitive SH and BSH parameters, and the result is injected directly into standard 3DGS alpha compositing. This gives a normal-free relighting formulation compatible with volumetric splats and materials such as fur, hair, and translucent objects. BiGS reports 40–50 fps for relight+render on scenes of about 40k primitives, with about 1,089 appearance parameters per primitive and about 200 MB appearance memory at that scale (Liu et al., 2024).

“Gaussian Splatting with Discretized SDF for Relightable Assets” decomposes inverse rendering differently: it replaces per-Gaussian learned opacity with a discretized SDF sample $$\Sigma_i = R_i \operatorname{diag}(s_i^2) R_i^\top,$$9 and computes opacity through

$$G_i(x) = \exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right),$$0

The method then enforces projection-based consistency between the alpha-blended surface and the zero-level set implied by $$G_i(x) = \exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right),$$1, thereby avoiding continuous SDF networks, Eikonal losses, and ray marching. It reports the same per-Gaussian memory as GS, 4 GB training memory versus 22 GB for GS-ROR, and on Glossy Blender it reports mean PSNR/SSIM/LPIPS of 24.52/0.9229/0.0762 with mean normal MAE 6.48° on Shiny Blender and mean CD 0.0107 on Glossy Blender (Zhu et al., 21 Jul 2025).

These geometry- and intrinsic-factorized formulations share an important principle: decomposition is used to reduce interference between incompatible objectives, such as photometric fit versus geometry, or texture editing versus illumination preservation.

4. Frequency and spectral decomposition

Frequency-decomposed Gaussian splatting partitions either the geometry, the images, or the motion model into explicit bands. Wavelet-GS performs 3D wavelet decomposition of the input point cloud into low-frequency and high-frequency components, then optimizes them with different strategies. The low-frequency branch stabilizes global scene structure through voxel-guided Gaussian generation and grow-and-prune updates, while the high-frequency branch restores geometry and texture detail, adds a relight module, and is guided by a Laplacian–Wavelet loss computed from Laplacian pyramids and 2D DWTs of rendered and target images. The overall representation remains additive, with $$G_i(x) = \exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right),$$2. On Waymo, JHU-Drone, Tanks and Temples, and Mip-NeRF360, it reports SSIM/PSNR/LPIPS of 0.853/28.34/0.274, 0.892/29.92/0.082, 0.863/24.40/0.124, and 0.870/29.68/0.170, respectively (Zhao et al., 16 Jul 2025).

Frequency-Aware Gaussian Splatting Decomposition aligns Gaussian groups with Laplacian-pyramid subbands of the input images. Each Gaussian is assigned an integer level $$G_i(x) = \exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right),$$3, and rendering all Gaussians with $$G_i(x) = \exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right),$$4 is trained to match a low-pass target

$$G_i(x) = \exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right),$$5

Higher-frequency levels use residual colors in $$G_i(x) = \exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right),$$6 through

$$G_i(x) = \exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right),$$7

followed by final clamping to $$G_i(x) = \exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right),$$8. Band coherence is enforced with a spatial image loss and a DFT magnitude loss, and new levels are introduced every $$G_i(x) = \exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right),$$9 steps by duplicating the current Gaussians and reinitializing the optimizers. The method enables explicit 3D frequency editing, stylization, progressive rendering, streaming, and foveated rendering, but it also reports a modest reconstruction trade-off as the number of levels increases; on the Kitchen scene, PSNR changes from 33.683 at one level to 33.289 at four levels (Lavi et al., 27 Mar 2025).

“Laplacian Analysis Meets Dynamics Modelling: Gaussian Splatting for 4D Reconstruction” transfers spectral decomposition into motion modeling. It represents per-primitive motion as a low-frequency Laplacian/Fourier component plus a high-frequency hash residual,

$$C(p) = \sum_i \alpha_i(p)c_i(p)\prod_{j<i}(1-\alpha_j(p)).$$0

and fuses them with spectral-aware attention. The low-frequency branch is regularized by graph-Laplacian penalties, while the high-frequency branch receives only mild temporal Tikhonov smoothing. The method explicitly frames prior low-rank dynamic encodings as suffering from spectral conflict between deformation consistency and detail preservation. On HyperNeRF-vrig it reports average SSIM/PSNR/LPIPS of 0.720/25.83/0.253, improving over Grid4D 0.715/25.46/0.259 and 4DGaussians 0.681/25.05/0.346; on D-NeRF it reports 0.994/42.17/0.007 (Zhou et al., 7 Aug 2025).

A common misconception is that frequency decomposition is only a rendering convenience. In these papers it is also a control mechanism for optimization: low bands enforce global coherence, while high bands or residual branches prevent over-smoothing.

5. Decomposed space–time encodings and task-specific fields

Dynamic Gaussian splatting has also used decomposition at the level of space–time encoding. 4D-GS represents a scene with a single canonical set of 3D Gaussians and a time-varying deformation field built from six learned 2D planes over $$C(p) = \sum_i \alpha_i(p)c_i(p)\prod_{j<i}(1-\alpha_j(p)).$$1, $$C(p) = \sum_i \alpha_i(p)c_i(p)\prod_{j<i}(1-\alpha_j(p)).$$2, $$C(p) = \sum_i \alpha_i(p)c_i(p)\prod_{j<i}(1-\alpha_j(p)).$$3, $$C(p) = \sum_i \alpha_i(p)c_i(p)\prod_{j<i}(1-\alpha_j(p)).$$4, $$C(p) = \sum_i \alpha_i(p)c_i(p)\prod_{j<i}(1-\alpha_j(p)).$$5, and $$C(p) = \sum_i \alpha_i(p)c_i(p)\prod_{j<i}(1-\alpha_j(p)).$$6. Plane features are bilinearly sampled, multiplied within each resolution, concatenated across resolutions, and decoded into $$C(p) = \sum_i \alpha_i(p)c_i(p)\prod_{j<i}(1-\alpha_j(p)).$$7, $$C(p) = \sum_i \alpha_i(p)c_i(p)\prod_{j<i}(1-\alpha_j(p)).$$8, and $$C(p) = \sum_i \alpha_i(p)c_i(p)\prod_{j<i}(1-\alpha_j(p)).$$9 for each Gaussian. This factorization avoids storing a full per-frame Gaussian set and yields real-time dynamic rendering: 82 FPS at $SE(3)$0 on an RTX 3090 with about 18 MB storage, versus 418 MB for K-Planes and 440 MB for FFDNeRF in the reported synthetic setting (Wu et al., 2023).

Grid4D argues that plane-based 4D decompositions impose an inappropriate low-rank assumption and create excessive feature overlap. It replaces six 2D planes with one spatial 3D hash encoder $SE(3)$1 and three temporal 3D encoders $SE(3)$2, $SE(3)$3, and $SE(3)$4, then aggregates them through a directional attention mechanism

$SE(3)$5

The directional range $SE(3)$6 is used so that temporal features can be amplified or inverted depending on spatial context. Grid4D further adds smooth regularization directly in encoder feature space. On D-NeRF it reports mean PSNR/SSIM/LPIPS of 42.00/0.994/0.008, with “Grid4D + SC” reaching 42.41/0.994/0.008; on HyperNeRF it reports 25.50/0.856 on vrig and 28.56/0.933 on interp (Xu et al., 2024).

DEGSTalk shows that decomposed Gaussian splatting is not restricted to novel-view synthesis or autonomous driving. It uses field-level decomposition into separate face and mouth Gaussian fields, a preserved hair layer, and per-primitive embedding decomposition in which each Gaussian carries a learnable embedding $SE(3)$7 that conditions a deformation MLP. Hair is preserved by a dedicated composition stage,

$SE(3)$8

The method reports PSNR 37.30, LPIPS 0.0140, LMD 2.349, SSIM 0.9664, 114 FPS, and about 0.5h training time (Deng et al., 2024).

These examples broaden the scope of the term. Decomposition may refer to canonical-time encodings, motion bases, field separation, or compositional rendering specific to a domain such as talking-face synthesis.

6. Optimization patterns, evaluation, applications, and open issues

Despite their diversity, decomposed methods share several optimization patterns. They usually keep the standard splatting rasterizer and jointly optimize Gaussian geometry, opacity-like parameters, appearance features, and decomposition-specific variables such as instance poses, permutation matrices, geometry opacities, band levels, spectral coefficients, or auxiliary fields. Photometric supervision is commonly combined with SSIM-like terms, geometry or depth regularization, motion smoothness, opacity or scale priors, and density-control procedures such as cloning, splitting, pruning, or densification. At the framework level, LiteGS shows that decomposition can also be applied to the training pipeline itself, factorizing the computation into clustering, cluster culling, cluster compaction, projection, binning, rasterization, compact valid gradients, shared-memory multibatch reduction, sparse gradient assembly, and fused Adam. Without changing the core 3DGS projection or blending, it reports about 3.4× training speedup and approximately 30% lower GPU memory on Mip-NeRF 360 (Liao, 3 Mar 2025).

The applications of decomposed Gaussian splatting are correspondingly broad. IDSplat targets autonomous driving reconstruction, multi-sensor rendering, scene editing, and simulation (Lindström et al., 24 Nov 2025). DeSplat targets distractor-free novel view synthesis in real scenes (Wang et al., 2024). Intrinsic Gaussian decomposition targets multi-view-consistent texture editing while preserving lighting (Lanvin et al., 30 Jun 2026). Frequency-aware methods target stylization, progressive rendering, streaming, foveated rendering, and faster geometry interaction (Lavi et al., 27 Mar 2025). DEGSTalk targets long-hair-preserving talking-face synthesis (Deng et al., 2024).

The limitations are equally recurrent. Instance-level driving methods inherit the weaknesses of rigid-body assumptions, LiDAR field-of-view constraints, zero-shot masks, and missing track management; IDSplat reports lower cyclist DPSNR due to the rigid-body assumption and notes that camera-visible objects outside LiDAR FOV are not instantiated as dynamic (Lindström et al., 24 Nov 2025). Intrinsic decomposition with independent fields is slower than vanilla 3DGS because it requires multiple render calls and separate optimization stages, and its albedo quality depends on the quality of diffusion-based intrinsic predictions (Lanvin et al., 30 Jun 2026). BiGS avoids normals but pays a substantial appearance-parameter cost, and its SH/BSH basis is limited for very sharp shadows or highlights (Liu et al., 2024). Frequency-aware decomposition improves control but can slightly reduce final reconstruction metrics as the number of levels grows (Lavi et al., 27 Mar 2025). Dynamic space–time decompositions continue to negotiate a tension between low-rank smoothness, feature collision, and high-frequency motion fidelity, which later spectral methods explicitly identify as a central failure mode (Xu et al., 2024, Zhou et al., 7 Aug 2025).

Taken together, the literature does not support a single canonical definition of decomposed Gaussian splatting. Instead, it identifies a design pattern: explicit partitioning of a splat-based model into components whose roles are easier to constrain, optimize, or edit than those of a single entangled Gaussian field. That pattern has already produced distinct technical lineages—instance-aware, intrinsic, geometry-aware, spectral, dynamic, and pipeline-modular—and the current papers suggest that future work will continue to mix these axes rather than choose only one.