SD-GS: Structured Deformable 3D Gaussians

• SD-GS is a dynamic Gaussian splatting framework that uses a sparse deformable anchor grid to generate time-dependent 3D Gaussians for efficient scene reconstruction.
• It achieves an average 60% reduction in model size and 100% improvement in FPS, maintaining or surpassing visual quality compared to state-of-the-art methods.
• The method employs deformation-aware densification and a two-stage training process to balance storage efficiency with high motion fidelity in dynamic environments.

Searching arXiv for the specified SD-GS paper and closely related Gaussian-splatting work. SD-GS denotes “Structured Deformable 3D Gaussians,” a dynamic Gaussian splatting framework for complex dynamic scene reconstruction that replaces large sets of independent 3D Gaussian primitives with a sparse grid of deformable anchors (Yao et al., 10 Jul 2025). The method is presented as a compact, anchor-based representation in which each anchor serves as the geometric backbone of a local spatiotemporal region and spawns multiple time-dependent Gaussians whose positions, shapes, colors, and opacities are predicted on the fly (Yao et al., 10 Jul 2025). Within the broader literature on 4D Gaussian methods, SD-GS addresses the storage–expressivity trade-off that arises when high visual fidelity and real-time rendering are pursued using explicit per-Gaussian dynamic representations (Yao et al., 10 Jul 2025). Experimental results reported for SD-GS indicate an average of $60\%$ reduction in model size and an average of $100\%$ improvement in FPS while maintaining or surpassing visual quality relative to state-of-the-art methods (Yao et al., 10 Jul 2025).

1. Position within dynamic Gaussian splatting

Dynamic scene reconstruction with Gaussian splatting extends the real-time rendering paradigm of 3D Gaussian Splatting to time-varying scenes, but explicit dynamic Gaussian models can incur substantial storage costs when they attempt to capture complex motions. SD-GS is introduced specifically to address this limitation through a structured representation built around a deformable anchor grid rather than a large cloud of independent time-varying Gaussians (Yao et al., 10 Jul 2025).

The core motivation is closely related to a recurrent issue in the 4D Gaussian literature: the need to balance compactness against motion fidelity. The paper frames prior dynamic “4D” approaches as relying on an enormous cloud of independent 3D Gaussian primitives, whereas SD-GS substitutes a sparse hierarchy of anchors with learned local offsets and a deformation field (Yao et al., 10 Jul 2025). This design is intended to reduce redundancy in static areas while preserving capacity in regions undergoing complex motion.

A useful comparison point is SDD-4DGS, which focuses on probabilistic static–dynamic decoupling inside a 4D Gaussian reconstruction pipeline through a learnable dynamic perception coefficient $w_i$ and uncertainty-guided supervision (Sun et al., 12 Mar 2025). That framework emphasizes separating static and dynamic components, whereas SD-GS emphasizes a compact anchor scaffold and deformation-aware densification (Yao et al., 10 Jul 2025). This suggests that dynamic Gaussian splatting methods are differentiating along two main axes: representational structure and motion-specific allocation of model capacity.

2. Deformable anchor grid representation

The central representational unit in SD-GS is the deformable anchor. SD-GS begins by initializing a sparse set of anchor points from Structure-from-Motion (COLMAP), typically a few tens of thousands per scene (Yao et al., 10 Jul 2025). Each anchor $i$ stores a 3D location $\mathbf{x}_i\in\mathbb{R}^3$, a 6-vector of base scales $\mathbf{l}_i\in\mathbb{R}^6$, a unit quaternion $\mathbf{q}_i\in\mathbb{R}^4$ for view-frustum culling, a context feature $\mathbf{f}_i\in\mathbb{R}^{32}$, and $k$ learnable 3D offsets $\{\mathbf{o}_{i,j}\in\mathbb{R}^3\}_{j=1}^k$ (Yao et al., 10 Jul 2025).

Rather than storing the parameters of the $100\%$0 Gaussians directly, SD-GS predicts each Gaussian mean from its host anchor: $100\%$1 This parameterization makes the anchor a local geometric support from which a small field of Gaussians can be derived (Yao et al., 10 Jul 2025). Because $100\%$2 rescales the offsets and $100\%$3 later scales the covariance, a single anchor can generate multiple related Gaussians without storing them as independent entities (Yao et al., 10 Jul 2025).

The paper describes this arrangement as a hierarchical and memory-efficient scene representation. A plausible implication is that the anchor grid imposes a structured prior over local geometry and motion: local Gaussians inherit coarse organization from anchors, while fine-grained variation is expressed through learned offsets and predicted per-Gaussian attributes. This differs from methods that optimize each Gaussian as an effectively independent primitive.

3. Dynamic deformation and neural Gaussian instantiation

To model temporal variation, SD-GS introduces a deformation field $100\%$4 parameterized by a spatial-temporal encoder $100\%$5 and three MLP heads $100\%$6 (Yao et al., 10 Jul 2025). Given anchor features and a time $100\%$7, the framework computes

$100\%$8

followed by

$100\%$9

and the deformed anchor state

$w_i$0

These equations show that deformation is expressed at the anchor level rather than independently for every Gaussian (Yao et al., 10 Jul 2025).

At each rendered frame $w_i$1, SD-GS instantiates neural Gaussians from the deformed anchors. Each Gaussian $w_i$2 has a mean $w_i$3, covariance

$w_i$4

opacity $w_i$5, and color $w_i$6 (Yao et al., 10 Jul 2025). Its density contribution is given as

$w_i$7

Colors and opacities are predicted by four small MLPs $w_i$8 conditioned on anchor features, view direction, and temporal embedding (Yao et al., 10 Jul 2025). Rendering uses differentiable Gaussian splatting, with each projected Gaussian contributing a Gaussian footprint in image space and pixels accumulated in front-to-back order through

$w_i$9

The paper states that gradients flow through all parameters end-to-end (Yao et al., 10 Jul 2025).

4. Deformation-aware densification

A distinctive component of SD-GS is its deformation-aware densification strategy, which aims to allocate anchors preferentially to under-reconstructed high-dynamic regions while reducing redundancy in static areas (Yao et al., 10 Jul 2025). The method departs from uniform growth in high-error regions by weighting the usual image-space Gaussian gradient by anchor motion.

Over $i$0 optimization steps, SD-GS collects

$i$1

where $i$2 is the 2D positional gradient of Gaussians spawned by anchor $i$3 at iteration $i$4, and

$i$5

Here $i$6 are normalization constants defined as the 90th percentile of each deformation type, and $i$7 (Yao et al., 10 Jul 2025).

Anchors that move or deform heavily over recent optimization steps therefore receive greater gradient weight (Yao et al., 10 Jul 2025). If $i$8 exceeds a growth threshold $i$9, new anchors are spawned in the local neighborhood; if it falls below a static pruning threshold $\mathbf{x}_i\in\mathbb{R}^3$0, under-utilized anchors are removed (Yao et al., 10 Jul 2025). The paper summarizes this rule as follows:

$\{\mathbf{o}_{i,j}\in\mathbb{R}^3\}_{j=1}^k$5

This mechanism is explicitly intended to balance model capacity against storage by concentrating anchors in high-dynamic, under-reconstructed regions (Yao et al., 10 Jul 2025). The contrast with pruning-oriented compact 3DGS methods is instructive. GS$\mathbf{x}_i\in\mathbb{R}^3$1, for example, combines ELBO-guided densification, opacity-aware pruning, and graph-based spatial redistribution for compact static-scene Gaussian representations (Yang et al., 2 Apr 2026). SD-GS uses a related compactness objective in spirit, but its triggering signal is dynamic deformation rather than compression-oriented reconstruction-complexity trade-offs.

5. Training procedure and objective

SD-GS is trained in two stages (Yao et al., 10 Jul 2025). In a coarse static stage, anchors are fixed and the model optimizes a purely static canonical model $\mathbf{x}_i\in\mathbb{R}^3$2 over all multi-view frames, providing initial coverage of the scene. In the dynamic stage, the framework enables the deformation field $\mathbf{x}_i\in\mathbb{R}^3$3, the temporal branches of the neural Gaussian decoder $\mathbf{x}_i\in\mathbb{R}^3$4, and the densification process (Yao et al., 10 Jul 2025).

The total loss is

$\mathbf{x}_i\in\mathbb{R}^3$5

where $\mathbf{x}_i\in\mathbb{R}^3$6 is the pixel-wise $\mathbf{x}_i\in\mathbb{R}^3$7 photometric error, $\mathbf{x}_i\in\mathbb{R}^3$8 is the structural similarity term with $\mathbf{x}_i\in\mathbb{R}^3$9, $\mathbf{l}_i\in\mathbb{R}^6$0 is a grid-based spatiotemporal total variation term with $\mathbf{l}_i\in\mathbb{R}^6$1, and $\mathbf{l}_i\in\mathbb{R}^6$2 is a volumetric regularizer with $\mathbf{l}_i\in\mathbb{R}^6$3 (Yao et al., 10 Jul 2025). All anchor parameters and MLP weights are optimized jointly using Adam on a single NVIDIA RTX 3090, and typical convergence requires 80–90 minutes per scene (Yao et al., 10 Jul 2025).

The two-stage organization distinguishes SD-GS from methods whose principal innovation lies in explicit static/dynamic classification. SDD-4DGS, for instance, introduces a binary-entropy regularized decoupling coefficient, progressive constraints, and an automatic supervision loss $\mathbf{l}_i\in\mathbb{R}^6$4 based on uncertainty masks (Sun et al., 12 Mar 2025). SD-GS instead retains a unified dynamic model but alters the representational substrate and densification policy. This suggests that SD-GS treats motion complexity primarily as a capacity-allocation problem rather than as a latent segmentation problem.

6. Empirical performance, ablations, and limitations

The reported results place SD-GS among high-efficiency dynamic Gaussian reconstruction methods. On the N3DV benchmark, SD-GS achieves $\mathbf{l}_i\in\mathbb{R}^6$5 dB PSNR, $\mathbf{l}_i\in\mathbb{R}^6$6 SSIM, and $\mathbf{l}_i\in\mathbb{R}^6$7 LPIPS, compared with 4DGS at $\mathbf{l}_i\in\mathbb{R}^6$8 dB/$\mathbf{l}_i\in\mathbb{R}^6$9/$\mathbf{q}_i\in\mathbb{R}^4$0 and Realtime-4DGS at $\mathbf{q}_i\in\mathbb{R}^4$1 dB/$\mathbf{q}_i\in\mathbb{R}^4$2/$\mathbf{q}_i\in\mathbb{R}^4$3 (Yao et al., 10 Jul 2025). It renders at $\mathbf{q}_i\in\mathbb{R}^4$4 FPS, versus $\mathbf{q}_i\in\mathbb{R}^4$5 FPS for Realtime-4DGS and $\mathbf{q}_i\in\mathbb{R}^4$6 FPS for 4DGS, while average model size is $\mathbf{q}_i\in\mathbb{R}^4$7 MB, compared with $\mathbf{q}_i\in\mathbb{R}^4$8 MB for 4DGS and $\mathbf{q}_i\in\mathbb{R}^4$9 GB for Realtime-4DGS (Yao et al., 10 Jul 2025). On HyperNeRF, the paper reports $\mathbf{f}_i\in\mathbb{R}^{32}$0 dB PSNR, $\mathbf{f}_i\in\mathbb{R}^{32}$1 SSIM, and $\mathbf{f}_i\in\mathbb{R}^{32}$2 LPIPS at $\mathbf{f}_i\in\mathbb{R}^{32}$3 FPS with a $\mathbf{f}_i\in\mathbb{R}^{32}$4 MB model (Yao et al., 10 Jul 2025).

The paper’s summary statistics emphasize efficiency: an average of $\mathbf{f}_i\in\mathbb{R}^{32}$5 reduction in model size and an average of $\mathbf{f}_i\in\mathbb{R}^{32}$6 improvement in FPS while maintaining or surpassing visual quality (Yao et al., 10 Jul 2025). It also states that scenes of a few seconds of video can fit in $\mathbf{f}_i\in\mathbb{R}^{32}$7–$\mathbf{f}_i\in\mathbb{R}^{32}$8 MB, compared to $\mathbf{f}_i\in\mathbb{R}^{32}$9 MB–$k$0 GB for explicit 4D-Gaussian methods (Yao et al., 10 Jul 2025).

Ablation results on a “flame_steak” scene isolate the contribution of specific components. Removing deformation-aware densification drops PSNR to $k$1 from $k$2, increases anchors by $k$3, and raises storage from $k$4 MB to $k$5 MB (Yao et al., 10 Jul 2025). Omitting temporal injection in $k$6 reduces PSNR to $k$7, and disabling position deformation $k$8 is reported as most harmful, yielding PSNR $k$9, whereas scale $\{\mathbf{o}_{i,j}\in\mathbb{R}^3\}_{j=1}^k$0 and rotation $\{\mathbf{o}_{i,j}\in\mathbb{R}^3\}_{j=1}^k$1 have smaller but measurable effects (Yao et al., 10 Jul 2025).

The limitations identified for SD-GS are also explicit. The method requires tuning of densification thresholds $\{\mathbf{o}_{i,j}\in\mathbb{R}^3\}_{j=1}^k$2 and deformation weights $\{\mathbf{o}_{i,j}\in\mathbb{R}^3\}_{j=1}^k$3, and it may face challenges in scenes with extremely fast, small-scale motions that still outpace the coarse anchor grid (Yao et al., 10 Jul 2025). A plausible implication is that the anchor abstraction, while compact, introduces a scale-selection problem: if anchor granularity is too coarse relative to motion frequency, compactness can constrain temporal precision.

7. Relation to adjacent Gaussian-splatting research

Within the Gaussian-splatting literature, SD-GS belongs to a family of methods that modify the primitive structure or optimization dynamics to address specific deficiencies of baseline 3DGS and 4DGS pipelines. Spectral-GS introduces spectral entropy for shape-aware splitting and view-consistent filtering to suppress needle-like artifacts in static-scene 3DGS (Huang et al., 2024). DiGS embeds Signed Distance Field learning into 3DGS to obtain more accurate and complete surface reconstruction while retaining high rendering fidelity (Guo et al., 9 Sep 2025). GS$\{\mathbf{o}_{i,j}\in\mathbb{R}^3\}_{j=1}^k$4 focuses on compact static-scene rendering through ELBO-based adaptive densification, opacity-aware pruning, and graph-based spatial distribution optimization (Yang et al., 2 Apr 2026). SDI-GS addresses sparse-view initialization by segmentation-driven down-sampling of dense point clouds before Gaussian optimization (Li et al., 15 Sep 2025). SDD-4DGS addresses dynamic scenes through probabilistic static–dynamic decoupling rather than anchor-based compression (Sun et al., 12 Mar 2025).

Against this background, SD-GS is distinguished by combining a structured anchor scaffold with time-conditioned deformation and motion-weighted densification (Yao et al., 10 Jul 2025). Its representational economy derives not from post hoc pruning alone, nor from explicit static/dynamic separation, but from parameter sharing across locally organized groups of Gaussians. This suggests a broader methodological trend in Gaussian splatting: moving from flat collections of primitives toward structured latent organizations—anchors, graphs, SDF-coupled centers, or probabilistic decoupling variables—that encode geometric or temporal inductive bias directly into the representation.

The paper’s own stated future directions include learned density pruning, multi-scale anchors, and tighter integration of geometric priors (Yao et al., 10 Jul 2025). In the context of related work, these directions are consistent with the broader progression of Gaussian splatting research toward representations that are simultaneously compact, controllable, and better aligned with scene geometry and motion structure.