---
title: 'SD-GS: Structured Deformable 3D Gaussians'
url: https://www.emergentmind.com/topics/sd-gs
type: topic
---

# SD-GS: Structured Deformable 3D Gaussians

Searching arXiv for the specified SD-GS paper and closely related Gaussian-splatting work.
SD-GS denotes “Structured Deformable 3D Gaussians,” a dynamic Gaussian splatting framework for complex dynamic scene reconstruction that replaces large sets of independent 3D Gaussian primitives with a sparse grid of deformable anchors [2507.07465]. The method is presented as a compact, anchor-based representation in which each anchor serves as the geometric backbone of a local spatiotemporal region and spawns multiple time-dependent Gaussians whose positions, shapes, colors, and opacities are predicted on the fly [2507.07465]. Within the broader literature on 4D Gaussian methods, SD-GS addresses the storage–expressivity trade-off that arises when high visual fidelity and real-time rendering are pursued using explicit per-Gaussian dynamic representations [2507.07465]. Experimental results reported for SD-GS indicate an average of \(60\%\) reduction in model size and an average of \(100\%\) improvement in FPS while maintaining or surpassing visual quality relative to state-of-the-art methods [2507.07465].

## 1. Position within dynamic Gaussian splatting

Dynamic scene reconstruction with Gaussian splatting extends the real-time rendering paradigm of 3D Gaussian Splatting to time-varying scenes, but explicit dynamic Gaussian models can incur substantial storage costs when they attempt to capture complex motions. SD-GS is introduced specifically to address this limitation through a structured representation built around a deformable anchor grid rather than a large cloud of independent time-varying Gaussians [2507.07465].

The core motivation is closely related to a recurrent issue in the 4D Gaussian literature: the need to balance compactness against motion fidelity. The paper frames prior dynamic “4D” approaches as relying on an enormous cloud of independent 3D Gaussian primitives, whereas SD-GS substitutes a sparse hierarchy of anchors with learned local offsets and a deformation field [2507.07465]. This design is intended to reduce redundancy in static areas while preserving capacity in regions undergoing complex motion.

A useful comparison point is SDD-4DGS, which focuses on probabilistic static–dynamic decoupling inside a 4D Gaussian reconstruction pipeline through a learnable dynamic perception coefficient \(w_i\) and uncertainty-guided supervision [2503.09332]. That framework emphasizes separating static and dynamic components, whereas SD-GS emphasizes a compact anchor scaffold and deformation-aware densification [2507.07465]. This suggests that dynamic Gaussian splatting methods are differentiating along two main axes: representational structure and motion-specific allocation of model capacity.

## 2. Deformable anchor grid representation

The central representational unit in SD-GS is the deformable anchor. SD-GS begins by initializing a sparse set of anchor points from Structure-from-Motion (COLMAP), typically a few tens of thousands per scene [2507.07465]. Each anchor \(i\) stores a 3D location \(\mathbf{x}_i\in\mathbb{R}^3\), a 6-vector of base scales \(\mathbf{l}_i\in\mathbb{R}^6\), a unit quaternion \(\mathbf{q}_i\in\mathbb{R}^4\) for view-frustum culling, a context feature \(\mathbf{f}_i\in\mathbb{R}^{32}\), and \(k\) learnable 3D offsets \(\{\mathbf{o}_{i,j}\in\mathbb{R}^3\}_{j=1}^k\) [2507.07465].

Rather than storing the parameters of the \(k\) Gaussians directly, SD-GS predicts each Gaussian mean from its host anchor:
\[
\mu_{i,j} \;=\; \mathbf{x}_i \;+\; \mathrm{diag}(\mathbf{l}_{i,1:3})\,\mathbf{o}_{i,j}\,.
\]
This parameterization makes the anchor a local geometric support from which a small field of Gaussians can be derived [2507.07465]. Because \(\mathbf{l}_{i,1:3}\) rescales the offsets and \(\mathbf{l}_{i,4:6}\) later scales the covariance, a single anchor can generate multiple related Gaussians without storing them as independent entities [2507.07465].

The paper describes this arrangement as a hierarchical and memory-efficient scene representation. A plausible implication is that the anchor grid imposes a structured prior over local geometry and motion: local Gaussians inherit coarse organization from anchors, while fine-grained variation is expressed through learned offsets and predicted per-Gaussian attributes. This differs from methods that optimize each Gaussian as an effectively independent primitive.

## 3. Dynamic deformation and neural Gaussian instantiation

To model temporal variation, SD-GS introduces a deformation field \(\mathcal{F}\) parameterized by a spatial-temporal encoder \(\mathcal{H}\) and three MLP heads \(\phi_x,\phi_l,\phi_q\) [2507.07465]. Given anchor features and a time \(t\), the framework computes
\[
\mathbf{f}_i^{(t)} \;=\; \mathcal{H}(\mathbf{x}_i,\mathbf{l}_i,\mathbf{q}_i,\mathbf{f}_i,\;t)\,,
\]
followed by
\[
(\Delta \mathbf{x}_i,\;\Delta \mathbf{l}_i,\;\Delta \mathbf{q}_i)\;=\;\bigl(\phi_x(\mathbf{f}_i^{(t)}),\;\phi_l(\mathbf{f}_i^{(t)}),\;\phi_q(\mathbf{f}_i^{(t)})\bigr)\,,
\]
and the deformed anchor state
\[
\mathbf{x}_i' \;=\;\mathbf{x}_i + \Delta \mathbf{x}_i,\quad
\mathbf{l}_i' \;=\;\mathbf{l}_i + \Delta \mathbf{l}_i,\quad
\mathbf{q}_i' \;=\;\mathrm{Normalize}\bigl(\mathbf{q}_i + \Delta \mathbf{q}_i\bigr)\,.
\]
These equations show that deformation is expressed at the anchor level rather than independently for every Gaussian [2507.07465].

At each rendered frame \(t\), SD-GS instantiates neural Gaussians from the deformed anchors. Each Gaussian \(G_{i,j}\) has a mean \(\mu_{i,j}\in\mathbb{R}^3\), covariance
\[
\Sigma_{i,j} = R(\mathbf{r}_{i,j})\,\mathrm{diag}(\mathbf{s}_{i,j})^2\,R(\mathbf{r}_{i,j})^\top,
\]
opacity \(\alpha_{i,j}\in[0,1]\), and color \(\mathbf{c}_{i,j}\in\mathbb{R}^3\) [2507.07465]. Its density contribution is given as
\[
G_{i,j}(\mathbf{x})
\;=\;
\alpha_{i,j}\,\exp\!\Bigl(-(\mathbf{x}-\mu_{i,j})^\top\Sigma_{i,j}^{-1}(\mathbf{x}-\mu_{i,j})\Bigr)\,.
\]

Colors and opacities are predicted by four small MLPs \(F_\alpha,F_c,F_r,F_s\) conditioned on anchor features, view direction, and temporal embedding [2507.07465]. Rendering uses differentiable Gaussian splatting, with each projected Gaussian contributing a Gaussian footprint in image space and pixels accumulated in front-to-back order through
\[
C_{\rm pixel}
=\sum_{i,j} \mathbf{c}_{i,j}\,\alpha_{i,j}\,\prod_{(p,q)\,<\,(i,j)}\bigl(1-\alpha_{p,q}\bigr)\,.
\]
The paper states that gradients flow through all parameters end-to-end [2507.07465].

## 4. Deformation-aware densification

A distinctive component of SD-GS is its deformation-aware densification strategy, which aims to allocate anchors preferentially to under-reconstructed high-dynamic regions while reducing redundancy in static areas [2507.07465]. The method departs from uniform growth in high-error regions by weighting the usual image-space Gaussian gradient by anchor motion.

Over \(N\) optimization steps, SD-GS collects
\[
\nabla g_i \;=\;\frac{\sum_{k=1}^N w_{i,k}\,\|\nabla_{\!2D}^k\|}{\sum_{k=1}^N w_{i,k}}\,,
\]
where \(\nabla_{\!2D}^k\) is the 2D positional gradient of Gaussians spawned by anchor \(i\) at iteration \(k\), and
\[
w_{i,k} =\alpha\,\frac{\|\Delta \mathbf{x}_i^{(k)}\|}{s_x}
+\beta\,\frac{\|\Delta \mathbf{l}_i^{(k)}\|}{s_l}
+\gamma\,\frac{\|\Delta \mathbf{q}_i^{(k)}\|}{s_q}\,.
\]
Here \(s_x,s_l,s_q\) are normalization constants defined as the 90th percentile of each deformation type, and \(\alpha=0.8,\beta=0.1,\gamma=0.1\) [2507.07465].

Anchors that move or deform heavily over recent optimization steps therefore receive greater gradient weight [2507.07465]. If \(\nabla g_i\) exceeds a growth threshold \(\tau_{\rm grow}\), new anchors are spawned in the local neighborhood; if it falls below a static pruning threshold \(\tau_{\rm prune}\), under-utilized anchors are removed [2507.07465]. The paper summarizes this rule as follows:

```text
for each anchor i every N steps:
   compute ∇g_i via weighted gradients
   if ∇g_i > τ_grow: spawn new anchor(s) near x_i'
   if ∇g_i < τ_prune: remove anchor i
```

This mechanism is explicitly intended to balance model capacity against storage by concentrating anchors in high-dynamic, under-reconstructed regions [2507.07465]. The contrast with pruning-oriented compact 3DGS methods is instructive. GS\(^2\), for example, combines ELBO-guided densification, opacity-aware pruning, and graph-based spatial redistribution for compact static-scene Gaussian representations [2604.01884]. SD-GS uses a related compactness objective in spirit, but its triggering signal is dynamic deformation rather than compression-oriented reconstruction-complexity trade-offs.

## 5. Training procedure and objective

SD-GS is trained in two stages [2507.07465]. In a coarse static stage, anchors are fixed and the model optimizes a purely static canonical model \(\mathcal{M}\) over all multi-view frames, providing initial coverage of the scene. In the dynamic stage, the framework enables the deformation field \(\mathcal{F}\), the temporal branches of the neural Gaussian decoder \(\mathcal{N}\), and the densification process [2507.07465].

The total loss is
\[
\mathcal{L}
=\mathcal{L}_1(\text{RGB})
+\lambda_{\rm SSIM}\,\mathcal{L}_{\rm SSIM}
+\lambda_{\rm tv}\,\mathcal{L}_{\rm tv}
+\lambda_{\rm vol}\,\mathcal{L}_{\rm vol},
\]
where \(\mathcal{L}_1\) is the pixel-wise \(L_1\) photometric error, \(\mathcal{L}_{\rm SSIM}\) is the structural similarity term with \(\lambda_{\rm SSIM}=0.2\), \(\mathcal{L}_{\rm tv}\) is a grid-based spatiotemporal total variation term with \(\lambda_{\rm tv}=0.01\), and \(\mathcal{L}_{\rm vol}\) is a volumetric regularizer with \(\lambda_{\rm vol}=0.01\) [2507.07465]. All anchor parameters and MLP weights are optimized jointly using Adam on a single NVIDIA RTX 3090, and typical convergence requires 80–90 minutes per scene [2507.07465].

The two-stage organization distinguishes SD-GS from methods whose principal innovation lies in explicit static/dynamic classification. SDD-4DGS, for instance, introduces a binary-entropy regularized decoupling coefficient, progressive constraints, and an automatic supervision loss \(\mathcal{L}_{asg}\) based on uncertainty masks [2503.09332]. SD-GS instead retains a unified dynamic model but alters the representational substrate and densification policy. This suggests that SD-GS treats motion complexity primarily as a capacity-allocation problem rather than as a latent segmentation problem.

## 6. Empirical performance, ablations, and limitations

The reported results place SD-GS among high-efficiency dynamic Gaussian reconstruction methods. On the N3DV benchmark, SD-GS achieves \(31.35\) dB PSNR, \(0.942\) SSIM, and \(0.047\) LPIPS, compared with 4DGS at \(30.94\) dB/\(0.936\)/\(0.056\) and Realtime-4DGS at \(31.11\) dB/\(0.939\)/\(0.050\) [2507.07465]. It renders at \(82\) FPS, versus \(46\) FPS for Realtime-4DGS and \(35\) FPS for 4DGS, while average model size is \(22\) MB, compared with \(59\) MB for 4DGS and \(7.97\) GB for Realtime-4DGS [2507.07465]. On HyperNeRF, the paper reports \(25.79\) dB PSNR, \(0.737\) SSIM, and \(0.221\) LPIPS at \(79.7\) FPS with a \(43\) MB model [2507.07465].

The paper’s summary statistics emphasize efficiency: an average of \(60\%\) reduction in model size and an average of \(100\%\) improvement in FPS while maintaining or surpassing visual quality [2507.07465]. It also states that scenes of a few seconds of video can fit in \(20\)–\(30\) MB, compared to \(60\) MB–\(8\) GB for explicit 4D-Gaussian methods [2507.07465].

Ablation results on a “flame_steak” scene isolate the contribution of specific components. Removing deformation-aware densification drops PSNR to \(32.61\) from \(33.09\), increases anchors by \(30\%\), and raises storage from \(22\) MB to \(26\) MB [2507.07465]. Omitting temporal injection in \(\mathcal{N}\) reduces PSNR to \(31.15\), and disabling position deformation \(\Delta\mathbf{x}\) is reported as most harmful, yielding PSNR \(30.20\), whereas scale \(\Delta\mathbf{l}\) and rotation \(\Delta\mathbf{q}\) have smaller but measurable effects [2507.07465].

The limitations identified for SD-GS are also explicit. The method requires tuning of densification thresholds \(\tau_{\rm grow},\tau_{\rm prune}\) and deformation weights \((\alpha,\beta,\gamma)\), and it may face challenges in scenes with extremely fast, small-scale motions that still outpace the coarse anchor grid [2507.07465]. A plausible implication is that the anchor abstraction, while compact, introduces a scale-selection problem: if anchor granularity is too coarse relative to motion frequency, compactness can constrain temporal precision.

## 7. Relation to adjacent Gaussian-splatting research

Within the Gaussian-splatting literature, SD-GS belongs to a family of methods that modify the primitive structure or optimization dynamics to address specific deficiencies of baseline 3DGS and 4DGS pipelines. Spectral-GS introduces spectral entropy for shape-aware splitting and view-consistent filtering to suppress needle-like artifacts in static-scene 3DGS [2409.12771]. DiGS embeds Signed Distance Field learning into 3DGS to obtain more accurate and complete surface reconstruction while retaining high rendering fidelity [2509.07493]. GS\(^2\) focuses on compact static-scene rendering through ELBO-based adaptive densification, opacity-aware pruning, and graph-based spatial distribution optimization [2604.01884]. SDI-GS addresses sparse-view initialization by segmentation-driven down-sampling of dense point clouds before Gaussian optimization [2509.11853]. SDD-4DGS addresses dynamic scenes through probabilistic static–dynamic decoupling rather than anchor-based compression [2503.09332].

Against this background, SD-GS is distinguished by combining a structured anchor scaffold with time-conditioned deformation and motion-weighted densification [2507.07465]. Its representational economy derives not from post hoc pruning alone, nor from explicit static/dynamic separation, but from parameter sharing across locally organized groups of Gaussians. This suggests a broader methodological trend in Gaussian splatting: moving from flat collections of primitives toward structured latent organizations—anchors, graphs, SDF-coupled centers, or probabilistic decoupling variables—that encode geometric or temporal inductive bias directly into the representation.

The paper’s own stated future directions include learned density pruning, multi-scale anchors, and tighter integration of geometric priors [2507.07465]. In the context of related work, these directions are consistent with the broader progression of Gaussian splatting research toward representations that are simultaneously compact, controllable, and better aligned with scene geometry and motion structure.

Source: https://www.emergentmind.com/topics/sd-gs