---
title: Deformable 3D Gaussian Splatting
url: https://www.emergentmind.com/topics/deformable-3d-gaussian-splatting
type: topic
---

# Deformable 3D Gaussian Splatting

Deformable 3D Gaussian Splatting is a volumetric scene representation and deformation technique that synthesizes novel views of complex, potentially dynamic or edited 3D geometry through explicit, anisotropic 3D Gaussian primitives. While static 3D Gaussian Splatting provides highly efficient, real-time novel-view rendering for fixed geometry, extension to deformable settings requires precise alignment of geometry and preservation of high-frequency appearance during global or local deformations. Recent developments—including cage-based control, neural deformation fields, rigid-body rigging, and embedding-based temporal models—enable the seamless adaptation of existing 3DGS models to arbitrary target shapes or time-varying scenarios with little loss of fidelity.

## 1. Mathematical Foundations of Deformable 3D Gaussian Splatting

A deformable 3DGS scene is represented as a collection of N explicit, parameterized 3D Gaussians:  
$$
g_i = (\mu_i, \Sigma_i, c_i, \alpha_i)
$$  
where $\mu_i \in \mathbb{R}^3$ is the center, $\Sigma_i \in \mathbb{R}^{3 \times 3}$ the symmetric positive-definite covariance (typically factored as $\Sigma_i = R_i S_i S_i^\top R_i^\top$ for rotation $R_i$ and axis-aligned scale $S_i$), $c_i$ the (possibly view-dependent) color, and $\alpha_i$ the opacity or density. Rendering projects each Gaussian to the screen, computes per-pixel weights via the analytic projection of ellipsoids, sorts by depth, and composites using alpha-blending:  
$$
C(p) = \sum_{i=1}^N c_i \, \alpha_i \prod_{j<i} (1-\alpha_j)
$$

Deformation is modeled as a mapping from the canonical Gaussians (typically at $t=0$ or a reference shape) to a deformed configuration through one of several parameterizations:
- Cage-based deformation: associates each $\mu_i$ to a surrounding cage by mean-value/harmonic coordinates $\omega_i$; as the cage deforms, each $\mu_i$ is updated via  
  $$
  \mu_i' = \sum_{j=1}^M \omega_j(\mu_i) \cdot v_j'
  $$
  where $v_j'$ are new cage vertex positions. Covariances transform via the local deformation Jacobian $J(\mu_i)$:  
  $$
  \Sigma_i' = J(\mu_i) \Sigma_i J(\mu_i)^\top
  $$
  [2504.12800][2411.12168]
- Neural deformation fields: learn per-Gaussian deformations as functions of (potentially fused) geometric and temporal embeddings, e.g.,  
  $$
  (\Delta\mu_i, \Delta r_i, \Delta s_i) = F_\theta(e_i, t)
  $$
  with $e_i$ a per-Gaussian embedding, $t$ a temporal embedding (e.g., positional encoding or grid), and $F_\theta$ an MLP [2404.03613][2603.20857][2508.19786].
- Flexible basis curves: model temporal evolution of each Gaussian attribute (mean, covariance, opacity) by a sum of radial basis functions in time [2405.17835][2501.01101].
- Rigid/skinned body rigging: combine canonical-to-pose mapping via skinning (e.g., SMPL/SMPL-X) with per-Gaussian refinement via neural modules for articulated avatars [2312.15059][2312.09228].

The color and opacity are typically kept fixed under deformation to preserve high-frequency texture unless explicitly handled for dynamic appearance or view/time dependence.

## 2. Cage-Based and Hybrid Deformation Control

Cage-based deformation schemes place the 3DGS point cloud inside a "cage" mesh whose deformation induces a volumetric mapping propagated deterministically, thereby enabling:
- Intuitive global and semiglobal control (structural shape editing)
- Preservation of high-frequency textural detail via Jacobian-corrected covariance propagation, avoiding splat stretching or blurring [2504.12800][2411.12168]
- Flexible alignment to diverse target modalities: target shapes may be specified as text prompts (via proxy 3DGS), single images (AtlasNet, mesh proxy), point clouds, meshes, or 3DGS models, with shape alignment driven by PointNet-AtlasNet pipelines optimized for physical plausibility, coordinate positivity, and normal consistency.

Integration with neural Jacobian fields provides precise, per-face control of local differential mappings, increasing smoothness and regularity, particularly in user-guided or sketch-based editing scenarios [2411.12168]. However, cage-induced deformations cannot guarantee the preservation of strict invariants such as straight lines or sharp features for CAD-like geometries, and local manipulations may require alternative parameterizations (e.g., direct handle dragging).

## 3. Neural Deformation Fields and Per-Gaussian Embedding

Most dynamic or temporal 3DGS representations employ neural deformation fields, where each Gaussian's position and shape is predicted by MLPs taking as input per-Gaussian and temporal embeddings. Prominent designs include:
- Joint per-Gaussian and per-frame embeddings for accurate local deformation, decomposed into slow/fast (coarse/fine) pathways for modeling both gradual and abrupt motion [2404.03613][2603.20857][2508.19786].
- Geometry-aware deformation, in which features derived from local 3D neighborhoods (via sparse 3D U-Nets or PointNet) enrich prediction and enforce spatial consistency [2404.06270].
- Partitioning schemes, such as MAPo, that decouple highly dynamic Gaussians from quasi-static components via motion scoring, recursively specializing deformation networks for different spatiotemporal regimes [2508.19786].
- Physics-based or rigid-body deformation, e.g., deformable linear objects (ropes/cables) modeled with PBD solvers coupled with GS-based observation filtering [2505.08644], or mesh-rigged splats for intraoperative navigation with joint photometric and geometric supervision [2509.18501].

Ablations consistently show that neglecting per-Gaussian adaptation (e.g., using global or purely coordinate-based deformation) produces contamination across static/dynamic regions and loss of geometric detail [2508.19786][2404.03613].

## 4. Advanced Deformation Modeling: Temporal, View, and Hierarchical

Emerging techniques further refine deformable 3DGS models by addressing dynamic scene complexities:
- Temporal basis or RBF expansions for per-Gaussian attribute curves, supporting both reversible (elastic) and irreversible (shearing, topological change) deformations [2501.01101][2505.19138].
- Adaptive hierarchical motion masks, distinguishing static from deformable regions by evaluating deformation magnitude or rendering-loss differences, restricting neural inference and deformation updates to relevant Gaussians and maximizing rendering speed [2501.01101].
- View-dependent deformable fields: augment deformation conditioning with view/camera direction, enabling modeling of subtle appearance and emission changes, notably in domains with challenging radiometric phenomena (e.g., thermal-infrared) [2505.19138].
- Group trajectory analysis and rigid-group factorization (e.g., GroupFlow in SpeeDe3DGS), which clusters Gaussians by similar motion, replacing redundant per-Gaussian neural inference with compact per-group rigid transforms and dramatically accelerating rendering and reducing memory [2506.07917].

Quantitative evaluations indicate that these strategies yield state-of-the-art quality (e.g., PSNR $>$ 35 dB, real-time rendering $>$ 300 FPS) across diverse real and synthetic benchmarks while addressing scalability [2501.01101][2506.07917].

## 5. Applications: Editing, Medical Imaging, Tracking, and Animation

Deformable 3D Gaussian Splatting underpins several lines of application:
- Interactive 3D geometric editing and animation: cage-based, sketch-driven workflows enable scene-level deformation or animation through explicit or diffusion-prior guided systems, supporting modalities ranging from text and images to arbitrary sketch keyframes [2504.12800][2411.12168].
- Dynamic scene synthesis: per-Gaussian embedding fields, temporal decomposition, and deformation hierarchy enable photorealistic, real-time reconstruction and free-viewpoint rendering in scenes with large, non-rigid motion (e.g., articulated avatars, human-body performance capture, dynamic objects) [2312.15059][2312.09228][2404.03613][2508.19786].
- Real-time 3D medical reconstruction and navigation: explicit and flexible deformation modeling of tissue and organs, combined with rapid point-cloud initialization and segregation of static/deformable components, addresses intraoperative speed and fidelity requirements in endoscopic and surgical applications [2401.13352][2401.11535][2405.17835][2501.01101][2509.18501].
- Physics-guided and object-specific tracking: hybrid pipelines combine physics-based dynamics (e.g., PBD) with differentiable GS rendering, enabling robust 3D tracking under occlusion in robotics or manipulation scenarios [2505.08644].
- Dynamic transient and occlusion handling: deformable transient fields and memory-efficient masking allow high-quality synthesis in settings with occlusions, dynamic distractors, or uncontrolled capture [2503.06179].

## 6. Quantitative Evaluation, Limitations, and Future Directions

Benchmarks systematically report geometry and appearance fidelity (Chamfer distance, PSNR, SSIM, LPIPS), perceptual metrics (user study votes, DINO cosine), and practical speed/memory (FPS, training time, parameter count). For example, CAGE-GS achieves leading Chamfer and DINO scores on ShapeNet, and quantitative PSNR increases of 1–2 dB have been attributed to geometry-aware or embedding-based deformation compared to earlier coordinate-only or global models [2504.12800][2404.06270][2508.19786][2603.20857].

Key ablation findings:
- Omission of Jacobian-based covariance update produces severe blurring or texture drift [2504.12800].
- Single deformation network for the entire sequence induces over-averaging and blurry renderings in fast dynamic regions unless partitioned (MAPo, GroupFlow) [2508.19786][2506.07917].
- Explicit per-Gaussian embedding is essential to prevent leakage of fast motion into neighboring static geometry [2404.03613].

Limitations include:
- Incomplete preservation of strict geometric invariants (lines, planes) under global cage-based methods [2504.12800].
- Trade-off between per-Gaussian neural inference overhead and real-time constraint; group/rank pruning and trajectory factorization mitigate this [2506.07917].
- Residual temporal blur or artifacts in extreme motion regions or highly stochastic texture domains [2404.03613][2505.19138].

Future directions cited include:
- End-to-end learning of Jacobian fields and deformation priors.
- Interactive/VR deformation and editing interfaces.
- Deformation-aware neural surface coupling and mutual supervision for guaranteed geometry correspondence [2412.03910].
- Spatio-temporal deformation and frequency modeling in domains such as thermal or multispectral data [2505.19138].
- Extension to bounded-support kernels and new functional bases (e.g., deformable Beta splatting for adaptive kernel shape and improved densification) [2501.18630].

---

**References**:  
- [2504.12800] "CAGE-GS: High-fidelity Cage Based 3D Gaussian Splatting Deformation"  
- [2411.12168] "Sketch-guided Cage-based 3D Gaussian Splatting Deformation"  
- [2404.06270] "3D Geometry-aware Deformable Gaussian Splatting for Dynamic View Synthesis"  
- [2603.20857] "Fast and Robust Deformable 3D Gaussian Splatting"  
- [2506.07917] "Speedy Deformable 3D Gaussian Splatting: Fast Rendering and Compression of Dynamic Scenes"  
- [2508.19786] "MAPo : Motion-Aware Partitioning of Deformable 3D Gaussian Splatting for High-Fidelity Dynamic Scene Reconstruction"  
- [2501.01101] "Deformable Gaussian Splatting for Efficient and High-Fidelity Reconstruction of Surgical Scenes"  
- [2405.17835] "Deform3DGS: Flexible Deformation for Fast Surgical Scene Reconstruction with Gaussian Splatting"  
- [2404.03613] "Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting"  
- [2505.08644] "DLO-Splatting: Tracking Deformable Linear Objects Using 3D Gaussian Splatting"  
- [2312.15059] "Deformable 3D Gaussian Splatting for Animatable Human Avatars"  
- [2312.09228] "3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting"  
- [2412.03910] "DGNS: Deformable Gaussian Splatting and Dynamic Neural Surface for Monocular Dynamic 3D Reconstruction"  
- [2505.19138] "Veta-GS: View-dependent deformable 3D Gaussian Splatting for thermal infrared Novel-view Synthesis"  
- [2501.18630] "Deformable Beta Splatting"  
- [2401.13352], [2401.11535], [2509.18501], [2503.06179] (medical/tracking/editing/transient)

Source: https://www.emergentmind.com/topics/deformable-3d-gaussian-splatting