Deformable3DGS: Dynamic 3D Gaussian Splatting

• Deformable3DGS is a family of techniques that dynamically deform explicit 3D Gaussian primitives to capture motion and complex scene deformations.
• It employs methods such as canonical-space deformation, proxy-driven editing, and user-guided manipulation to update attributes like position, rotation, scale, and opacity.
• These approaches enable real-time dynamic view synthesis, surgical simulation, and 3D tracking, demonstrating broad applicability in research and practical applications.

In the 3D Gaussian Splatting literature, dynamic scene modeling with 3DGS is often called Deformable3DGS. The core need is to enable the Gaussians—which are explicit scene primitives with position, rotation, scale, color, and opacity—to evolve over time, capturing scene motion and deformation. Because 3DGS is explicit and rasterized rather than sampled through a dense implicit field, recent work uses deformable Gaussian representations for dynamic view synthesis, geometry editing, physical simulation, and other downstream applications (Wu et al., 2024).

1. Representation scope and main design families

The recent literature organizes deformable 3DGS around a small number of recurring design patterns. The survey literature distinguishes direct attribute animation, canonical-space models with learned deformation fields, explicit temporal parametric models, and geometry-editing pipelines that bind Gaussians to meshes, cages, or other proxies (Wu et al., 2024). This suggests that “Deformable3DGS” is best understood as a family of representations rather than a single algorithm.

Paradigm	Representative papers	Characteristic mechanism
Canonical + deformation field	GaGS (Lu et al., 2024), SurgicalGaussian (Xie et al., 2024), E-D3DGS (Bae et al., 2024)	canonical Gaussians are warped by time-conditioned deformation models
Mesh/cage/proxy-driven deformation	Mesh-based GS (Gao et al., 2024), GSDeformer (Huang et al., 2024), UniMGS (Xiao et al., 27 Jan 2026)	explicit topology or proxy geometry drives Gaussian updates
User-driven editing	Sketch-guided cage-based deformation (Xie et al., 2024), ARAP-GS (Han et al., 17 Apr 2025), CAGE-GS (Tong et al., 17 Apr 2025)	sketches, drags, or target shapes specify deformation goals
Efficient or predictive dynamic models	DGS-LRM (Lin et al., 11 Jun 2025), SpeeDe3DGS (Tu et al., 9 Jun 2025), SD-GS (Yao et al., 10 Jul 2025)	feed-forward prediction, structured anchors, pruning, grouping

A persistent technical theme across these families is that deformation affects not only Gaussian centers but also covariance, rotation, opacity, and sometimes appearance or physically based material parameters. A second theme is that the explicitness of 3DGS makes editing and manipulation straightforward, but also makes the representation sensitive to pose errors, temporal artifacts, and proxy quality unless additional geometric or physical constraints are imposed (Wu et al., 2024).

2. Canonical-space deformation fields and temporal parameterizations

A dominant formulation decomposes a dynamic scene into a canonical field and a deformation field. In the geometry-aware formulation of GaGS, each Gaussian can independently move, scale, and rotate over time, and deformation is predicted from fused geometry-aware features together with positional encodings of space and time:

$$\Delta \mathbf{x}_t,\Delta \mathbf{r}_t,\Delta \mathbf{s}_t = \mathcal{G}_\Phi(F_{\text{fuse}},\gamma(\mathbf{x}),\gamma(t)),$$

with transformed parameters

$$\mathbf{x}_t=\mathbf{x}+\Delta \mathbf{x}_t,\qquad \mathbf{s}_t=\mathbf{s}+\Delta \mathbf{s}_t,\qquad \mathbf{r}_t=f_{\text{V2M}}(\Delta \mathbf{r}_t)\times f_{\text{V2M}}(\mathbf{r}).$$

GaGS explicitly extracts 3D geometry features with a sparse 3D U-Net, uses a 6D continuous rotation representation, and applies temporal-dependent density control to transformed rather than canonical Gaussians (Lu et al., 2024).

SurgicalGaussian follows the same canonical-to-observed paradigm but uses a forward-mapping deformation MLP. Canonical Gaussians are stored in a reference space, and for each frame $t$ the network predicts

$$(\delta \mathbf{x}, \delta \mathbf{s}, \delta \mathbf{q}) = \mathbf{F}_\Theta(\gamma(\mathbf{x}_c),\gamma(t)),$$

with

$$\mathbf{x}_o=\mathbf{x}_c+\delta \mathbf{x},\qquad \mathbf{s}_o=\mathbf{s}_c\cdot \exp(\delta \mathbf{s}),\qquad \mathbf{q}_o=\mathbf{q}_c\cdot \delta \mathbf{q}.$$

The deformation network is an MLP with 8 hidden layers of width 256, and the system couples this with depth initialization, tool mask-guided training, and local deformation regularization for endoscopic scenes (Xie et al., 2024).

E-D3DGS challenges the coordinate-based deformation field itself. It attributes reconstruction failure in prior dynamic 3DGS methods to parameter drift and entanglement caused by coordinate-based functions, and instead defines deformation as a function of per-Gaussian embeddings and temporal embeddings:

$$\mathcal{F}_\theta:(z_i,z_t)\rightarrow(\Delta x,\Delta r,\Delta s,\Delta \sigma,\Delta Y).$$

Each Gaussian receives a learnable embedding $z_i\in\mathbb{R}^{32}$, each frame a shared embedding $z_t\in\mathbb{R}^{256}$, and deformation is decomposed into coarse and fine components to model slow and fast motion separately (Bae et al., 2024). This suggests that deformable 3DGS increasingly treats motion as Gaussian-indexed structure rather than merely a coordinate-conditioned warp.

Deform3DGS represents another temporal parameterization strategy. Instead of MLPs or heavy feature planes, it models the time evolution of each Gaussian attribute as a linear combination of learnable Gaussian kernels,

$$\tilde{b}(t;\theta,\sigma)=\exp\left(-\frac{1}{2\sigma^2}(t-\theta)^2\right),$$

with 17 basis functions per attribute in experiments. The paper presents this as a flexible deformation modeling scheme at the level of individual Gaussians (Yang et al., 2024).

3. Geometry-aware, mesh-bound, and proxy-based deformation

Another major direction introduces explicit topology or proxy geometry into 3DGS. “Mesh-based Gaussian Splatting for Real-time Large-scale Deformation” defines 3D Gaussians over an explicit mesh and binds the two representations bidirectionally (Gao et al., 2024). A Gaussian center is parameterized on a triangle face by barycentric coordinates and a learnable normal offset,

$$\mu=(w_a \mathbf{v}_a+w_b \mathbf{v}_b+w_c \mathbf{v}_c)+\tau R\mathbf{n}.$$

Rendered Gaussian quality guides adaptive mesh face splitting, mesh face splits direct Gaussian splitting, and explicit mesh constraints suppress poor-quality Gaussians such as misaligned Gaussians and long-narrow shaped Gaussians. The method further regularizes Gaussian scale with

$$\mathbf{x}_t=\mathbf{x}+\Delta \mathbf{x}_t,\qquad \mathbf{s}_t=\mathbf{s}+\Delta \mathbf{s}_t,\qquad \mathbf{r}_t=f_{\text{V2M}}(\Delta \mathbf{r}_t)\times f_{\text{V2M}}(\mathbf{r}).$$0

and reports a high frame rate of 65 FPS on average (Gao et al., 2024).

GSDeformer connects cage-based deformation and 3DGS through a proxy point-cloud representation generated from 3D Gaussians. Each Gaussian is converted to an isocontour ellipsoid, represented by four proxy points, deformed in cage coordinates, and mapped back to Gaussian parameters via an affine transform $$\mathbf{x}_t=\mathbf{x}+\Delta \mathbf{x}_t,\qquad \mathbf{s}_t=\mathbf{s}+\Delta \mathbf{s}_t,\qquad \mathbf{r}_t=f_{\text{V2M}}(\Delta \mathbf{r}_t)\times f_{\text{V2M}}(\mathbf{r}).$$1 (Huang et al., 2024). A splitting process addresses bending, the approach does not modify or extend the core architecture of 3D Gaussian Splatting, and it is compatible with any trained vanilla 3DGS or its variants. The same paper also automates cage construction with a render-and-reconstruct pipeline based on voxelization, marching cubes, smoothing, and decimation (Huang et al., 2024).

UniMGS pushes proxy-based deformation further with a Gaussian-centric binding strategy. Rather than optimizing Gaussians to fit a proxy mesh, it starts from an already trained 3DGS and binds each Gaussian to mesh faces through ray casting from training cameras. To improve robustness, the method associates not only Gaussian centers but also oriented bounding-box corners, typically yielding bindings to eight faces, and transfers deformation by averaging per-corner transformations. The Gaussian update takes the form

$$\mathbf{x}_t=\mathbf{x}+\Delta \mathbf{x}_t,\qquad \mathbf{s}_t=\mathbf{s}+\Delta \mathbf{s}_t,\qquad \mathbf{r}_t=f_{\text{V2M}}(\Delta \mathbf{r}_t)\times f_{\text{V2M}}(\mathbf{r}).$$2

The authors argue that this avoids retraining, reduces sensitivity to mesh defects, and lowers rendering artifacts relative to mesh-centric schemes (Xiao et al., 27 Jan 2026). A common misconception is that proxy-based deformation is inherently mesh-centric; UniMGS explicitly reverses that dependency (Xiao et al., 27 Jan 2026).

4. Interactive editing and controllable deformation

User-facing editing systems turn deformable 3DGS from a reconstruction primitive into an authoring primitive. “Sketch-guided Cage-based 3D Gaussian Splatting Deformation” allows a user to draw a silhouette sketch from a single viewpoint and optimizes a cage deformation so that the deformed 3DGS matches the sketch while remaining semantically plausible across views (Xie et al., 2024). The system combines cage-based deformation, a variant of Neural Jacobian Fields, ControlNet-conditioned image generation, and a 3D-aware SDS loss from Zero123-XL. For each Gaussian,

$$\mathbf{x}_t=\mathbf{x}+\Delta \mathbf{x}_t,\qquad \mathbf{s}_t=\mathbf{s}+\Delta \mathbf{s}_t,\qquad \mathbf{r}_t=f_{\text{V2M}}(\Delta \mathbf{r}_t)\times f_{\text{V2M}}(\mathbf{r}).$$3

where $$\mathbf{x}_t=\mathbf{x}+\Delta \mathbf{x}_t,\qquad \mathbf{s}_t=\mathbf{s}+\Delta \mathbf{s}_t,\qquad \mathbf{r}_t=f_{\text{V2M}}(\Delta \mathbf{r}_t)\times f_{\text{V2M}}(\mathbf{r}).$$4 is the deformation Jacobian. The paper reports that deformation retains 99% of original image quality versus 75% for MLP+HexPlane according to CLIP-IQA (Xie et al., 2024).

ARAP-GS introduces drag-driven editing by directly adapting As-Rigid-As-Possible deformation to 3D Gaussian centers (Han et al., 17 Apr 2025). A representative subset of Gaussians is optimized under ARAP energy with handle constraints, and the remaining Gaussians inherit position and rotation through $$\mathbf{x}_t=\mathbf{x}+\Delta \mathbf{x}_t,\qquad \mathbf{s}_t=\mathbf{s}+\Delta \mathbf{s}_t,\qquad \mathbf{r}_t=f_{\text{V2M}}(\Delta \mathbf{r}_t)\times f_{\text{V2M}}(\mathbf{r}).$$5-nearest-neighbor interpolation. Because geometric deformation alone causes color bleed and blur, ARAP-GS adds iterative fine-tuning with the StableSR diffusion prior. The reported Dragging Accuracy Index is 0.0968, user preference is 77.8%, GPT-4o image scoring reaches 8.07/10, and end-to-end editing requires 10 to 20 minutes on a single RTX 3090 GPU (Han et al., 17 Apr 2025).

CAGE-GS learns a deformation cage from a user-defined target shape and aligns a source 3DGS to that target (Tong et al., 17 Apr 2025). Its target modalities include texts, images, point clouds, meshes, and 3DGS models. Because preserving textural appearance is difficult if only centers move, the method updates covariance parameters analytically through the local Jacobian,

$$\mathbf{x}_t=\mathbf{x}+\Delta \mathbf{x}_t,\qquad \mathbf{s}_t=\mathbf{s}+\Delta \mathbf{s}_t,\qquad \mathbf{r}_t=f_{\text{V2M}}(\Delta \mathbf{r}_t)\times f_{\text{V2M}}(\mathbf{r}).$$6

and then recovers rotation and scale by SVD. On the user study reported in the paper, CAGE-GS receives 63.3% of votes, compared with 21.7% for GSDeformer, 11.7% for NeuralCage, and 3.3% for cycle-consistency (Tong et al., 17 Apr 2025). Across interactive editors, a recurrent technical point is that center-only deformation is insufficient for high-fidelity appearance preservation.

5. Domain-driven systems: surgery and deformable-object tracking

The surgical domain has become a concentrated testbed for deformable 3DGS because endoscopic scenes combine monocular ambiguity, soft-tissue deformation, and instrument occlusion. SurgicalGaussian models spatio-temporal tissue features at each time stamp with a forward-mapping deformation MLP, introduces Gaussian Initialization with Depth and Mask, and applies tool mask-guided losses so that the model reconstructs only tissue and removes surgical instruments (Xie et al., 2024). The paper reports real-time rendering at $$\mathbf{x}_t=\mathbf{x}+\Delta \mathbf{x}_t,\qquad \mathbf{s}_t=\mathbf{s}+\Delta \mathbf{s}_t,\qquad \mathbf{r}_t=f_{\text{V2M}}(\Delta \mathbf{r}_t)\times f_{\text{V2M}}(\mathbf{r}).$$7 FPS on RTX3090, GPU usage of 3–4 GB, and superior rendering quality, rendering speed, and GPU usage compared to previous neural or Gaussian methods (Xie et al., 2024).

Deform3DGS emphasizes intraoperative latency. It introduces Motion-Aware Point Fusion for point-cloud initialization and a flexible deformation modeling scheme defined directly on individual Gaussians (Yang et al., 2024). On DaVinci robotic surgery videos, the paper reports PSNR 37.90, rendering speed 338.80 FPS, and training time 64 s on EndoNeRF; on StereoMIS it reports PSNR 30.48, rendering speed 330.37 FPS, and training time 66 s. The authors present these numbers as evidence that deformable 3DGS can move from offline reconstruction toward time-critical clinical workflows (Yang et al., 2024).

Outside surgery, DeformGS addresses scene flow in highly deformable scenes for robot manipulation. It learns a deformation function from canonical Gaussian space into world space with neural-voxel encoding and an MLP, and adds physics-inspired regularization based on local rigidity, local isometry, and conservation of momentum (Duisterhof et al., 2023). The paper reports an average 55.8% improvement in 3D tracking compared to the state of the art and a median tracking error of 3.3 mm on a cloth of 1.5 x 1.5 m in area when sufficient texture is available (Duisterhof et al., 2023). This suggests that Deformable3DGS is not limited to view synthesis; it also functions as an explicit state representation for dense 3D tracking.

6. Physical, multimodal, and analytic extensions

Some of the most consequential extensions move beyond pure geometry editing. DR-GS introduces a unified Gaussian framework for physically-based inverse rendering, relighting, and deformation-aware manipulation (Li et al., 28 Jun 2026). It explicitly disentangles geometry, illumination, and material, stores material parameters such as albedo, roughness, and metallicity, evaluates appearance with the rendering equation using Monte Carlo integration and Multiple Importance Sampling, and updates Gaussians under particle- or mesh-driven deformation through deformation gradients and GMLS interpolation. The paper frames this as a remedy for the baked-in illumination problem of earlier deformable GS systems and reports reliable preservation of reflections and specular highlights on glossy surfaces (Li et al., 28 Jun 2026).

DEGS extends deformable 3DGS to a hybrid RGB-plus-event setting. It uses event motion priors extracted by the LoCM unsupervised fine-tuning framework, establishes geometry-aware event-Gaussian motion correspondence through depth-based association and nearest Gaussian assignment, decomposes motion into camera ego-motion and scene flow, and supplements RGB supervision with event-based pseudo-labels at intermediate timepoints (He et al., 9 Oct 2025). The paper states that this joint optimization outperforms existing image and event-based approaches across synthetic and real scenes, particularly under large and fast deformation where low-framerate RGB alone is ambiguous (He et al., 9 Oct 2025).

VolSegGS shows that deformable Gaussians can support analysis tasks in dynamic volumetric scenes rather than only rendering. It uses view-independent Gaussian colors for coarse segmentation, refines with an affinity field network for fine segmentation, and embeds segmentation results within the Gaussians so that deformation yields continuous tracking of segmented regions over time (Yao et al., 16 Jul 2025). The paper reports real-time rendering and segmentation/tracking under low computational demands, indicating that deformable Gaussian representations can serve exploratory visualization as well as reconstruction (Yao et al., 16 Jul 2025).

7. Efficiency, feed-forward prediction, and open problems

A central practical issue is that dynamic 3DGS often requires per-Gaussian neural inference at every frame. SpeeDe3DGS addresses this bottleneck with temporal sensitivity pruning and GroupFlow (Tu et al., 9 Jun 2025). The method prunes low-contribution Gaussians via a second-order sensitivity score, adds annealing smooth pruning for robustness under imprecise poses, and clusters Gaussians by trajectory similarity so that each group shares one rigid transformation per frame. On NeRF-DS, the paper reports $$\mathbf{x}_t=\mathbf{x}+\Delta \mathbf{x}_t,\qquad \mathbf{s}_t=\mathbf{s}+\Delta \mathbf{s}_t,\qquad \mathbf{r}_t=f_{\text{V2M}}(\Delta \mathbf{r}_t)\times f_{\text{V2M}}(\mathbf{r}).$$8 faster rendering, $$\mathbf{x}_t=\mathbf{x}+\Delta \mathbf{x}_t,\qquad \mathbf{s}_t=\mathbf{s}+\Delta \mathbf{s}_t,\qquad \mathbf{r}_t=f_{\text{V2M}}(\Delta \mathbf{r}_t)\times f_{\text{V2M}}(\mathbf{r}).$$9 smaller models, and $t$0 shorter training; it further reports $t$1 and $t$2 rendering speedups on D-NeRF and HyperNeRF vrig (Tu et al., 9 Jun 2025).

SD-GS attacks the same problem through representation structure. It introduces a deformable anchor grid in which each anchor point derives multiple 3D Gaussians in its local spatiotemporal region, plus a deformation-aware densification strategy that grows anchors in under-reconstructed high-dynamic regions while reducing redundancy in static areas (Yao et al., 10 Jul 2025). Compared to state-of-the-art methods, the paper reports an average 60% reduction in model size and an average 100% improvement in FPS, with 82 FPS on N3DV and 79.7 FPS on HyperNeRF, using 22 MB and 43 MB respectively (Yao et al., 10 Jul 2025).

DGS-LRM shifts from optimization to prediction. It is presented as the first feed-forward method predicting deformable 3D Gaussian splats from a monocular posed video of any dynamic scene, using a 24-layer multi-head self-attention transformer and a per-pixel deformable 3D Gaussian representation with explicit scene flow (Lin et al., 11 Jun 2025). The paper reports 0.5 seconds reconstruction time on an A100 GPU, dynamic scene reconstruction quality comparable to optimization-based methods, a 6.1 PSNR improvement over L4GM in dynamic mask regions on DyCheck, and 3D tracking performance on par with state-of-the-art monocular video 3D tracking methods (Lin et al., 11 Jun 2025).

The survey literature identifies the outstanding open problems as robustness under large deformations and limited views, generalization and data efficiency, independent editing of geometry, texture, and lighting, physical plausibility, scalability, interoperability with mesh and field representations, and higher-dimensional generative models (Wu et al., 2024). Recent papers make these pressures concrete: cage extraction can take several minutes and diffusion remains computationally expensive in sketch-guided editing (Xie et al., 2024); proxy-mesh topology quality can still induce artifacts in joint mesh–3DGS systems (Xiao et al., 27 Jan 2026); and feed-forward dynamic models still report weaker behavior under discontinuous camera motion or extreme deformations outside the training distribution (Lin et al., 11 Jun 2025). Taken together, the field is moving toward hybrid systems in which explicit Gaussian primitives are coupled with stronger geometry, stronger physics, and stronger priors rather than treated as an unstructured set of splats.