Dynamic Gaussian Splatting Techniques

Updated 19 October 2025

Dynamic Gaussian Splatting is a technique that models 4D scenes using time-evolving 3D or 4D Gaussian primitives to capture changes in geometry and appearance.
It employs deformation fields, keyframe interpolation, and motion graphs to accurately represent nonrigid motion and ensure temporal consistency in rendering.
Advanced regularization and compression strategies mitigate optimization brittleness, supporting applications in autonomous driving, robotic manipulation, and real-time rendering.

Dynamic Gaussian Splatting is a family of techniques that extends the explicit 3D Gaussian Splatting representation from static to dynamic scenes by modeling changes in geometric and appearance attributes of Gaussian primitives over time. These methods enable high-fidelity, temporally consistent novel view synthesis and scene reconstruction in scenarios with complex, nonrigid, or articulated motion—including settings such as autonomous driving, robotic manipulation, crowd simulation, and dynamic scene SLAM. The core innovation lies in representing 4D (space + time) scenes by evolving a collection of explicit 3D (or 4D) Gaussian ellipsoids, with their parameters dynamically predicted, interpolated, or transformed through canonical deformation fields, keyframe interpolation, kinematic skinning, motion graphs, or explicit spatio-temporal hybridization schemes.

1. Fundamental Representation and Dynamic Extension

Gaussian Splatting represents a scene as a set of anisotropic Gaussian primitives, each specified by a position μ, covariance Σ (capturing anisotropy and orientation, often reparameterized as Σ = R S SᵀRᵀ), color/appearance, and opacity. The rendered color per pixel, under differentiable splatting, is computed by alpha-compositing sorted, projected 2D Gaussians:

$C(u,v) = \sum_{j} c_j\, \alpha_j\, p_j(u,v) \prod_{k=1}^{j-1} (1 - \alpha_k\, p_k(u,v))$

with $p_j(u,v)$ the projected influence and $\alpha_j$ the pixel-dependent opacity.

Dynamic Gaussian Splatting augments this with temporal evolution:

Per-Gaussian time-dependent transformation: The mean $\mu_t$ and rotation $r_t$ at time $t$ are parameterized as $\mu_{t+1} = \mu_t + \Delta\mu_t$ , $r_{t+1} = r_t + \Delta r_t$ .
Canonical and deformation field approaches: Each Gaussian exists in a canonical space. Deformation is realized via MLPs, field-based models, or explicit transformation modules that take time (and often spatial location) as input, outputting $\Delta\mu_t$ and related changes (Shaw et al., 2023, Lu et al., 9 Apr 2024).
Explicit 4D (‘space-time’) Gaussian parameterization: Some methods define each Gaussian with a 4D mean and a 4×4 covariance, projecting to 3D space at a given $t$ via:

$\mu_{xyz|t} = \mu_{1:3} + \Sigma_{1:3,4} \Sigma_{4,4}^{-1}(t-\mu_t),\ \Sigma_{xyz|t} = \Sigma_{1:3,1:3} - \Sigma_{1:3,4} \Sigma_{4,4}^{-1} \Sigma_{4,1:3}$

(Oh et al., 19 May 2025).

Hybrid static/dynamic allocation: To reduce redundancy, hybrid frameworks assign 3D Gaussians to temporally invariant regions and retain full 4D Gaussians only in dynamic regions (Oh et al., 19 May 2025).

This explicit scene model allows for time-continuous rendering, rapid evaluation, and fine-grained motion handling, but presents new challenges in motion modeling, density/pruning, and dynamic scene consistency.

2. Deformation Fields, Motion Interpolation, and Dynamic Modeling

Dynamic Gaussian Splatting methods vary widely in their motion modeling:

Method Class	Motion Parametrization	Notable Features
Iterative per-frame	$\mu_{t} = \mu_{t-1} + \delta G_{i,t}$	Susceptible to error accumulation
Canonical + Field	$G_{i,t} = G_{i} + f_\theta(z_{i}, t)$	Field-based deformation, shared params
Explicit 4D	$(\mu_{i}, t)$ , $\Sigma_i\in\mathbb{R}^{4\times4}$	Flexible, can be underconstrained
Keyframe Interp.	Sparse sampling + CHip/Slerp interp	Reduces storage, ensures continuity
Motion Graph / DQS	Blending via dual quaternion skinning	Enables manipulation, explicit contol
Spectral/Laplacian	Sinusoidal/Laplace time expansion ( $L(t)$ )	Frequency-aware, prevents oversmooth.

Many frameworks learn a canonical static representation and train an MLP to predict per-timestep deformations, often leveraging NeRF-style positional encodings for spatio-temporal input (Shaw et al., 2023, Lu et al., 9 Apr 2024). Dynamic Gaussian Splatting also encompasses:

Sliding window training: Adaptive partitioning of temporal sequences, each modeled by a local field, mitigating long-range deformation drift (Shaw et al., 2023).
Explicit opacity and appearance modeling: Temporal opacity functions (e.g., Gaussian mixtures for appear/disappear events (Lee et al., 21 Oct 2024)) encode temporal support.
Explicit motion blending using graphs/skins: State-of-the-art methods (e.g., MBGS) use motion graphs and weight painting with dual quaternion skinning, allowing direct manipulation of underlying motions and facilitating animation or robotic control (Zhang et al., 12 Mar 2025).

3. Integration with Additional Scene Priors, Initialization, and Regularization

Successful dynamic Gaussian Splatting pipelines often integrate strong geometric or semantic priors and regularization mechanisms:

LiDAR or SfM priors: For large-scale outdoor or autonomous driving environment modeling, dynamic Gaussian Splatting frameworks, such as DrivingGaussian, utilize LiDAR-based geometric bins to robustly initialize and constrain background and foreground placement (Zhou et al., 2023).
Disentanglement and masking: Methods employ explicit per-Gaussian blending parameters $\alpha$ to separate static and dynamic regions in training (Shaw et al., 2023). Object discovery is performed using bounding boxes, segmentation models, or depth/optical flow fusion (Zhou et al., 2023, Li et al., 6 Jun 2025).
Regularization (ARAP, normal, Laplacian, etc.): As-rigid-as-possible (ARAP) constraints, normal regularization, and spectral-aware Laplacian encoding control nonphysical deformation and enforce motion smoothness without oversmoothing (Cai et al., 26 Aug 2024, Zhou et al., 7 Aug 2025).
Uncertainty-aware optimization: To handle underconstrained regions (e.g. due to occlusion or poor visibility), per-Gaussian uncertainty is estimated and used to build a spatio-temporal optimization graph, allowing reliable primitives to anchor the optimization and propagate motion to ambiguous ones (Guo et al., 14 Oct 2025).

These enhancements ensure structurally coherent dynamic reconstructions even when available views are limited or the scene exhibits high frequency, nonrigid motion.

4. Compression, Storage, and Real-Time Rendering

Dynamic Gaussians' time-varying nature increases the storage and computational burden relative to static models. Several compression and efficiency strategies are developed:

Sparsification and iterative pruning: Mask-based dynamic Gaussian pruning, as well as density and region-based adaptive control, reduce the set of active dynamic Gaussians (e.g., Top20 loss, clustering) (Wang et al., 1 Jul 2025, Javed et al., 7 Dec 2024).
Keypoint trajectory compression: Dynamic trajectories are stored sparsely and interpolated using the Ramer-Douglas-Peucker algorithm or wavelet-based transforms (Javed et al., 7 Dec 2024, Lee et al., 23 Jul 2025).
Mixed-precision quantization: Parameters are adaptively quantized based on their measured gradient impact on image quality (Javed et al., 7 Dec 2024).
End-to-end rate-distortion optimization: Recent work applies full RD loss frameworks (with L_dist for rendering quality, L_rate for entropy coding) and wavelet domain compression for dynamic Gaussians, achieving compression ratios up to 91× with little visual degradation (Lee et al., 23 Jul 2025).
Efficient rasterization: Unified GPU pipelines combine 3D and 4D Gaussians in rasterization, with explicit tile-based back-to-front compositing, allowing frame rates upwards of 60–160 FPS even in dynamic settings (Lee et al., 21 Oct 2024, Oh et al., 19 May 2025).

These advances enable deployment in real-time and resource-constrained scenarios—e.g., augmented/virtual reality and edge robotics—where storage, latency, and power are critical.

5. Benchmarking, Limitations, and Optimization Brittleness

Extensive benchmarking reveals key behaviors and trade-offs:

Trade-off: Flexibility vs. Robustness: Canonical field-based or low-order function (polynomial/Fourier/RBF) motion models are generally more robust and stable, especially in monocular settings or with narrow camera baselines. Fully 4D Gaussians, while maximally expressive, often struggle with optimization instability and overfitting, particularly when dynamic regions are not well constrained (Liang et al., 5 Dec 2024).
Optimization brittleness: All dynamic Gaussian methods benefit from efficiently parallel splatting rasterizers, but adaptive densification (cloning/splitting/pruning) introduces brittleness, with possible catastrophic failures in overpruning or optimization divergence (Liang et al., 5 Dec 2024).
Scene complexity and specularity: In complex real-world data, reflective and specular objects, inaccurate camera poses, and limited view baselines stress motion models; differences between approaches may be less pronounced compared to synthetic controlled settings (Liang et al., 5 Dec 2024).
Metrics and performance: Dynamic Gaussian Splatting methods achieve PSNR, SSIM, and LPIPS on par with or exceeding comparable neural radiance field and mesh-based approaches, often with superior temporal consistency, real-time rendering, and—when properly designed—robustness to dynamic occlusion and ambiguity (Zhou et al., 2023, Lu et al., 13 Mar 2024, Zhang et al., 12 Mar 2025, Sun et al., 29 Jan 2025).

6. Applications, Impact, and Future Directions

Dynamic Gaussian Splatting frameworks have demonstrated substantial practical advances:

Real-world 4D scene reconstruction: Autonomous driving (using composite dynamic graphs and LiDAR priors), robotic manipulation (Gaussian world models for predicting action-conditioned dynamics), dynamic SLAM (Gaussian Splatting-based real-time mapping in dynamic scenes), and real-time crowd rendering (LoD-aware splatting with skinning) are all enabled or advanced by these methods (Zhou et al., 2023, Lu et al., 13 Mar 2024, Kong et al., 16 Nov 2024, Sun et al., 29 Jan 2025).
Photorealistic dynamic synthesis: Methods such as DynaSurfGS deliver both mesh-quality geometric reconstructions and photorealistic view synthesis, addressing the historical trade-off between geometry-first and image-first pipelines (Cai et al., 26 Aug 2024).
Controllability and manipulation: By enabling explicit, sparse motion graphs and dual quaternion skinning, frameworks like MBGS support direct motion editing, retargeting, and robot learning from human demonstration (Zhang et al., 12 Mar 2025).
Compression and storage: RD-optimized, temporally compressed 4DGS pipelines make large-scale 4D content streaming and edge inference tractable (Lee et al., 23 Jul 2025, Javed et al., 7 Dec 2024).

Open directions include more stable and data-efficient motion representations for highly ambiguous monocular input, further exploitation of semantic priors for object discovery in complex dynamic scenes, and principled regularization strategies (including uncertainty modeling) to ensure robust synthesis under non-ideal conditions (Guo et al., 14 Oct 2025). A plausible implication is that as integration with semantic scene understanding and uncertainty modeling advances, dynamic Gaussian splatting will become central to deployed autonomous and mixed reality systems requiring real-time, photorealistic, and controllable 4D scene understanding.