---
title: Dynamic 3D Gaussian Splatting
url: https://www.emergentmind.com/topics/dynamic-3d-gaussian-splatting-3dgs
type: topic
---

# Dynamic 3D Gaussian Splatting

Dynamic 3D Gaussian Splatting (Dynamic 3DGS) extends the explicit, rasterization-based 3D Gaussian Splatting representation into the time domain, so that a scene is modeled not only as a set of anisotropic Gaussian primitives in space but as a temporally varying radiance field whose geometry, opacity, and appearance may evolve over time. In the literature, the central design goals are temporal coherence, real-time rendering, and scalability with sequence length; the resulting method family spans per-Gaussian motion models, canonical-space deformation fields, explicit sparse temporal keyframes, compact time bases, and streaming or hierarchical schemes for long sequences [2407.17418] [2510.26694].

## 1. Mathematical foundations and rendering model

Dynamic 3DGS inherits the core primitive of static 3DGS: a scene is represented by Gaussians indexed by \(i\), each with mean \(\mu_i \in \mathbb{R}^3\), covariance \(\Sigma_i \in \mathbb{R}^{3\times 3}\), opacity, and view-dependent color features. A standard parameterization writes the covariance as
\[
\Sigma_i = R_i S_i S_i^\top R_i^\top,
\]
where \(R_i\) is a rotation and \(S_i\) is diagonal with scale radii. The corresponding volumetric density is
\[
G_i(x)=\exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right).
\]
Projection to image space uses a local Jacobian of the camera mapping, yielding a 2D elliptical footprint whose screen-space covariance is of the form
\[
\Sigma_i^{\text{screen}} = J \Sigma_i J^\top
\quad\text{or}\quad
\Sigma'_i = J W \Sigma_i W^\top J^\top,
\]
depending on whether the viewing transform is written explicitly. Rendering then proceeds by front-to-back alpha compositing of depth-sorted splats, with transmittance \(T_i\) and weights \(w_i\) defined by multiplicative visibility factors, and pixel color obtained by summing weighted colors [2508.21444] [2510.18101].

The dynamic extension replaces static attributes by time-varying ones:
\[
\mu_i(t),\ \Sigma_i(t),\ \alpha_i(t),\ c_i(t).
\]
This time dependence may be expressed directly at the Gaussian level, through rigid or non-rigid transforms, through a canonical-space deformation map, or through compact time bases. Not every method makes every attribute time-dependent. For example, one compact representation models positions by Fourier approximation and rotations by linear quaternion functions while keeping scale, color, and opacity invariant across time; this reduces dynamic memory from \(O(TN)\) to \(O(LN)\), where \(T\) is the number of timesteps and \(L\) is the number of time-basis parameters per Gaussian [2311.12897].

A common misconception is that Dynamic 3DGS changes the rendering principle itself. In fact, the surveys emphasize that the rendering core remains the same tile-based, depth-ordered splatting pipeline used in static 3DGS; the essential change is the introduction of time-varying Gaussian attributes and the optimization machinery needed to estimate them robustly [2403.11134].

## 2. Temporal parameterizations and scene dynamics

The literature organizes Dynamic 3DGS primarily by how temporal variation is parameterized. Surveys distinguish per-Gaussian rigid \(SE(3)\) motion, velocity or polynomial models, deformation graphs and sparse control points, skeletal or part-based models, and continuous 4D encodings that treat time as an additional domain variable [2407.17418].

| Family | Representative mechanism | Example papers |
|---|---|---|
| Time functions per Gaussian | Fourier or polynomial trajectories, linear quaternion time functions | [2311.12897], [2408.03822] |
| Canonical-space deformation | Static canonical Gaussians warped by learned time-dependent deformation fields | [2403.11134], [2403.11447] |
| Sparse temporal states | Explicit positions and rotations at sparse timestamps with interpolation | [2410.15629] |

A compact time-function formulation represents each coordinate of the mean by a Fourier series,
\[
x(t)=w_{x,0}+\sum_{i=1}^{L} w_{x,2i-1}\sin(2\pi i t)+w_{x,2i}\cos(2\pi i t),
\]
with analogous expressions for \(y(t)\) and \(z(t)\), while each quaternion component is linear in \(t\). This parameterization shares motion parameters across time, relaxes the strict multi-view assumption, and supports monocular as well as multi-view reconstruction [2311.12897].

Another compact formulation, built on STG, uses a canonical position \(sp_n\), canonical rotation \(sr_n\), and scale \(s_n\), then models position and rotation at time \(t\) by polynomial bases around a temporal center \(\mu_n\):
\[
p_n(t)=sp_n+\sum_{k=1}^{no_p} u_{n,k}(t-\mu_n)^k,\qquad
r_n(t)=sr_n+\sum_{k=1}^{no_r} v_{n,k}(t-\mu_n)^k.
\]
Temporal visibility is handled by
\[
o_n(t)=so_n \exp\!\big(-\xi_n |t-\mu_n|^2\big),
\]
which localizes a Gaussian’s active duration in time [2408.03822].

A more explicit trajectory model stores sparse states at keyframes and interpolates between them. In Fully Explicit Dynamic Gaussian Splatting, dynamic Gaussians store positions and rotations only at sparse timestamps; positions are interpolated with cubic Hermite spline (CHip), rotations with quaternion Slerp, and temporal visibility with a piecewise two-Gaussian schedule. Static Gaussians are initially allowed a global linear motion, and only the top-\(\eta\) percent by normalized image-space motion criterion are converted to dynamic Gaussians, with empirical \(\eta = 2\) [2410.15629].

The phrase “4DGS” is therefore not semantically uniform across the literature. In some works it refers to explicit time-dependent Gaussian attributes; in others it refers to canonical Gaussians coupled to deformation fields or to higher-dimensional spatiotemporal encodings. This suggests that “dynamic 3DGS” is better understood as a family of temporal parameterizations built on a common splatting renderer than as a single canonical model.

## 3. Optimization objectives, temporal priors, and supervision

The dominant training signal remains photometric reconstruction. Representative formulations use mixtures of \(L_1\) and SSIM or D-SSIM, for example
\[
L_{\text{recon}}=(1-\lambda)\|\hat I-I\|_1+\lambda L_{D\text{-}SSIM}(\hat I,I),
\]
with \(\lambda=0.2\) in the compact dynamic representation, or per-level photometric losses combined with SSIM in multi-scale streaming systems [2311.12897] [2508.21444].

Dynamic methods then add temporal structure. Surveyed regularizers include temporal smoothness, local rigidity, local rotation similarity, and long-term local isometry. In persistent-motion formulations these terms encourage neighboring Gaussians to move coherently while preserving local geometry over time; in deformation-field methods, they stabilize otherwise underconstrained spatiotemporal warps [2407.17418].

Optical flow is a recurring supervisory signal. One compact dynamic representation obtains adjacent-frame optical flow with RAFT and adds a bidirectional flow loss derived from projected 3D Gaussian center motion. Motion-aware 3D Gaussian Splatting goes further by explicitly establishing a correspondence between 3D Gaussian movements and pixel-level flow, using an uncertainty-aware heteroscedastic loss and a transient-aware deformation auxiliary module. Its total objectives are written as
\[
L_{\text{total}}^I=L_{\text{photo}}+\lambda_p L_{\text{phys}}+\lambda_{\text{flow}}L_{\text{flow}},
\qquad
L_{\text{total}}^D=L_{\text{photo}}+\lambda_{\text{flow}}L_{\text{flow\_aux}},
\]
for iterative and deformation-based paradigms respectively [2403.11447].

Masking is another major design axis. Scale-GS introduces bidirectional adaptive masking for streaming content: forward masking identifies dynamic anchors from temporal frame differences, while view masking ranks cameras by viewpoint novelty and directional weighting. The resulting loss at time \(t\) is
\[
\mathcal{L}_t=\sum_{u\in\mathcal{U}_t}\omega_t(c(u))\cdot M_{\text{static}}(u)\cdot \ell(I_t(u),I_t^{gt}(u)),
\]
and top-\(S(c_k)\) views form the active training batch [2508.21444].

Underwater reconstruction shows a different use of masking. UW-GS does not introduce time-conditioned Gaussian attributes or motion-aware losses; instead it uses a binary motion mask (BMM) to gate photometric losses and suppress moving distractors. The final mask is
\[
\omega^t=\omega_1^t\cup \omega_2^t\cup \omega_3^t,
\]
combining residual thresholding, spatial diffusion, and patch-level consistency, and is then applied to a masked photometric loss [2410.01517].

This divergence is important. In Dynamic 3DGS, “handling dynamics” can mean reconstructing temporal geometry and appearance, but it can also mean robustly excluding moving distractors from optimization when the modeling goal is the static scene behind them.

## 4. Efficiency, compactness, and streaming scalability

A central difficulty in Dynamic 3DGS is that naïve per-timestep parameter storage or per-frame optimization scales poorly. One compact dynamic representation makes this explicit: per-timestep dynamic 3DGS has \(O(TN)\) parameter growth, whereas its time-basis parameterization is \(O(LN)\), independent of sequence length. Reported memory drops are from 6.6 GB to about 338 MB on DyNeRF, from 913 MB to 159 MB on D-NeRF, and to about 720 MB on HyperNeRF, while rendering reaches \(118\) FPS at \(1{,}352\times 1{,}014\) on a single RTX A6000 GPU [2311.12897].

Compact 3D Gaussian Splatting for Static and Dynamic Radiance Fields attacks storage from a different angle. It combines a learnable mask strategy, a shared grid-based neural field for view-dependent color, and residual vector quantization (RVQ) of geometric and temporal attributes. On DyNeRF, it reports PSNR \(31.73\), SSIM \(0.945\), LPIPS \(0.053\), FPS \(186\), and storage \(21.8\) MB, while post-processing reduces storage to \(15.4\) MB; on Technicolor, the post-processed model reaches PSNR \(33.1\), SSIM \(0.910\), LPIPS \(0.098\), FPS \(116\), and \(0.16\) MB per frame [2408.03822].

Speedy Deformable 3D Gaussian Splatting targets the runtime bottleneck of per-Gaussian neural inference. It introduces temporal sensitivity pruning and GroupFlow, which clusters Gaussians by trajectory similarity and predicts a single rigid transformation per group. On NeRF-DS it reports \(10.37\times\) faster rendering, \(7.71\times\) smaller model size, and \(2.71\times\) faster training; on D-NeRF and HyperNeRF vrig, the reported rendering-speed improvements are \(4.20\times\) and \(58.23\times\), respectively [2506.07917].

Scale-GS addresses scalability in streaming and long multi-view sequences by organizing Gaussians hierarchically by scale within an anchor-based structure, activating finer levels only where gradient-driven gating warrants it, and combining deformation, spawning, and redundancy filtering. Its effective per-frame cost is
\[
O\!\left(M_{\text{active}}\rho \sum_l p_l N\right),
\]
where \(M_{\text{active}}\ll M\) is the number of selected cameras, \(\rho\) is the active pixel fraction after masking, and \(p_lN\) is the number of active Gaussians at level \(l\). Reported average per-frame training times are \(3.2\) s on NV3D, \(3.0\) s on MeetRoom, and \(37.3\) s on Google Immersive, with real-time FPS of \(274\), \(276\), and \(199\), respectively [2508.21444].

The practical implication is that Dynamic 3DGS efficiency is now addressed on at least three orthogonal fronts: compact temporal parameterizations, pruning and quantization of explicit attributes, and selective activation of Gaussians, views, or temporal regions during training.

## 5. Representative systems and application domains

Dynamic 3DGS has developed into a heterogeneous application layer rather than a single benchmark-driven niche. Surveys place dynamic view synthesis, free-viewpoint video, avatars, urban driving, streaming reconstruction, and robotics among the main use cases [2407.17418] [2410.12262].

In dynamic view synthesis, explicit trajectory models and canonical deformation backbones dominate. Fully Explicit Dynamic Gaussian Splatting reports \(62\) fps on a single 2080Ti GPU and \(120.6\) fps on an NVIDIA 4090 in the half-resolution N3V setting, with \(0.6\) hours training and a typical model size of \(115\) MB in N3V experiments [2410.15629]. The compact Fourier-based dynamic representation reports up to \(188\) FPS on HyperNeRF, \(150\) FPS on D-NeRF, and competitive monocular and multi-view quality [2311.12897].

Dynamic appearance modeling has become a distinct subproblem. Neural Texture Splatting augments each Gaussian with a local RGBA texture field generated by a shared global tri-plane and neural decoder, conditioned on position, view direction, and time. When paired with SplatFields4D on Owlii, it reports mean PSNR \(29.25\) versus \(27.87\) for SplatFields4D and \(26.45\) for 4DGaussians, with corresponding SSIM \(95.86\) and LPIPS \(6.70\); the method reduces floaters and boundary artifacts while recovering fine details such as faces and hands [2511.18873].

Some domain-specific variants redefine what “dynamic” means. UW-GS is designed for underwater scenes where moving fish and particulates act as distractors and where absorption and scattering alter the appearance model. It introduces a distance-dependent color transformation,
\[
c_i^m = T_i^D \cdot c_i + (1-T_i^B)\cdot b_i,
\]
a physics-based density control strategy, and the binary motion mask described above. The result is a distractor-aware underwater 3DGS rather than a general motion-reconstruction framework [2410.01517].

Robotics-oriented surveys emphasize that the explicit Gaussian representation is particularly compatible with online scene understanding, navigation, and manipulation because it supports object-centric updates, free-space reasoning, and real-time rendering. Dynamic urban methods such as PVG, VDG, DrivingGaussian, Street Gaussians, and HUGS are cited as examples of dynamic 3DGS in driving scenarios, while deformation-field methods such as 4DGaussians, 4DGF, SC-GS, Gaussian-Flow, SWAGS, and 3DGStream are highlighted for broader non-rigid or streaming settings [2410.12262].

## 6. Limitations, misconceptions, and open directions

The literature is consistent on several failure modes. Extremely large non-rigid deformations, topology changes, fast motion, heavy occlusions, and abrupt viewpoint changes remain difficult. In Scale-GS, missed activations can underfit fine motion, while over-activation increases cost; occlusion-heavy scenes can confuse forward masking and cause over-masking of useful areas or missed spawns behind newly revealed surfaces [2508.21444]. In compact time-function models, appearing or disappearing structures and fluids are difficult because Gaussians are assumed to exist across all times, and inaccurate camera poses degrade reconstruction quality [2311.12897].

A second persistent issue is ambiguity between scene motion and camera motion, especially in monocular settings. Motion-aware 3DGS addresses this with uncertainty-aware flow supervision and a transient-aware deformation auxiliary, but the paper still identifies severe occlusions, topological changes, strong motion blur, and long-range motion as challenging cases [2403.11447]. Robotics-oriented surveys likewise identify robust tracking, lifelong mapping, and large-scale relocalization as unresolved problems for dynamic Gaussian maps [2410.12262].

There is also a conceptual misconception in equating “4DGS” with a single architecture. Survey evidence shows that the label covers persistent per-Gaussian motion, canonical-plus-deformation models, explicit sparse temporal keyframes, space-time feature splatting, and temporal hierarchies for long sequences [2510.26694]. The common thread is not a unique 4D primitive but a shared reliance on explicit Gaussian splats, differentiable rasterization, and temporally structured optimization.

Open directions identified across the surveys include scaling to long videos and large scenes, better handling of complex deformations and topology changes, joint lighting and material modeling over time, robust monocular and sparse-view reconstruction, stronger editability and control, and tighter integration of compression, pruning, and hardware-aware rendering [2407.17418]. A plausible implication is that future Dynamic 3DGS systems will be increasingly hybrid: explicit splats for rendering efficiency, compact temporal bases or grouped motion for scalability, and learned spatial-temporal fields only where continuous deformation modeling is indispensable.

Source: https://www.emergentmind.com/topics/dynamic-3d-gaussian-splatting-3dgs