Dynamic 3D Gaussian Splatting

• Dynamic 3D Gaussian Splatting is a method that extends static 3D Gaussian representations into the time domain by evolving anisotropic primitives with dynamic geometry, opacity, and appearance.
• It employs various temporal parameterizations—such as Fourier series, polynomial models, and canonical deformation fields—to ensure temporal coherence and efficient rendering.
• The approach integrates optimization techniques, motion-aware losses, and selective activation strategies to enhance scalability and real-time performance in dynamic scene reconstruction.

Dynamic 3D Gaussian Splatting (Dynamic 3DGS) extends the explicit, rasterization-based 3D Gaussian Splatting representation into the time domain, so that a scene is modeled not only as a set of anisotropic Gaussian primitives in space but as a temporally varying radiance field whose geometry, opacity, and appearance may evolve over time. In the literature, the central design goals are temporal coherence, real-time rendering, and scalability with sequence length; the resulting method family spans per-Gaussian motion models, canonical-space deformation fields, explicit sparse temporal keyframes, compact time bases, and streaming or hierarchical schemes for long sequences (Bao et al., 2024, Kerbl, 30 Oct 2025).

1. Mathematical foundations and rendering model

Dynamic 3DGS inherits the core primitive of static 3DGS: a scene is represented by Gaussians indexed by $i$, each with mean $\mu_i \in \mathbb{R}^3$, covariance $\Sigma_i \in \mathbb{R}^{3\times 3}$, opacity, and view-dependent color features. A standard parameterization writes the covariance as

$\Sigma_i = R_i S_i S_i^\top R_i^\top,$

where $R_i$ is a rotation and $S_i$ is diagonal with scale radii. The corresponding volumetric density is

$$G_i(x)=\exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right).$$

Projection to image space uses a local Jacobian of the camera mapping, yielding a 2D elliptical footprint whose screen-space covariance is of the form

$$\Sigma_i^{\text{screen}} = J \Sigma_i J^\top \quad\text{or}\quad \Sigma'_i = J W \Sigma_i W^\top J^\top,$$

depending on whether the viewing transform is written explicitly. Rendering then proceeds by front-to-back alpha compositing of depth-sorted splats, with transmittance $T_i$ and weights $w_i$ defined by multiplicative visibility factors, and pixel color obtained by summing weighted colors (Yang et al., 29 Aug 2025, Matias et al., 20 Oct 2025).

The dynamic extension replaces static attributes by time-varying ones: $\mu_i \in \mathbb{R}^3$0 This time dependence may be expressed directly at the Gaussian level, through rigid or non-rigid transforms, through a canonical-space deformation map, or through compact time bases. Not every method makes every attribute time-dependent. For example, one compact representation models positions by Fourier approximation and rotations by linear quaternion functions while keeping scale, color, and opacity invariant across time; this reduces dynamic memory from $\mu_i \in \mathbb{R}^3$1 to $\mu_i \in \mathbb{R}^3$2, where $\mu_i \in \mathbb{R}^3$3 is the number of timesteps and $\mu_i \in \mathbb{R}^3$4 is the number of time-basis parameters per Gaussian (Katsumata et al., 2023).

A common misconception is that Dynamic 3DGS changes the rendering principle itself. In fact, the surveys emphasize that the rendering core remains the same tile-based, depth-ordered splatting pipeline used in static 3DGS; the essential change is the introduction of time-varying Gaussian attributes and the optimization machinery needed to estimate them robustly (Wu et al., 2024).

2. Temporal parameterizations and scene dynamics

The literature organizes Dynamic 3DGS primarily by how temporal variation is parameterized. Surveys distinguish per-Gaussian rigid $\mu_i \in \mathbb{R}^3$5 motion, velocity or polynomial models, deformation graphs and sparse control points, skeletal or part-based models, and continuous 4D encodings that treat time as an additional domain variable (Bao et al., 2024).

Family	Representative mechanism	Example papers
Time functions per Gaussian	Fourier or polynomial trajectories, linear quaternion time functions	(Katsumata et al., 2023, Lee et al., 2024)
Canonical-space deformation	Static canonical Gaussians warped by learned time-dependent deformation fields	(Wu et al., 2024, Guo et al., 2024)
Sparse temporal states	Explicit positions and rotations at sparse timestamps with interpolation	(Lee et al., 2024)

A compact time-function formulation represents each coordinate of the mean by a Fourier series,

$\mu_i \in \mathbb{R}^3$6

with analogous expressions for $\mu_i \in \mathbb{R}^3$7 and $\mu_i \in \mathbb{R}^3$8, while each quaternion component is linear in $\mu_i \in \mathbb{R}^3$9. This parameterization shares motion parameters across time, relaxes the strict multi-view assumption, and supports monocular as well as multi-view reconstruction (Katsumata et al., 2023).

Another compact formulation, built on STG, uses a canonical position $\Sigma_i \in \mathbb{R}^{3\times 3}$0, canonical rotation $\Sigma_i \in \mathbb{R}^{3\times 3}$1, and scale $\Sigma_i \in \mathbb{R}^{3\times 3}$2, then models position and rotation at time $\Sigma_i \in \mathbb{R}^{3\times 3}$3 by polynomial bases around a temporal center $\Sigma_i \in \mathbb{R}^{3\times 3}$4: $\Sigma_i \in \mathbb{R}^{3\times 3}$5 Temporal visibility is handled by

$\Sigma_i \in \mathbb{R}^{3\times 3}$6

which localizes a Gaussian’s active duration in time (Lee et al., 2024).

A more explicit trajectory model stores sparse states at keyframes and interpolates between them. In Fully Explicit Dynamic Gaussian Splatting, dynamic Gaussians store positions and rotations only at sparse timestamps; positions are interpolated with cubic Hermite spline (CHip), rotations with quaternion Slerp, and temporal visibility with a piecewise two-Gaussian schedule. Static Gaussians are initially allowed a global linear motion, and only the top-$\Sigma_i \in \mathbb{R}^{3\times 3}$7 percent by normalized image-space motion criterion are converted to dynamic Gaussians, with empirical $\Sigma_i \in \mathbb{R}^{3\times 3}$8 (Lee et al., 2024).

The phrase “4DGS” is therefore not semantically uniform across the literature. In some works it refers to explicit time-dependent Gaussian attributes; in others it refers to canonical Gaussians coupled to deformation fields or to higher-dimensional spatiotemporal encodings. This suggests that “dynamic 3DGS” is better understood as a family of temporal parameterizations built on a common splatting renderer than as a single canonical model.

3. Optimization objectives, temporal priors, and supervision

The dominant training signal remains photometric reconstruction. Representative formulations use mixtures of $\Sigma_i \in \mathbb{R}^{3\times 3}$9 and SSIM or D-SSIM, for example

$\Sigma_i = R_i S_i S_i^\top R_i^\top,$0

with $\Sigma_i = R_i S_i S_i^\top R_i^\top,$1 in the compact dynamic representation, or per-level photometric losses combined with SSIM in multi-scale streaming systems (Katsumata et al., 2023, Yang et al., 29 Aug 2025).

Dynamic methods then add temporal structure. Surveyed regularizers include temporal smoothness, local rigidity, local rotation similarity, and long-term local isometry. In persistent-motion formulations these terms encourage neighboring Gaussians to move coherently while preserving local geometry over time; in deformation-field methods, they stabilize otherwise underconstrained spatiotemporal warps (Bao et al., 2024).

Optical flow is a recurring supervisory signal. One compact dynamic representation obtains adjacent-frame optical flow with RAFT and adds a bidirectional flow loss derived from projected 3D Gaussian center motion. Motion-aware 3D Gaussian Splatting goes further by explicitly establishing a correspondence between 3D Gaussian movements and pixel-level flow, using an uncertainty-aware heteroscedastic loss and a transient-aware deformation auxiliary module. Its total objectives are written as

$\Sigma_i = R_i S_i S_i^\top R_i^\top,$2

for iterative and deformation-based paradigms respectively (Guo et al., 2024).

Masking is another major design axis. Scale-GS introduces bidirectional adaptive masking for streaming content: forward masking identifies dynamic anchors from temporal frame differences, while view masking ranks cameras by viewpoint novelty and directional weighting. The resulting loss at time $\Sigma_i = R_i S_i S_i^\top R_i^\top,$3 is

$\Sigma_i = R_i S_i S_i^\top R_i^\top,$4

and top-$\Sigma_i = R_i S_i S_i^\top R_i^\top,$5 views form the active training batch (Yang et al., 29 Aug 2025).

Underwater reconstruction shows a different use of masking. UW-GS does not introduce time-conditioned Gaussian attributes or motion-aware losses; instead it uses a binary motion mask (BMM) to gate photometric losses and suppress moving distractors. The final mask is

$\Sigma_i = R_i S_i S_i^\top R_i^\top,$6

combining residual thresholding, spatial diffusion, and patch-level consistency, and is then applied to a masked photometric loss (Wang et al., 2024).

This divergence is important. In Dynamic 3DGS, “handling dynamics” can mean reconstructing temporal geometry and appearance, but it can also mean robustly excluding moving distractors from optimization when the modeling goal is the static scene behind them.

4. Efficiency, compactness, and streaming scalability

A central difficulty in Dynamic 3DGS is that naïve per-timestep parameter storage or per-frame optimization scales poorly. One compact dynamic representation makes this explicit: per-timestep dynamic 3DGS has $\Sigma_i = R_i S_i S_i^\top R_i^\top,$7 parameter growth, whereas its time-basis parameterization is $\Sigma_i = R_i S_i S_i^\top R_i^\top,$8, independent of sequence length. Reported memory drops are from 6.6 GB to about 338 MB on DyNeRF, from 913 MB to 159 MB on D-NeRF, and to about 720 MB on HyperNeRF, while rendering reaches $\Sigma_i = R_i S_i S_i^\top R_i^\top,$9 FPS at $R_i$0 on a single RTX A6000 GPU (Katsumata et al., 2023).

Compact 3D Gaussian Splatting for Static and Dynamic Radiance Fields attacks storage from a different angle. It combines a learnable mask strategy, a shared grid-based neural field for view-dependent color, and residual vector quantization (RVQ) of geometric and temporal attributes. On DyNeRF, it reports PSNR $R_i$1, SSIM $R_i$2, LPIPS $R_i$3, FPS $R_i$4, and storage $R_i$5 MB, while post-processing reduces storage to $R_i$6 MB; on Technicolor, the post-processed model reaches PSNR $R_i$7, SSIM $R_i$8, LPIPS $R_i$9, FPS $S_i$0, and $S_i$1 MB per frame (Lee et al., 2024).

Speedy Deformable 3D Gaussian Splatting targets the runtime bottleneck of per-Gaussian neural inference. It introduces temporal sensitivity pruning and GroupFlow, which clusters Gaussians by trajectory similarity and predicts a single rigid transformation per group. On NeRF-DS it reports $S_i$2 faster rendering, $S_i$3 smaller model size, and $S_i$4 faster training; on D-NeRF and HyperNeRF vrig, the reported rendering-speed improvements are $S_i$5 and $S_i$6, respectively (Tu et al., 9 Jun 2025).

Scale-GS addresses scalability in streaming and long multi-view sequences by organizing Gaussians hierarchically by scale within an anchor-based structure, activating finer levels only where gradient-driven gating warrants it, and combining deformation, spawning, and redundancy filtering. Its effective per-frame cost is

$S_i$7

where $S_i$8 is the number of selected cameras, $S_i$9 is the active pixel fraction after masking, and $$G_i(x)=\exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right).$$0 is the number of active Gaussians at level $$G_i(x)=\exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right).$$1. Reported average per-frame training times are $$G_i(x)=\exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right).$$2 s on NV3D, $$G_i(x)=\exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right).$$3 s on MeetRoom, and $$G_i(x)=\exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right).$$4 s on Google Immersive, with real-time FPS of $$G_i(x)=\exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right).$$5, $$G_i(x)=\exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right).$$6, and $$G_i(x)=\exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right).$$7, respectively (Yang et al., 29 Aug 2025).

The practical implication is that Dynamic 3DGS efficiency is now addressed on at least three orthogonal fronts: compact temporal parameterizations, pruning and quantization of explicit attributes, and selective activation of Gaussians, views, or temporal regions during training.

5. Representative systems and application domains

Dynamic 3DGS has developed into a heterogeneous application layer rather than a single benchmark-driven niche. Surveys place dynamic view synthesis, free-viewpoint video, avatars, urban driving, streaming reconstruction, and robotics among the main use cases (Bao et al., 2024, Zhu et al., 2024).

In dynamic view synthesis, explicit trajectory models and canonical deformation backbones dominate. Fully Explicit Dynamic Gaussian Splatting reports $$G_i(x)=\exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right).$$8 fps on a single 2080Ti GPU and $$G_i(x)=\exp\!\left(-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right).$$9 fps on an NVIDIA 4090 in the half-resolution N3V setting, with $$\Sigma_i^{\text{screen}} = J \Sigma_i J^\top \quad\text{or}\quad \Sigma'_i = J W \Sigma_i W^\top J^\top,$$0 hours training and a typical model size of $$\Sigma_i^{\text{screen}} = J \Sigma_i J^\top \quad\text{or}\quad \Sigma'_i = J W \Sigma_i W^\top J^\top,$$1 MB in N3V experiments (Lee et al., 2024). The compact Fourier-based dynamic representation reports up to $$\Sigma_i^{\text{screen}} = J \Sigma_i J^\top \quad\text{or}\quad \Sigma'_i = J W \Sigma_i W^\top J^\top,$$2 FPS on HyperNeRF, $$\Sigma_i^{\text{screen}} = J \Sigma_i J^\top \quad\text{or}\quad \Sigma'_i = J W \Sigma_i W^\top J^\top,$$3 FPS on D-NeRF, and competitive monocular and multi-view quality (Katsumata et al., 2023).

Dynamic appearance modeling has become a distinct subproblem. Neural Texture Splatting augments each Gaussian with a local RGBA texture field generated by a shared global tri-plane and neural decoder, conditioned on position, view direction, and time. When paired with SplatFields4D on Owlii, it reports mean PSNR $$\Sigma_i^{\text{screen}} = J \Sigma_i J^\top \quad\text{or}\quad \Sigma'_i = J W \Sigma_i W^\top J^\top,$$4 versus $$\Sigma_i^{\text{screen}} = J \Sigma_i J^\top \quad\text{or}\quad \Sigma'_i = J W \Sigma_i W^\top J^\top,$$5 for SplatFields4D and $$\Sigma_i^{\text{screen}} = J \Sigma_i J^\top \quad\text{or}\quad \Sigma'_i = J W \Sigma_i W^\top J^\top,$$6 for 4DGaussians, with corresponding SSIM $$\Sigma_i^{\text{screen}} = J \Sigma_i J^\top \quad\text{or}\quad \Sigma'_i = J W \Sigma_i W^\top J^\top,$$7 and LPIPS $$\Sigma_i^{\text{screen}} = J \Sigma_i J^\top \quad\text{or}\quad \Sigma'_i = J W \Sigma_i W^\top J^\top,$$8; the method reduces floaters and boundary artifacts while recovering fine details such as faces and hands (Wang et al., 24 Nov 2025).

Some domain-specific variants redefine what “dynamic” means. UW-GS is designed for underwater scenes where moving fish and particulates act as distractors and where absorption and scattering alter the appearance model. It introduces a distance-dependent color transformation,

$$\Sigma_i^{\text{screen}} = J \Sigma_i J^\top \quad\text{or}\quad \Sigma'_i = J W \Sigma_i W^\top J^\top,$$9

a physics-based density control strategy, and the binary motion mask described above. The result is a distractor-aware underwater 3DGS rather than a general motion-reconstruction framework (Wang et al., 2024).

Robotics-oriented surveys emphasize that the explicit Gaussian representation is particularly compatible with online scene understanding, navigation, and manipulation because it supports object-centric updates, free-space reasoning, and real-time rendering. Dynamic urban methods such as PVG, VDG, DrivingGaussian, Street Gaussians, and HUGS are cited as examples of dynamic 3DGS in driving scenarios, while deformation-field methods such as 4DGaussians, 4DGF, SC-GS, Gaussian-Flow, SWAGS, and 3DGStream are highlighted for broader non-rigid or streaming settings (Zhu et al., 2024).

6. Limitations, misconceptions, and open directions

The literature is consistent on several failure modes. Extremely large non-rigid deformations, topology changes, fast motion, heavy occlusions, and abrupt viewpoint changes remain difficult. In Scale-GS, missed activations can underfit fine motion, while over-activation increases cost; occlusion-heavy scenes can confuse forward masking and cause over-masking of useful areas or missed spawns behind newly revealed surfaces (Yang et al., 29 Aug 2025). In compact time-function models, appearing or disappearing structures and fluids are difficult because Gaussians are assumed to exist across all times, and inaccurate camera poses degrade reconstruction quality (Katsumata et al., 2023).

A second persistent issue is ambiguity between scene motion and camera motion, especially in monocular settings. Motion-aware 3DGS addresses this with uncertainty-aware flow supervision and a transient-aware deformation auxiliary, but the paper still identifies severe occlusions, topological changes, strong motion blur, and long-range motion as challenging cases (Guo et al., 2024). Robotics-oriented surveys likewise identify robust tracking, lifelong mapping, and large-scale relocalization as unresolved problems for dynamic Gaussian maps (Zhu et al., 2024).

There is also a conceptual misconception in equating “4DGS” with a single architecture. Survey evidence shows that the label covers persistent per-Gaussian motion, canonical-plus-deformation models, explicit sparse temporal keyframes, space-time feature splatting, and temporal hierarchies for long sequences (Kerbl, 30 Oct 2025). The common thread is not a unique 4D primitive but a shared reliance on explicit Gaussian splats, differentiable rasterization, and temporally structured optimization.

Open directions identified across the surveys include scaling to long videos and large scenes, better handling of complex deformations and topology changes, joint lighting and material modeling over time, robust monocular and sparse-view reconstruction, stronger editability and control, and tighter integration of compression, pruning, and hardware-aware rendering (Bao et al., 2024). A plausible implication is that future Dynamic 3DGS systems will be increasingly hybrid: explicit splats for rendering efficiency, compact temporal bases or grouped motion for scalability, and learned spatial-temporal fields only where continuous deformation modeling is indispensable.