Spacetime Gaussians in Graphics & Wave Physics

• Spacetime Gaussians (STG) are Gaussian-structured representations applied across domains such as dynamic scene rendering, dispersive wave propagation, and quantum field theory.
• They enable time-aware modeling by incorporating parameters like temporal opacity, motion, and rotation, optimizing dynamic and transient content in visual synthesis.
• STG methods leverage structured evolution laws, including state estimation, SU(2) modal symmetries, and Gaussian damping, to improve performance across diverse applications.

Searching arXiv for papers on Spacetime Gaussians and related STG usages. Spacetime Gaussians (STG) denotes two distinct research usages that share Gaussian structure but differ substantially in domain and formalism. In dynamic scene reconstruction and rendering, STG refers to a dynamic-scene extension of 3D Gaussian Splatting in which each Gaussian becomes a spacetime primitive with time-dependent opacity, motion, and rotation, enabling a single sequence-aware representation for static, dynamic, and transient content (Li et al., 2023). In wave physics, the same abbreviation denotes spatiotemporal Gaussian modes: Gaussian wave packets in the joint space–time plane governed by a paraxial, quasi-monochromatic propagation equation in isotropic dispersive media, where the mode family exhibits an $SU(2)$ symmetry analogous to that of spatial Hermite–Gaussian and Laguerre–Gaussian beams (Tang et al., 30 Aug 2025). A further, more speculative usage appears in quantum field theory, where Gaussian suppression arises from averaging over quantum fluctuations of spacetime points, producing a Gaussian-damped field theory (Cahill, 2024). Because these literatures are non-equivalent, precise interpretation of “STG” depends on disciplinary context.

1. Terminological scope and domain-specific meanings

In computer vision and graphics, Spacetime Gaussians were introduced as a representation for real-time dynamic view synthesis. The core idea is to extend a static 3D Gaussian primitive into a 4D spacetime primitive whose spatial position, rotation, opacity, and appearance are conditioned on time, while retaining differentiable splatting as the rendering mechanism (Li et al., 2023). In this usage, STG is closely tied to 3D Gaussian Splatting, dynamic novel-view synthesis, rasterization-based rendering, and compact scene representations.

In optical wave theory, spatiotemporal Gaussian modes are not generic Gaussian pulses, but a complete orthogonal family of Gaussian-like solutions of a spatiotemporal propagation equation in the $(x,\xi)$ plane, with $\xi=z-v_g t$ the retarded time coordinate. The formalism treats dimensionless spatiotemporal coordinates as those of an effective two-dimensional harmonic oscillator, from which spatiotemporal Hermite-Gaussian and spatiotemporal Laguerre-Gaussian modes arise (Tang et al., 30 Aug 2025). This usage is algebraic and modal rather than scene-representational.

A third use concerns quantum spacetime fluctuations. There, Gaussian behavior does not define a Gaussian scene primitive or optical mode family, but arises when the plane-wave factor $e^{ik\cdot(x+q(x))}$ is averaged over normally distributed fluctuations $q(x)$, yielding Gaussian damping in momentum space (Cahill, 2024). This suggests that “spacetime Gaussian” can also denote Gaussian structure induced by uncertainty in spacetime coordinates rather than a primitive or mode basis.

A common misconception is to assume a single unified STG framework across these domains. The available literature does not support that interpretation. Instead, the shared term reflects a common Gaussian mathematical motif applied to different objects: scene primitives in rendering (Li et al., 2023), mode families in dispersive wave propagation (Tang et al., 30 Aug 2025), and damping kernels in quantum-field-theoretic regularization (Cahill, 2024).

2. STG in dynamic scene rendering and view synthesis

The rendering-oriented formulation defines a standard 3D Gaussian by position $\mu_i$, covariance $\Sigma_i$, opacity $\sigma_i$, and spherical-harmonic coefficients in the baseline 3DGS setting, with spatial opacity

$$\alpha_i = \sigma_i \exp\left( -\frac{1}{2} (\mathbf{x} - {\mu_i})^T \Sigma_{i}^{-1} (\mathbf{x} - \mu_i) \right),$$

and covariance decomposition

$\Sigma_i = R_i S_i S_i^T R_i^T.$

STG lifts this formulation into spacetime by making opacity, position, and rotation time dependent: $(x,\xi)$0 The temporal opacity is modeled as

$(x,\xi)$1

the spatial center follows a polynomial trajectory

$(x,\xi)$2

and the quaternion rotation is time-dependent through

$(x,\xi)$3

In the reported experiments, the method uses $(x,\xi)$4 and $(x,\xi)$5 (Li et al., 2023).

This parameterization allows a single learned Gaussian set to represent three categories of content. Static content corresponds to broad temporal opacity and near-zero motion. Dynamic content corresponds to nontrivial trajectory and rotation. Transient content corresponds to localized temporal opacity, allowing a Gaussian to become active only during a limited temporal interval (Li et al., 2023). This replaces a per-frame modeling strategy with a sequence-aware representation.

Rendering remains splatting-based. The projected 2D Gaussian uses

$(x,\xi)$6

$(x,\xi)$7

followed by alpha compositing

$(x,\xi)$8

A distinctive modification is “splatted feature rendering,” which replaces spherical harmonics with a feature vector $(x,\xi)$9, split into base, directional, and temporal components, and decoded by a small MLP: $\xi=z-v_g t$0 The paper states that 3-degree spherical harmonics require 48 parameters per Gaussian versus 9 for the feature representation (Li et al., 2023).

The framework further augments standard 3DGS density control with guided sampling based on training error and coarse depth. Error-guided patch selection identifies regions with substantial reconstruction error; coarse depth from the Gaussian centers then restricts the sampling interval along selected rays. New Gaussians are sampled within that interval and later pruned if unnecessary (Li et al., 2023). This suggests a hybrid density-control strategy combining local splitting/cloning around existing Gaussians with explicit insertion into under-covered regions.

On the Neural 3D Video Dataset, the full model reports 140 FPS at $\xi=z-v_g t$1, 32.05 PSNR, 0.026 DSSIM$\xi=z-v_g t$2, 0.014 DSSIM$\xi=z-v_g t$3, 0.044 LPIPS, and 200 MB total model size for 300 frames. On the Google Immersive Dataset it reports 29.2 PSNR, 0.042 DSSIM$\xi=z-v_g t$4, 0.081 LPIPS, 99 FPS, and model size per frame about 1.2 MB. On the Technicolor Dataset it reports 33.6 PSNR, 0.040 DSSIM$\xi=z-v_g t$5, 0.019 DSSIM$\xi=z-v_g t$6, 0.084 LPIPS, 86.7 FPS, and 1.1 MB per frame (Li et al., 2023). The lite version drops the MLP and is shown to render 8K 6-DoF video at 66 FPS on an Nvidia RTX 4090 GPU; in supplementary Neural 3D Video comparisons, the lite model reaches 310 FPS with 103 MB total size (Li et al., 2023).

3. State-space and Wasserstein refinements of dynamic Gaussian evolution

A subsequent line of work reframes dynamic Gaussian evolution as a state estimation problem. “Gaussians on their Way: Wasserstein-Constrained 4D Gaussian Splatting with State-Space Modeling” treats each Gaussian as a dynamic state whose canonical parameters are initialized from an SFM point cloud, then mapped by a neural deformation field to an observed Gaussian at time $\xi=z-v_g t$7: $\xi=z-v_g t$8 with

$\xi=z-v_g t$9

The method then introduces a time-independent linear dynamics predictor and a Kalman-like state update (Deng et al., 2024).

The Euclidean prediction is written as

$e^{ik\cdot(x+q(x))}$0

with $e^{ik\cdot(x+q(x))}$1. The State Consistency Filter merges the observed Gaussian $e^{ik\cdot(x+q(x))}$2 and the predicted Gaussian $e^{ik\cdot(x+q(x))}$3, producing the updated state $e^{ik\cdot(x+q(x))}$4 via

$e^{ik\cdot(x+q(x))}$5

The paper explicitly characterizes this as a prediction-update filtering formulation akin to a Kalman filter, but with the Gaussian distribution itself as the state (Deng et al., 2024).

The same work replaces Euclidean regularization with a 2-Wasserstein metric between Gaussians,

$e^{ik\cdot(x+q(x))}$6

together with a symmetric formulation for stable computation and a covariance decomposition based on

$e^{ik\cdot(x+q(x))}$7

Two losses are introduced: the State-Observation Alignment loss

$e^{ik\cdot(x+q(x))}$8

and Wasserstein regularization

$e^{ik\cdot(x+q(x))}$9

The total objective is

$q(x)$0

This formulation is explicitly motivated by the claim that Euclidean penalties treat mean and covariance separately instead of as a distribution (Deng et al., 2024).

The method also uses logarithmic and exponential maps on the manifold of Gaussian distributions,

$q(x)$1

with translation retained as Euclidean mean evolution and covariance treated geometrically on $q(x)$2 (Deng et al., 2024). A plausible implication is that this line of work reinterprets STG-style dynamic Gaussians less as temporally indexed primitives and more as recursively estimated probability distributions evolving on a structured geometric state space.

Empirically, the method evaluates on D-NeRF and the Plenoptic Video Dataset. On D-NeRF it reports 34.45 PSNR, 0.970 SSIM, 0.026 LPIPS, and 45.5 FPS. On Plenoptic Video it reports 31.62 PSNR, 0.940 SSIM, 0.140 LPIPS, and 37 FPS (Deng et al., 2024). In the State Consistency Filter ablation, optical flow AEPE on Plenoptic changes from 1.45 without the filter to 1.02 with the filter, a 29.7% reduction (Deng et al., 2024).

4. STG for animatable human avatars

In avatar reconstruction, STG is used as a non-rigid refinement layer coupled to a skeleton-driven deformation model rather than as a standalone dynamic scene representation. “STG-Avatar: Animatable Human Avatars via Spacetime Gaussian” introduces a rigid-nonrigid coupled deformation framework that integrates STG with linear blend skinning (LBS) (Jiang et al., 25 Oct 2025).

The method is organized as a three-stage monocular pipeline: SMPL-guided spatiotemporal Gaussian initialization, rigid-nonrigid co-optimization, and dynamic-aware neural rendering. Gaussian centers are sampled on the SMPL surface in canonical pose; LBS then applies skeletal deformation, and STG refines the deformed Gaussians through time-dependent offsets and rotations (Jiang et al., 25 Oct 2025).

Each Gaussian $q(x)$3 has time-dependent opacity, mean, and covariance: $q(x)$4 with

$q(x)$5

The quaternion-valued rotation is modeled as

$q(x)$6

and the center is given by

$q(x)$7

Here $q(x)$8 is the LBS-deformed anchor. The reported choice is $q(x)$9 for translation and $\mu_i$0 for rotation (Jiang et al., 25 Oct 2025).

The LBS stage uses

$\mu_i$1

where $\mu_i$2 is the number of bones, $\mu_i$3 is the skinning weight, and $\mu_i$4 is the rigid transformation of bone $\mu_i$5. STG then applies residual, time-dependent refinement about this posed anchor (Jiang et al., 25 Oct 2025). This suggests a division of labor: LBS encodes coarse articulated kinematics, while STG targets local, temporally varying deviations such as cloth wrinkles, hair motion, and rapid limb-dependent detail changes.

Training uses the objective

$\mu_i$6

where $\mu_i$7 combines photometric losses such as $\mu_i$8 and SSIM, $\mu_i$9 enforces optical flow consistency, $\Sigma_i$0 reduces temporal flickering, and $\Sigma_i$1 regularizes sparsity and motion smoothness (Jiang et al., 25 Oct 2025).

A central contribution is optical flow-guided adaptive densification. Dynamic regions are identified by a rendering-error criterion,

$\Sigma_i$2

and by accumulated motion magnitude,

$\Sigma_i$3

New Gaussians are sampled anisotropically along the optical flow direction: $\Sigma_i$4 Temporal consistency checking uses

$\Sigma_i$5

and pruning is flow-aware through

$\Sigma_i$6

The color decoder is

$\Sigma_i$7

with positional, motion, pose, and view-direction encodings (Jiang et al., 25 Oct 2025).

On ZJU-MoCap, the reported average performance is PSNR: 31.6 dB, SSIM: 0.954, and LPIPS: 0.038. Relative to 3DGS-Avatar, the method improves dynamic-region PSNR by about 1.5 dB and reduces LPIPS on clothing wrinkles by 23%. The ablation without STG, described as LBS-only, reports 25.59 PSNR, 0.9198 SSIM, and 0.056 LPIPS. The system trains in about 25 minutes and renders at 60 FPS on an RTX 4090 (Jiang et al., 25 Oct 2025). Within the evidence provided, these numbers position STG as the principal non-rigid detail carrier rather than merely a temporal bookkeeping device.

5. Spatiotemporal Gaussian modes and $\Sigma_i$8 symmetry

In dispersive wave propagation, spatiotemporal Gaussian modes arise from the paraxial, quasi-monochromatic propagation equation in an isotropic dispersive medium,

$\Sigma_i$9

with $\sigma_i$0 (Tang et al., 30 Aug 2025). The system is treated as Schrödinger-like, with dimensionless coordinates $\sigma_i$1 forming an effective two-dimensional harmonic oscillator.

The corresponding Gaussian basis states are spatiotemporal Hermite-Gaussian modes,

$\sigma_i$2

with separate widths, curvatures, and mode-dependent Gouy phase (Tang et al., 30 Aug 2025). The creation and annihilation operators are introduced as

$\sigma_i$3

$\sigma_i$4

satisfying

$\sigma_i$5

The oscillator Hamiltonian is

$\sigma_i$6

and the conserved quantities

$\sigma_i$7

$\sigma_i$8

$\sigma_i$9

satisfy

$$\alpha_i = \sigma_i \exp\left( -\frac{1}{2} (\mathbf{x} - {\mu_i})^T \Sigma_{i}^{-1} (\mathbf{x} - \mu_i) \right),$$0

This establishes the $$\alpha_i = \sigma_i \exp\left( -\frac{1}{2} (\mathbf{x} - {\mu_i})^T \Sigma_{i}^{-1} (\mathbf{x} - \mu_i) \right),$$1 algebra, and the fixed-order subspace with

$$\alpha_i = \sigma_i \exp\left( -\frac{1}{2} (\mathbf{x} - {\mu_i})^T \Sigma_{i}^{-1} (\mathbf{x} - \mu_i) \right),$$2

has dimension $$\alpha_i = \sigma_i \exp\left( -\frac{1}{2} (\mathbf{x} - {\mu_i})^T \Sigma_{i}^{-1} (\mathbf{x} - \mu_i) \right),$$3, forming an irreducible representation of $$\alpha_i = \sigma_i \exp\left( -\frac{1}{2} (\mathbf{x} - {\mu_i})^T \Sigma_{i}^{-1} (\mathbf{x} - \mu_i) \right),$$4 (Tang et al., 30 Aug 2025).

The spatiotemporal Laguerre-Gaussian basis is written as

$$\alpha_i = \sigma_i \exp\left( -\frac{1}{2} (\mathbf{x} - {\mu_i})^T \Sigma_{i}^{-1} (\mathbf{x} - \mu_i) \right),$$5

with

$$\alpha_i = \sigma_i \exp\left( -\frac{1}{2} (\mathbf{x} - {\mu_i})^T \Sigma_{i}^{-1} (\mathbf{x} - \mu_i) \right),$$6

Using $$\alpha_i = \sigma_i \exp\left( -\frac{1}{2} (\mathbf{x} - {\mu_i})^T \Sigma_{i}^{-1} (\mathbf{x} - \mu_i) \right),$$7 representation theory, the STLG basis is expressed as a linear combination of the STHG basis via the Wigner $$\alpha_i = \sigma_i \exp\left( -\frac{1}{2} (\mathbf{x} - {\mu_i})^T \Sigma_{i}^{-1} (\mathbf{x} - \mu_i) \right),$$8-matrix: $$\alpha_i = \sigma_i \exp\left( -\frac{1}{2} (\mathbf{x} - {\mu_i})^T \Sigma_{i}^{-1} (\mathbf{x} - \mu_i) \right),$$9 The far-field multi-petal intensity patterns of spatiotemporal Laguerre-Gaussian modes are explained by this $\Sigma_i = R_i S_i S_i^T R_i^T.$0 structure: propagation rotates the state within the degenerate order-$\Sigma_i = R_i S_i S_i^T R_i^T.$1 subspace, mixing HG components whose interference generates petal-like patterns (Tang et al., 30 Aug 2025).

The propagation law is written as

$\Sigma_i = R_i S_i S_i^T R_i^T.$2

which is simplified to

$\Sigma_i = R_i S_i S_i^T R_i^T.$3

up to phase conventions (Tang et al., 30 Aug 2025). The normalized phase decomposition is

$\Sigma_i = R_i S_i S_i^T R_i^T.$4

with

$\Sigma_i = R_i S_i S_i^T R_i^T.$5

$\Sigma_i = R_i S_i S_i^T R_i^T.$6

and

$\Sigma_i = R_i S_i S_i^T R_i^T.$7

where $\Sigma_i = R_i S_i S_i^T R_i^T.$8. The intermodal Gouy phase $\Sigma_i = R_i S_i S_i^T R_i^T.$9 serves as the rotation angle on the spatiotemporal modal Poincaré sphere (Tang et al., 30 Aug 2025).

The paper identifies three dispersion regimes. For zero dispersion $(x,\xi)$00, $(x,\xi)$01 evolves monotonically from $(x,\xi)$02 to $(x,\xi)$03 to $(x,\xi)$04. For normal dispersion $(x,\xi)$05, $(x,\xi)$06 varies from $(x,\xi)$07 to $(x,\xi)$08 to $(x,\xi)$09. For anomalous dispersion $(x,\xi)$10, $(x,\xi)$11 becomes non-monotonic, producing distortion and revival of the intensity pattern (Tang et al., 30 Aug 2025). The special case

$(x,\xi)$12

gives

$(x,\xi)$13

for which the mode intensity is invariant during propagation. When this condition is not met, the paper interprets the recurrence as a phase-locked mechanism analogous to the Talbot effect (Tang et al., 30 Aug 2025).

6. Quantum-spacetime Gaussian damping and finite field theory

A conceptually distinct use of Gaussian spacetime structure appears in the proposal that spacetime itself should be treated as a quantum field. In this formulation, the spacetime point operator is decomposed as

$(x,\xi)$14

with flat-space mean

$(x,\xi)$15

The argument begins from Einstein’s equation and the embedding-space expression for the metric,

$(x,\xi)$16

from which the paper infers that if the metric is quantum then the spacetime points $(x,\xi)$17 must also be quantum variables (Cahill, 2024).

The fluctuations $(x,\xi)$18 are assumed normally distributed. For the full four-component fluctuation,

$(x,\xi)$19

Averaging the Fourier factor over these fluctuations yields

$(x,\xi)$20

Thus the fluctuating spacetime coordinates turn plane waves into Gaussian-damped plane waves (Cahill, 2024).

The corresponding smeared scalar field is

$(x,\xi)$21

The Feynman propagator becomes

$(x,\xi)$22

The paper argues that the Gaussian factor suppresses the high-momentum region, so loop integrals become finite (Cahill, 2024).

The equal-time commutator is modified to

$(x,\xi)$23

which evaluates to

$(x,\xi)$24

This replaces the usual delta singularity by a Gaussian in position space (Cahill, 2024).

The same damping enters scattering amplitudes, for example in the lowest-order $(x,\xi)$25 amplitude of $(x,\xi)$26 theory,

$(x,\xi)$27

Vacuum energy also becomes finite: $(x,\xi)$28 with the massless boson case

$(x,\xi)$29

For the total vacuum energy density, the paper writes

$(x,\xi)$30

However, it concludes that with the known particle spectrum the vacuum energy is negative and too large in magnitude to explain dark energy unless additional bosonic modes exist (Cahill, 2024). This remains a speculative interpretation rather than a consensus STG framework.

7. Conceptual correspondences, distinctions, and open questions

Across these literatures, Gaussian structure serves as a compact, analytically tractable means of encoding variation over spacetime, but the ontological status of the Gaussian differs sharply. In dynamic rendering, an STG is a learned primitive with time-conditioned geometry and appearance (Li et al., 2023). In avatar modeling, it is a non-rigid residual carrier anchored by LBS (Jiang et al., 25 Oct 2025). In dispersive optics, it is a basis state in a harmonic-oscillator representation with $(x,\xi)$31 symmetry (Tang et al., 30 Aug 2025). In quantum-field-theoretic regularization, Gaussianity appears as the averaged consequence of quantum spacetime fluctuations (Cahill, 2024).

The following table summarizes the domain-specific meanings.

Domain	Meaning of STG	Defining mechanism
Dynamic view synthesis	Spacetime Gaussian primitive	Time-dependent opacity, motion, rotation, and feature splatting (Li et al., 2023)
Dynamic Gaussian state modeling	Dynamic Gaussian state in a filtered system	State Consistency Filter and Wasserstein-constrained evolution (Deng et al., 2024)
Human avatar reconstruction	Non-rigid refinement layer after LBS	Rigid-nonrigid coupled deformation with flow-guided densification (Jiang et al., 25 Oct 2025)
Dispersive wave propagation	Spatiotemporal Gaussian mode family	2D harmonic-oscillator structure and $(x,\xi)$32 symmetry (Tang et al., 30 Aug 2025)
Quantum field theory	Gaussian-damped field modes from spacetime uncertainty	Averaging over normally distributed coordinate fluctuations (Cahill, 2024)

A recurring research theme is the replacement of framewise or pointwise representations by structured evolution laws. In graphics, this appears as polynomial trajectories, temporal opacity, filtering, and geometry-aware regularization (Li et al., 2023, Deng et al., 2024, Jiang et al., 25 Oct 2025). In wave theory, it appears as unitary propagation in a fixed-order $(x,\xi)$33 multiplet with the intermodal Gouy phase as rotation angle (Tang et al., 30 Aug 2025). In the quantum-spacetime proposal, it appears as Gaussian suppression induced directly by uncertainty in the coordinates (Cahill, 2024). This suggests a broad methodological affinity: Gaussian spacetime formulations often trade local unconstrained variation for low-dimensional parametric or algebraic structure.

At the same time, these parallels should not be overstated. The rendering literature uses explicit scene primitives and differentiable rasterization; the optical literature is grounded in symmetry, mode algebra, and dispersive propagation; the quantum-field-theoretic proposal addresses ultraviolet finiteness through nonlocal smearing. No paper in the supplied set derives one usage from another. A plausible implication is that “Spacetime Gaussians” functions less as a unified theory than as a cross-disciplinary label for Gaussian parameterizations indexed by space and time.

Within dynamic scene modeling specifically, current developments indicate three active directions. One is greater expressivity within the primitive itself, as in temporal opacity, polynomial motion, and feature splatting (Li et al., 2023). A second is structured temporal estimation via filtering and Wasserstein geometry (Deng et al., 2024). A third is task-specific hybridization, exemplified by coupling STG to skeletal priors for animatable avatars (Jiang et al., 25 Oct 2025). In the optical domain, the main advance represented here is the explicit identification of $(x,\xi)$34 symmetry, the spatiotemporal modal Poincaré sphere, and closed-form propagation for arbitrary radial and angular indices (Tang et al., 30 Aug 2025). These trajectories indicate that STG, while terminologically heterogeneous, has become a significant organizing concept for several technically mature but distinct areas of current research.