4D Gaussian Reconstruction Techniques

• 4D Gaussian Reconstruction is a technique that models dynamic scenes using time-dependent Gaussian primitives, enabling queries at arbitrary timestamps and viewpoints.
• It employs diverse temporal parameterizations—such as explicit deformations, feed-forward sequence models, and unified spatiotemporal covariance—to capture motion and maintain consistency.
• The approach demonstrates efficiency improvements through real-time rendering and domain-specific adaptations, addressing challenges like aliasing and static–dynamic decoupling.

Searching arXiv for papers on 4D Gaussian Reconstruction and related dynamic Gaussian splatting methods. I’m checking for relevant arXiv papers spanning foundational dynamic Gaussian splatting, feed-forward 4D reconstruction, anti-aliasing, sparse-frame reconstruction, and domain-specific variants. 4D Gaussian Reconstruction denotes a family of representations and optimization pipelines that model a scene or volume as Gaussian primitives evolving over time, so that geometry, appearance, and motion can be queried at arbitrary timestamps and rendered or projected from novel viewpoints. In the visual reconstruction literature, the dominant formulation extends 3D Gaussian Splatting to dynamic scenes by making Gaussian attributes time-dependent; in tomographic and photoacoustic settings, the same principle is coupled to physics-based forward models such as Beer–Lambert attenuation or acoustic propagation rather than RGB rasterization (Lin et al., 2023, Fu et al., 7 Jan 2025).

1. Definition and conceptual scope

At its most general, 4D Gaussian reconstruction seeks a time-varying scene representation that supports rendering at arbitrary camera viewpoints and timestamps $t$ across a sequence (Lin et al., 2023). In monocular and multi-view video, this usually means recovering dynamic geometry, radiance, and motion from RGB observations; in cone-beam CT and photoacoustic imaging, it means recovering a temporally continuous attenuation or pressure field from sparse projections or sensor traces (Yu et al., 27 Mar 2025, Li et al., 2024).

Across the surveyed papers, the basic primitive remains a Gaussian with center, covariance, opacity or density, and appearance or radiodensity attributes. What changes is the interpretation of time. In deformation-based systems, “4D” arises from time-indexed 3D Gaussians whose position, rotation, scale, or radiance evolve with $t$ rather than from a dense spatiotemporal covariance (Chen et al., 23 Nov 2025). In unified spatiotemporal formulations, time enters the primitive itself, as in a 4D Gaussian over $(\mathbf{x}, t)$ with conditional spatial slices at a requested timestamp (Wu et al., 27 Oct 2025). In feed-forward sequence models, 4D reconstruction is implemented as a time-indexed sequence of per-frame 3D Gaussian sets, with temporal consistency learned by attention or interpolation modules rather than by an explicit deformation field (Ren et al., 2024).

A recurrent distinction is between methods that treat all Gaussians as deformable and methods that explicitly separate static and dynamic content. UrbanGS keeps static Gaussians time-invariant and assigns time-conditional residuals only to potentially dynamic Gaussians (Li et al., 2024). SDD-4DGS introduces a probabilistic dynamic perception coefficient $w$ that gates deformation magnitude per Gaussian, while DrivingRecon and ReconDrive perform explicit static–dynamic decoupling for autonomous driving scenes (Sun et al., 12 Mar 2025, Lu et al., 2024, Yu et al., 8 Mar 2026). This suggests that “4D Gaussian reconstruction” is best understood as a broad design space rather than a single canonical model.

2. Gaussian primitives, projection, and rendering

The common geometric core is an anisotropic Gaussian in $\mathbb{R}^3$, typically written as

$$G(\mathbf{x}) = \exp\!\left(-\tfrac{1}{2}(\mathbf{x}-\boldsymbol{\mu})^\top \boldsymbol{\Sigma}^{-1}(\mathbf{x}-\boldsymbol{\mu})\right),$$

with covariance parameterized by a rotation and diagonal scale (Lin et al., 2023). A standard decomposition is

$$\boldsymbol{\Sigma}=R(\mathbf{q})\Lambda(\mathbf{s})\Lambda(\mathbf{s})^\top R(\mathbf{q})^\top,$$

or equivalently $R S S^\top R^\top$, where $R$ is induced by a quaternion and $S$ is diagonal (Lin et al., 2023, Chen et al., 23 Nov 2025).

For RGB rendering, the primitive is projected into screen space by the Jacobian of the camera projection. Alias-free 4D Gaussian Splatting states this as

$t$0

with a 2D Gaussian obtained by omitting the third row and column in ray space (Chen et al., 23 Nov 2025). Rendering then follows front-to-back alpha compositing: $t$1 or, in the notation of Gaussian-Flow,

$t$2

with view-dependent radiance usually represented by spherical harmonics (Lin et al., 2023, Chen et al., 23 Nov 2025).

Not all 4D Gaussian methods rely on image-plane splatting. Vidu4D uses dynamic Gaussian surfels, that is, surface-aligned Gaussian surfels warped over time and rendered by differentiable volume rendering over surfel-ray intersections rather than by standard projected-covariance splats (Wang et al., 2024). In medical imaging, the forward operator changes entirely. X$t$3-Gaussian models the attenuation field as a sum of anisotropic Gaussians and renders X-ray projections by closed-form Gaussian line integrals under Beer–Lambert attenuation (Yu et al., 27 Mar 2025). Spatiotemporal Gaussian Optimization for 4D-CBCT similarly accumulates rectified Gaussian contributions to optical depth, while 4D SlingBAG uses a differentiable photoacoustic forward operator over Gaussian balls (Fu et al., 7 Jan 2025, Li et al., 2024).

A common misconception is that all 4D Gaussian methods use a full $t$4 spatiotemporal covariance. Deformation-based methods explicitly do not: motion is modeled through time-varying position, rotation, or scale changes of 3D Gaussians (Chen et al., 23 Nov 2025). By contrast, EndoWave and OriGS introduce genuinely spatiotemporal parameterizations in which time is embedded into the primitive state, either through a 4D Gaussian block covariance or through a Hyper-Gaussian conditioned on time and orientation (Wu et al., 27 Oct 2025, Wu et al., 27 Sep 2025).

3. Temporal parameterizations and motion models

Several temporal parameterizations recur across the literature.

The most direct formulation is explicit per-attribute deformation. Gaussian-Flow models dynamic attributes $t$5 as

$t$6

where $t$7 is the sum of a polynomial term and a Fourier series term,

$t$8

This Dual-Domain Deformation Model is accompanied by per-particle timestamp scaling $t$9 to stabilize violent short-segment motion (Lin et al., 2023). The same paper emphasizes that polynomials capture smooth, long-term trends and Fourier series capture periodic or high-frequency motion.

A second family keeps a canonical Gaussian field and learns a deformation network. Sparse4DGS uses a canonical field plus an MLP that predicts $(\mathbf{x}, t)$0, thereby deforming canonical Gaussians into their state at time $(\mathbf{x}, t)$1 (Shi et al., 10 Nov 2025). Alias-free 4D Gaussian Splatting adopts the same deformation-based formulation and couples it to time-varying scale changes $(\mathbf{x}, t)$2 for anti-aliasing control (Chen et al., 23 Nov 2025). UrbanGS extends DeformGS-style time conditioning by concatenating canonical position, normalized timestamp, and a per-Gaussian learnable time embedding $(\mathbf{x}, t)$3, then predicting residuals for position, rotation, scale, and opacity only for potentially dynamic Gaussians (Li et al., 2024).

A third family introduces stronger structure into the motion field. OriGS defines a Hyper-Gaussian state

$(\mathbf{x}, t)$4

where $(\mathbf{x}, t)$5 is a local orientation supplied by a Global Orientation Field. Motion is inferred by Gaussian conditioning on $(\mathbf{x}, t)$6, yielding deterministic predictions for $(\mathbf{x}, t)$7 (Wu et al., 27 Sep 2025). 4D Scaffold Gaussian Splatting replaces explicit per-Gaussian 4D parameters with sparse 4D anchors and “neural 4D Gaussians,” each using a neural velocity

$(\mathbf{x}, t)$8

and a generalized-Gaussian temporal opacity

$(\mathbf{x}, t)$9

thereby separating spatial covariance from temporal support (Cho et al., 2024).

Feed-forward sequence models form another category. L4GM outputs a fresh set of Gaussian ellipsoids $w$0 for every timestep and enforces time consistency through temporal self-attention, not through tracked correspondences or a canonical deformation field (Ren et al., 2024). DrivingRecon predicts time-aware Gaussians directly from surround-view videos, advancing dynamic Gaussians one step with world-coordinate optical flow and using static/dynamic decoupling to supervise geometry and motion (Lu et al., 2024). ReconDrive similarly adopts feed-forward 4D Gaussian generation for autonomous driving, but uses Hybrid Gaussian Prediction Heads and a segment-wise Static-Dynamic 4D Composition with explicit linear velocity modeling (Yu et al., 8 Mar 2026).

Some formulations are domain-specific. Deblur4DGS transforms continuous dynamic representations estimation within an exposure time into exposure time estimation, discretizes each blurry frame into $w$1 latent instants, and interpolates dynamic Gaussians between neighboring integer-time states (Wu et al., 2024). EndoWave directly optimizes unified 4D Gaussians over $w$2, using the conditional mean

$w$3

and corresponding conditional covariance for endoscopic scene reconstruction (Wu et al., 27 Oct 2025).

4. Supervision, optimization, and efficiency

Most visual methods are trained primarily by photometric reconstruction, often augmented with structural or perceptual terms. Gaussian-Flow follows 3DGS-style photometric supervision and supplements it with a time-smoothness loss

$w$4

and a KNN rigid regularization

$w$5

activated once the point set is fixed (Lin et al., 2023). Sparse4DGS adds a Texture Intensity field, a Pearson-correlation texture loss $w$6, and Texture-Aware Deformation Regularization $w$7, while Texture-Aware Canonical Optimization injects texture-conditioned stochastic noise into canonical Gaussian updates (Shi et al., 10 Nov 2025).

Several recent methods add explicit static–dynamic supervision. SDD-4DGS introduces a Bernoulli-modeled dynamic perception coefficient $w$8 and minimizes the binary entropy

$w$9

with a progressive schedule $\mathbb{R}^3$0, while an automatic supervision loss separately constrains dynamic and static reconstructions (Sun et al., 12 Mar 2025). UrbanGS uses semantic consistency, KNN-based ground-surface regularization, depth supervision from LiDAR, and sky sparsity, keeping static Gaussians time-invariant by construction (Li et al., 2024). ReconDrive combines a rendering loss, a masked projection loss, and norm regularization, with explicit loss weights $\mathbb{R}^3$1, $\mathbb{R}^3$2, $\mathbb{R}^3$3, $\mathbb{R}^3$4, and $\mathbb{R}^3$5 (Yu et al., 8 Mar 2026).

Efficiency is a central motivation for the entire area. Gaussian-Flow reports a training time reduction of approximately $\mathbb{R}^3$6 relative to per-frame 3DGS and real-time rendering of about $\mathbb{R}^3$7 FPS on an RTX 4090 at the tested resolutions (Lin et al., 2023). L4GM moves further toward amortized inference, describing a single feed-forward pass that takes only a second and reporting a forward pass of about $\mathbb{R}^3$8 s for $\mathbb{R}^3$9 on an RTX 4080 Super, with a separate interpolation model for higher frame rate (Ren et al., 2024). ReconDrive processes a $$G(\mathbf{x}) = \exp\!\left(-\tfrac{1}{2}(\mathbf{x}-\boldsymbol{\mu})^\top \boldsymbol{\Sigma}^{-1}(\mathbf{x}-\boldsymbol{\mu})\right),$$0 s scene in $$G(\mathbf{x}) = \exp\!\left(-\tfrac{1}{2}(\mathbf{x}-\boldsymbol{\mu})^\top \boldsymbol{\Sigma}^{-1}(\mathbf{x}-\boldsymbol{\mu})\right),$$1 s with caching, versus $$G(\mathbf{x}) = \exp\!\left(-\tfrac{1}{2}(\mathbf{x}-\boldsymbol{\mu})^\top \boldsymbol{\Sigma}^{-1}(\mathbf{x}-\boldsymbol{\mu})\right),$$2–$$G(\mathbf{x}) = \exp\!\left(-\tfrac{1}{2}(\mathbf{x}-\boldsymbol{\mu})^\top \boldsymbol{\Sigma}^{-1}(\mathbf{x}-\boldsymbol{\mu})\right),$$3 minutes for per-scene optimization baselines (Yu et al., 8 Mar 2026).

Storage and anti-aliasing have become distinct optimization targets. 4D Scaffold Gaussian Splatting reports a 97.8% storage reduction over 4DGS through sparse 4D anchors, shared decoders, neural velocity, and temporal coverage-aware anchor growing (Cho et al., 2024). LGS reports a compression rate exceeding $$G(\mathbf{x}) = \exp\!\left(-\tfrac{1}{2}(\mathbf{x}-\boldsymbol{\mu})^\top \boldsymbol{\Sigma}^{-1}(\mathbf{x}-\boldsymbol{\mu})\right),$$4 times while maintaining real-time rendering efficiency in dynamic surgical scenes by combining Deformation-Aware Pruning, Gaussian-Attribute Pruning, and 4D Feature Field Condensation (Liu et al., 2024). Alias-free 4D Gaussian Splatting introduces a 4D scale-adaptive filter and scale loss to eliminate high-frequency artifacts under increased rendering frequencies while reducing redundant Gaussians in multi-view video reconstruction (Chen et al., 23 Nov 2025).

5. Domain-specialized systems and applications

The video reconstruction literature now spans a wide range of domains. In monocular and multi-view dynamic scene reconstruction, Gaussian-Flow targets both monocular and synchronized multi-view video; OriGS addresses casually captured monocular videos by combining a Global Orientation Field with orientation-conditioned slicing; Sparse4DGS focuses specifically on sparse-frame inputs, including NeRF-Synthetic, HyperNeRF, NeRF-DS, and iPhone-4D (Lin et al., 2023, Wu et al., 27 Sep 2025, Shi et al., 10 Nov 2025). Vidu4D extends the paradigm to a single generated video and couples dynamic Gaussian surfels with text-to-4D generation pipelines (Wang et al., 2024).

Autonomous driving has become a major testbed. DrivingRecon directly predicts 4D Gaussian reconstructions from surround-view videos using the Prune and Dilate Block and static/dynamic decoupling, and further explores applications in model pre-training, vehicle adaptation, and scene editing (Lu et al., 2024). ReconDrive extends the 3D foundation model VGGT with Hybrid Gaussian Prediction Heads and a Static-Dynamic 4D Composition strategy, benchmarking on nuScenes and reporting performance competitive with per-scene optimization while being orders of magnitude faster (Yu et al., 8 Mar 2026). UrbanGS uses 2D semantic maps to distinguish static from potentially dynamic urban content, preserving static backgrounds while modeling dynamic actors (Li et al., 2024).

Surgical and endoscopic reconstruction has produced several specialized variants. LGS targets resource-limited robotic surgical services by compressing Gaussian count, attributes, and deformation encoders (Liu et al., 2024). EndoWave formulates dynamic endoscopic reconstruction as unified 4D Gaussian Splatting with an optical-flow-based geometric constraint and multi-resolution rational orthogonal wavelet supervision, achieving state-of-the-art reconstruction quality on EndoNeRF and StereoMIS (Wu et al., 27 Oct 2025).

Hand and human-centered reconstruction have also adopted Gaussian-based 4D models. Hand-4DGS is described as the first feed-forward framework for reconstructing dynamic 4D hands directly from egocentric videos, using a mesh-guided representation, temporal convolutions, and standard Gaussian splatting supervision without requiring expensive 3D hand pose ground-truth annotations (Bae et al., 17 Jun 2026). Its representation anchors Gaussians on a posed MANO mesh and samples additional splats on each triangle face, producing a dense surface covering suitable for high-fidelity appearance and geometry (Bae et al., 17 Jun 2026).

Medical imaging extends the concept beyond RGB radiance fields. X$$G(\mathbf{x}) = \exp\!\left(-\tfrac{1}{2}(\mathbf{x}-\boldsymbol{\mu})^\top \boldsymbol{\Sigma}^{-1}(\mathbf{x}-\boldsymbol{\mu})\right),$$5-Gaussian enables continuous-time 4D-CT reconstruction with radiative Gaussian splatting and self-supervised respiratory period learning, replacing phase discretization by a continuous motion model (Yu et al., 27 Mar 2025). Spatiotemporal Gaussian Optimization for 4D-CBCT reconstructs 4D cone-beam CT from sparse projections by optimizing Gaussian position, covariance, rotation, and density jointly with a Gaussian deformation network (Fu et al., 7 Jan 2025). 4D SlingBAG addresses dynamic 3D photoacoustic iterative reconstruction by coupling each Gaussian ball’s amplitude, size, and position across time through Gaussian temporal bases, keeping memory consumption almost the same as single-frame SlingBAG while achieving more than $$G(\mathbf{x}) = \exp\!\left(-\tfrac{1}{2}(\mathbf{x}-\boldsymbol{\mu})^\top \boldsymbol{\Sigma}^{-1}(\mathbf{x}-\boldsymbol{\mu})\right),$$6 speedup over per-frame reconstruction (Li et al., 2024).

6. Limitations, misconceptions, and research directions

Several limitations recur across otherwise different formulations. Pose quality remains critical: Gaussian-Flow notes that large pose errors degrade reconstruction, and monocular methods in general remain vulnerable to limited parallax and underconstrained geometry (Lin et al., 2023). Sparse4DGS explicitly motivates its texture-aware losses by observing that dynamic reconstruction methods fail in both canonical and deformed spaces under sparse-frame settings, especially in areas with high texture richness (Shi et al., 10 Nov 2025). ReconDrive lists non-rigid motion, temporal redundancy, SAM2 boundary errors, and broader generalization across geography, weather, and driving styles as open problems (Yu et al., 8 Mar 2026).

Another common issue is temporal aliasing and sampling mismatch. Alias-free 4D Gaussian Splatting shows that changing focal length or camera distance can introduce inflation, erosion, or high-frequency artifacts because Gaussian scale remains fixed while the effective pixel footprint changes; its remedy is a frequency-aware 4D scale-adaptive filter tied to a maximum sampling frequency (Chen et al., 23 Nov 2025). This suggests that temporal fidelity and view fidelity cannot be treated independently once spatiotemporal Gaussians are rendered under varying sampling conditions.

Static–dynamic decoupling is both a solution and a source of modeling tension. UrbanGS, SDD-4DGS, DrivingRecon, and ReconDrive all argue that static and dynamic components should not be treated uniformly (Li et al., 2024, Sun et al., 12 Mar 2025, Lu et al., 2024, Yu et al., 8 Mar 2026). At the same time, unified 4D models such as EndoWave and OriGS show that tightly coupled spatiotemporal statistics can improve coherence in domains where static and dynamic boundaries are less semantically separable (Wu et al., 27 Oct 2025, Wu et al., 27 Sep 2025). A plausible implication is that the appropriate degree of decoupling depends on domain structure: urban driving strongly benefits from explicit static–dynamic partitioning, while deformable biological or articulated scenes may benefit from unified spatiotemporal conditioning.

A final misconception is that feed-forward 4D Gaussian models simply replace optimization-based methods. The evidence is more specific. L4GM, DrivingRecon, ReconDrive, and Hand-4DGS demonstrate that generalizable feed-forward models can achieve strong quality and substantial speed advantages in settings with large training corpora and consistent sensor structure (Ren et al., 2024, Lu et al., 2024, Yu et al., 8 Mar 2026, Bae et al., 17 Jun 2026). Yet optimization-based methods remain prominent where scene-specific fidelity, explicit deformation control, or physics-grounded inverse problems are central, as in Gaussian-Flow, X$$G(\mathbf{x}) = \exp\!\left(-\tfrac{1}{2}(\mathbf{x}-\boldsymbol{\mu})^\top \boldsymbol{\Sigma}^{-1}(\mathbf{x}-\boldsymbol{\mu})\right),$$7-Gaussian, 4D-CBCT reconstruction, and 4D SlingBAG (Lin et al., 2023, Yu et al., 27 Mar 2025, Fu et al., 7 Jan 2025, Li et al., 2024).

Taken together, the literature portrays 4D Gaussian Reconstruction as an explicit, splat-friendly, and increasingly domain-adapted framework for time-varying reconstruction. Its central technical question is no longer whether Gaussians can represent dynamic scenes, but how temporal structure should be encoded: analytically, by deformation networks, by unified spatiotemporal covariance, by anchor-based neural decoders, or by amortized sequence prediction.