TED-4DGS: Dynamic 4D Gaussian Splatting

• TED-4DGS is a dynamic 4D compression scheme that extends anchor-based 3D Gaussian Splatting with temporal activation to model real-world dynamic scenes.
• It integrates learnable temporal gating and a shared deformation bank to modulate Gaussian primitives, ensuring smooth transitions and enhanced feature fidelity.
• Its combination of an INR hyperprior and autoregressive coding achieves significant bitrate savings (up to 63%) while maintaining competitive PSNR on benchmark datasets.

TED-4DGS, or Temporally Activated and Embedding-based Deformation for 4DGS Compression, is a dynamic extension and compression scheme for 4D Gaussian Splatting (4DGS) representations in dynamic scene modeling. TED-4DGS addresses the challenge of building compact and temporally controllable representations of dynamic 3D scenes driven by real video data, unifying advantages of canonical-anchor-based and explicit space-time 4DGS paradigms, and is designed for rate–distortion (R–D)-optimized compression on real-world dynamic scene benchmarks (Ho et al., 5 Dec 2025).

1. Anchor-Based 3DGS Foundation and Dynamic Extension

TED-4DGS builds upon ScaffoldGS, a static anchor-based 3D Gaussian Splatting (3DGS) model. ScaffoldGS uses a sparse set of anchor points $\{x_a\}$ placed on a 3D grid, with each anchor $a$ associated with a feature vector $f_a\in\mathbb{R}^F$ that parameterizes $K$ local Gaussian primitives. An MLP decoder predicts offsets $\{O_{a,i}\}_{i=0}^{K-1}$ and scale vector $l_a$ so that each Gaussian mean $\mu_{a,i}=x_a+l_a\odot O_{a,i}$. Additional MLP outputs parameterize per-Gaussian scale $s_{a,i}$, rotation $r_{a,i}$, color $c_{a,i}$, and opacity $a$0. Rendering involves alpha compositing of these Gaussians in canonical space.

TED-4DGS extends this formulation to the dynamic 4D setting by:

• Injecting per-anchor, learnable temporal-activation parameters $a$1 for gating appearance/disappearance over time.
• Introducing per-anchor low-dimensional temporal embeddings $a$2, mapped via a shared global "deformation bank" $a$3 to produce anchor-specific deformation fields $a$4 and $a$5.
• Integrating an implicit neural representation (INR)-based hyperprior and a channel-wise autoregressive model for entropy-aware, rate–distortion-optimized attribute compression.

2. Temporal Activation Mechanism

To enable explicit, learnable control over dynamic object occlusion and disocclusion, TED-4DGS introduces a temporal-activation function for each anchor:

• Each anchor $a$6 learns four scalars $a$7, where $a$8 mark "soft" start and end frames and $a$9 control activation/deactivation smoothness.
• The temporal activation is defined as:

$f_a\in\mathbb{R}^F$0

• At render time, anchor opacities are modulated as $f_a\in\mathbb{R}^F$1. This construction allows Gaussians to fade in and out without resorting to spatial deformation for invisibility, sharply reducing parameter count and improving temporal realism.

3. Embedding-Based Deformation via Shared Deformation Bank

TED-4DGS represents nonrigid per-anchor, per-time motion with a lightweight embedding mechanism:

• A global deformation bank $f_a\in\mathbb{R}^F$2 (one $f_a\in\mathbb{R}^F$3-vector per two frames).
• Each anchor has a temporal embedding $f_a\in\mathbb{R}^F$4. At time $f_a\in\mathbb{R}^F$5, linear interpolation in $f_a\in\mathbb{R}^F$6 yields $f_a\in\mathbb{R}^F$7. Simple MLP $f_a\in\mathbb{R}^F$8 maps $f_a\in\mathbb{R}^F$9 to $K$0; then $K$1 parametrizes anchor-specific deformation.
• A compact MLP $K$2 maps the elementwise product to produce $K$3:

$K$4

• Anchor feature and position are time-modulated as $K$5.

Empirically, this multiplicative query design yields $K$60.9 dB PSNR gain over concatenation at equivalent size (Ho et al., 5 Dec 2025).

4. Rate–Distortion-Optimized Compression Framework

TED-4DGS aims for joint optimization of perceptual quality and bitrate by:

• Minimizing:

$K$7

where $K$8 is the sum of L1 and (1–SSIM) losses over renderings, $K$9 is average bits-per-anchor measured via the entropy model, $\{O_{a,i}\}_{i=0}^{K-1}$0 are sparsity regularizers for pruning, $\{O_{a,i}\}_{i=0}^{K-1}$1 promotes scale consistency, $\{O_{a,i}\}_{i=0}^{K-1}$2 regularizes the temporal deformation bank.

• An INR hyperprior models each anchor attribute’s probability as a Gaussian with analytically computed mean $\{O_{a,i}\}_{i=0}^{K-1}$3 and std $\{O_{a,i}\}_{i=0}^{K-1}$4, both being outputs of MLP $\{O_{a,i}\}_{i=0}^{K-1}$5 applied to sinusoidal positional encodings $\{O_{a,i}\}_{i=0}^{K-1}$6. Each quantized attribute $\{O_{a,i}\}_{i=0}^{K-1}$7 is assigned:

$\{O_{a,i}\}_{i=0}^{K-1}$8

• Channel-wise autoregressive modeling of anchor feature vectors $\{O_{a,i}\}_{i=0}^{K-1}$9, with each channel $l_a$0 decoded conditioned on prior channels using a masked MLP.
• Quantization step $l_a$1 is learnable per attribute.
• Arithmetic coding is performed over the learned probabilistic model; model hyperparameters and MLP weights are transmitted once.

Ablations show that the INR hyperprior achieves a 20% BD-Rate saving over a factorizable prior, outperforming triplane/hashing hyperpriors by $l_a$212%.

5. Training Pipeline and Implementation

The TED-4DGS compression workflow involves:

• Uniform quantization of each Gaussian attribute with a learned quantization step.
• Entropy coding using arithmetic coders against the INR hyperprior/auto-regressive model.
• Anchor coordinates stored in FP16, neural network weights and deformation bank in FP32.
• Progressive training with a 20k-iteration delay before learning $l_a$3, which stabilizes mask pruning and deformation learning; removing this delay causes $l_a$420% higher bitrates.

Typical training converges after $l_a$51M iterations per scene (∼12 hours, PyTorch on RTX 3090). Rendering achieves $l_a$670 fps for $l_a$7 resolution in novel-view synthesis (Ho et al., 5 Dec 2025).

6. Empirical Results and Comparisons

TED-4DGS is evaluated on Neu3D and HyperNeRF dynamic benchmarks, outperforming Light4GS and ADC-GS in rate–distortion metrics. For example:

Method	PSNR (Neu3D)	Size (MB, Neu3D)	PSNR (HyperNeRF)	Size (MB, HyperNeRF)
Light4GS (low)	31.48	3.77	25.35	5.15
ADC-GS (low)	31.41	4.04	25.42	4.02
TED-4DGS (low)	31.63	1.73	25.22	2.36
Light4GS (high)	31.69	5.46	25.55	8.87
ADC-GS (high)	31.67	6.57	25.68	6.67
TED-4DGS (high)	32.25	2.26	25.67	3.72

TED-4DGS achieves up to –63% average bitrate savings on Neu3D and –45% on HyperNeRF relative to ADC-GS at matched PSNR. Rate–distortion performance strictly dominates prior art over all tested operating points.

7. Ablation Studies and Implementation Nuances

Key findings from ablation and implementation studies:

• Temporal-activation module substantially improves compression, reducing size by ≈9% and avoiding unnatural anchor deformations.
• Use of multiplicative deformation queries is measurably superior to concatenation.
• Color correction via a per-camera MLP mitigates cross-view color bias, improving PSNR by ∼0.3 dB.
• For slow-motion scenes, most anchors have long-lived activation ($l_a$8 for 97%); fast-motion scenes see a larger fraction of short-lived anchors (18% with $l_a$9).
• Extremely large feature dimensions $\mu_{a,i}=x_a+l_a\odot O_{a,i}$0 or very fast motions marginally inflate the size of the deformation bank $\mu_{a,i}=x_a+l_a\odot O_{a,i}$1, but $\mu_{a,i}=x_a+l_a\odot O_{a,i}$2 bank entries suffice for most sequences.
• Currently, the number of Gaussians per anchor ($\mu_{a,i}=x_a+l_a\odot O_{a,i}$3) is fixed; adapting $\mu_{a,i}=x_a+l_a\odot O_{a,i}$4 per-anchor would require additional signaling infrastructure in the compressed stream.

---

TED-4DGS introduces a temporally activated extension to sparse anchor-based 3DGS, a compact per-anchor dynamic deformation mechanism, and the first INR+autoregressive coding strategy for true rate–distortion-optimized dynamic 4DGS compression. Its combination of temporal gating, shared low-rank deformation, and targeted entropy modeling yields leading compression rates with high fidelity on challenging real-world dynamic scene data (Ho et al., 5 Dec 2025).