--- title: 'TED-4DGS: Dynamic 4D Gaussian Splatting' url: https://www.emergentmind.com/topics/ted-4dgs type: topic --- # TED-4DGS: Dynamic 4D Gaussian Splatting TED-4DGS, or Temporally Activated and Embedding-based Deformation for 4DGS Compression, is a dynamic extension and compression scheme for 4D Gaussian Splatting (4DGS) representations in dynamic scene modeling. TED-4DGS addresses the challenge of building compact and temporally controllable representations of dynamic 3D scenes driven by real video data, unifying advantages of canonical-anchor-based and explicit space-time 4DGS paradigms, and is designed for rate–distortion (R–D)-optimized compression on real-world dynamic scene benchmarks [2512.05446]. ## 1. Anchor-Based 3DGS Foundation and Dynamic Extension TED-4DGS builds upon ScaffoldGS, a static anchor-based 3D Gaussian Splatting (3DGS) model. ScaffoldGS uses a sparse set of anchor points $\{x_a\}$ placed on a 3D grid, with each anchor $a$ associated with a feature vector $f_a\in\mathbb{R}^F$ that parameterizes $K$ local Gaussian primitives. An MLP decoder predicts offsets $\{O_{a,i}\}_{i=0}^{K-1}$ and scale vector $l_a$ so that each Gaussian mean $\mu_{a,i}=x_a+l_a\odot O_{a,i}$. Additional MLP outputs parameterize per-Gaussian scale $s_{a,i}$, rotation $r_{a,i}$, color $c_{a,i}$, and opacity $\alpha_{a,i}$. Rendering involves alpha compositing of these Gaussians in canonical space. TED-4DGS extends this formulation to the dynamic 4D setting by: - Injecting per-anchor, learnable temporal-activation parameters $\tau_a(t)$ for gating appearance/disappearance over time. - Introducing per-anchor low-dimensional temporal embeddings $\phi_a$, mapped via a shared global "deformation bank" $Z$ to produce anchor-specific deformation fields $\Delta x_a(t)$ and $\Delta f_a(t)$. - Integrating an implicit neural representation (INR)-based hyperprior and a channel-wise autoregressive model for entropy-aware, rate–distortion-optimized attribute compression. ## 2. Temporal Activation Mechanism To enable explicit, learnable control over dynamic object occlusion and disocclusion, TED-4DGS introduces a temporal-activation function for each anchor: - Each anchor $a$ learns four scalars $(a_s, b_s; a_f, b_f)$, where $a_s, a_f\in[0,T]$ mark "soft" start and end frames and $b_s, b_f$ control activation/deactivation smoothness. - The temporal activation is defined as: $$ \tau_a(t) = \begin{cases} \exp[-((t-a_s)/b_s)^2], & ta_f \end{cases} $$ - At render time, anchor opacities are modulated as $\alpha_{a,i}(t) = \alpha_{a,i} \cdot \tau_a(t)$. This construction allows Gaussians to fade in and out without resorting to spatial deformation for invisibility, sharply reducing parameter count and improving temporal realism. ## 3. Embedding-Based Deformation via Shared Deformation Bank TED-4DGS represents nonrigid per-anchor, per-time motion with a lightweight embedding mechanism: - A global deformation bank $Z\in\mathbb{R}^{F/2\times D}$ (one $D$-vector per two frames). - Each anchor has a temporal embedding $\phi_a\in\mathbb{R}^d$. At time $t$, linear interpolation in $Z$ yields $z^{(t)} \in \mathbb{R}^D$. Simple MLP $F_{proj}$ maps $\phi_a$ to $w_a\in\mathbb{R}^D$; then $w_a\odot z^{(t)}$ parametrizes anchor-specific deformation. - A compact MLP $F_{deform}$ maps the elementwise product to produce $(\Delta f_a(t), \Delta x_a(t))$: $$(\Delta f_a, \Delta x_a) = F_{deform}(w_a\odot z^{(t)})$$ - Anchor feature and position are time-modulated as $f_a(t) = f_a + \Delta f_a(t),\ x_a(t) = x_a + \Delta x_a(t)$. Empirically, this multiplicative query design yields $\sim$0.9 dB PSNR gain over concatenation at equivalent size [2512.05446]. ## 4. Rate–Distortion-Optimized Compression Framework TED-4DGS aims for joint optimization of perceptual quality and bitrate by: - Minimizing: $$ \mathcal{L} = D + \lambda R + \lambda_{offset} L_{offset} + \lambda_{temp} L_{temp} + \lambda_{vol} L_{vol} + \lambda_{tv} L_{tv} $$ where $D$ is the sum of L1 and (1–SSIM) losses over renderings, $R$ is average bits-per-anchor measured via the entropy model, $L_{offset},L_{temp}$ are sparsity regularizers for pruning, $L_{vol}$ promotes scale consistency, $L_{tv}$ regularizes the temporal deformation bank. - An INR hyperprior models each anchor attribute’s probability as a Gaussian with analytically computed mean $\mu_h$ and std $\sigma_h$, both being outputs of MLP $H$ applied to sinusoidal positional encodings $\gamma(x_a)$. Each quantized attribute $\hat{a}$ is assigned: $$ p(\hat{a}|x_a) = \int_{\hat{a}-q/2}^{\hat{a}+q/2} \mathcal{N}(\mu_h, \sigma_h) da $$ - Channel-wise autoregressive modeling of anchor feature vectors $f_a$, with each channel $f_{a,c}$ decoded conditioned on prior channels using a masked MLP. - Quantization step $q$ is learnable per attribute. - Arithmetic coding is performed over the learned probabilistic model; model hyperparameters and MLP weights are transmitted once. Ablations show that the INR hyperprior achieves a 20% BD-Rate saving over a factorizable prior, outperforming triplane/hashing hyperpriors by $\sim$12%. ## 5. Training Pipeline and Implementation The TED-4DGS compression workflow involves: - Uniform quantization of each Gaussian attribute with a learned quantization step. - Entropy coding using arithmetic coders against the INR hyperprior/auto-regressive model. - Anchor coordinates stored in FP16, neural network weights and deformation bank in FP32. - Progressive training with a 20k-iteration delay before learning $(a_s, a_f)$, which stabilizes mask pruning and deformation learning; removing this delay causes $\sim$20% higher bitrates. Typical training converges after $\sim$1M iterations per scene (∼12 hours, PyTorch on RTX 3090). Rendering achieves $\sim$70 fps for $536\times960$ resolution in novel-view synthesis [2512.05446]. ## 6. Empirical Results and Comparisons TED-4DGS is evaluated on Neu3D and HyperNeRF dynamic benchmarks, outperforming Light4GS and ADC-GS in rate–distortion metrics. For example: | Method | PSNR (Neu3D) | Size (MB, Neu3D) | PSNR (HyperNeRF) | Size (MB, HyperNeRF) | |-------------------|--------------|------------------|------------------|----------------------| | Light4GS (low) | 31.48 | 3.77 | 25.35 | 5.15 | | ADC-GS (low) | 31.41 | 4.04 | 25.42 | 4.02 | | TED-4DGS (low) | 31.63 | 1.73 | 25.22 | 2.36 | | Light4GS (high) | 31.69 | 5.46 | 25.55 | 8.87 | | ADC-GS (high) | 31.67 | 6.57 | 25.68 | 6.67 | | TED-4DGS (high) | 32.25 | 2.26 | 25.67 | 3.72 | TED-4DGS achieves up to –63% average bitrate savings on Neu3D and –45% on HyperNeRF relative to ADC-GS at matched PSNR. Rate–distortion performance strictly dominates prior art over all tested operating points. ## 7. Ablation Studies and Implementation Nuances Key findings from ablation and implementation studies: - Temporal-activation module substantially improves compression, reducing size by ≈9% and avoiding unnatural anchor deformations. - Use of multiplicative deformation queries is measurably superior to concatenation. - Color correction via a per-camera MLP mitigates cross-view color bias, improving PSNR by ∼0.3 dB. - For slow-motion scenes, most anchors have long-lived activation ($\Delta\tau>0.8$ for 97%); fast-motion scenes see a larger fraction of short-lived anchors (18% with $\Delta\tau<0.2$). - Extremely large feature dimensions $F$ or very fast motions marginally inflate the size of the deformation bank $Z$, but $F/2$ bank entries suffice for most sequences. - Currently, the number of Gaussians per anchor ($K=8$) is fixed; adapting $K$ per-anchor would require additional signaling infrastructure in the compressed stream. --- TED-4DGS introduces a temporally activated extension to sparse anchor-based 3DGS, a compact per-anchor dynamic deformation mechanism, and the first INR+autoregressive coding strategy for true rate–distortion-optimized dynamic 4DGS compression. Its combination of temporal gating, shared low-rank deformation, and targeted entropy modeling yields leading compression rates with high fidelity on challenging real-world dynamic scene data [2512.05446]. Source: https://www.emergentmind.com/topics/ted-4dgs