Optimized Minimal 4D Gaussian Splatting (2510.03857v1)

Abstract: 4D Gaussian Splatting has emerged as a new paradigm for dynamic scene representation, enabling real-time rendering of scenes with complex motions. However, it faces a major challenge of storage overhead, as millions of Gaussians are required for high-fidelity reconstruction. While several studies have attempted to alleviate this memory burden, they still face limitations in compression ratio or visual quality. In this work, we present OMG4 (Optimized Minimal 4D Gaussian Splatting), a framework that constructs a compact set of salient Gaussians capable of faithfully representing 4D Gaussian models. Our method progressively prunes Gaussians in three stages: (1) Gaussian Sampling to identify primitives critical to reconstruction fidelity, (2) Gaussian Pruning to remove redundancies, and (3) Gaussian Merging to fuse primitives with similar characteristics. In addition, we integrate implicit appearance compression and generalize Sub-Vector Quantization (SVQ) to 4D representations, further reducing storage while preserving quality. Extensive experiments on standard benchmark datasets demonstrate that OMG4 significantly outperforms recent state-of-the-art methods, reducing model sizes by over 60% while maintaining reconstruction quality. These results position OMG4 as a significant step forward in compact 4D scene representation, opening new possibilities for a wide range of applications. Our source code is available at https://minshirley.github.io/OMG4/.

• The paper introduces a multi-stage pipeline that reduces the Gaussian count by up to 99% with minimal perceptual loss.
• It details steps including SD-Score sampling, pruning, merging, and sub-vector quantization to optimize storage and performance.
• Empirical results demonstrate significant model size reduction with maintained PSNR and compatibility across diverse 4DGS backbones.

Optimized Minimal 4D Gaussian Splatting: A Technical Analysis

Introduction and Motivation

The paper "Optimized Minimal 4D Gaussian Splatting" (OMG4) addresses the critical challenge of storage and computational overhead in dynamic scene representation using 4D Gaussian Splatting (4DGS). While 4DGS enables real-time photorealistic rendering of dynamic scenes by parameterizing space-time volumes with millions of Gaussian primitives, the resulting models are prohibitively large for practical deployment, especially in resource-constrained or streaming scenarios. OMG4 introduces a multi-stage compression framework that systematically reduces the number of Gaussians and compresses their attributes, achieving substantial reductions in model size with minimal loss in reconstruction fidelity.

Figure 1: The OMG4 pipeline and rate-distortion performance, demonstrating significant improvements over prior methods with higher FPS and lower storage.

Multi-Stage Gaussian Minimization Pipeline

The OMG4 framework consists of four sequential stages: Gaussian Sampling, Gaussian Pruning, Gaussian Merging, and Attribute Compression. Each stage is designed to progressively eliminate redundancy and condense the representation while preserving the essential spatio-temporal information required for high-fidelity rendering.

Figure 2: The overall architecture of OMG4, detailing the flow from initial Gaussian set to compressed representation.

Gaussian Sampling via Static–Dynamic Score

Gaussian Sampling is the initial stage, where the importance of each Gaussian is quantified using a novel Static–Dynamic Score (SD-Score). The SD-Score combines the sensitivity of the reconstruction loss to spatial perturbations (Static Score) and temporal changes (Dynamic Score), computed as:

$$\textrm{SD}^{(i)} = S_{\textrm{grad}}^{(i)} \cdot T_{\textrm{grad}}^{(i)}$$

where $S_{\textrm{grad}}^{(i)}$ accumulates view-space gradients and $T_{\textrm{grad}}^{(i)}$ accumulates time gradients for the $i$-th Gaussian. This approach enables the selection of a subset of Gaussians that are most critical for both static and dynamic regions, typically retaining only 20% of the original set with negligible quality loss.

Figure 3: Comparison of rendered images before and after Gaussian Sampling, showing minimal perceptual degradation despite a substantial reduction in primitives.

Gaussian Pruning

Following sampling, Gaussian Pruning further refines the set by thresholding both static and dynamic scores. Gaussians with low values in both metrics are considered redundant and removed. The thresholds are set using quantiles of the score distributions, ensuring that only the most salient Gaussians are retained.

Figure 4: Visualization of the pruning process in SD-score space and its effect on rendered images.

Gaussian Merging

Gaussian Merging addresses inter-Gaussian redundancy by clustering primitives with high spatial and appearance similarity within spatio-temporal grid cells. Clusters are merged into single proxy Gaussians using learnable weights for position and appearance attributes, optimized via backpropagation. This stage is repeated with progressively coarser grids to maximize compression.

Attribute Compression and Sub-Vector Quantization

After minimizing the Gaussian set, OMG4 compresses the high-dimensional attributes using an MLP-based implicit appearance model, extended to handle time-varying properties. Sub-Vector Quantization (SVQ), originally developed for static 3DGS, is generalized to 4D attributes, partitioning vectors and quantizing each sub-vector with dedicated codebooks. A staged SVQ scheme is employed to stabilize optimization, first quantizing 3D attributes and then 4D attributes. Final compression is achieved using Huffman and LZMA encoding.

Experimental Results and Analysis

OMG4 is evaluated on the N3DV and MPEG datasets, demonstrating model size reductions of up to 99% compared to Real-Time4DGS, with storage requirements dropping from gigabytes to a few megabytes. Notably, OMG4 achieves a 65% reduction in storage relative to GIFStream while maintaining or improving PSNR and perceptual quality metrics.

Figure 5: Qualitative results on N3DV, illustrating the preservation of fine details after aggressive compression.

Ablation studies confirm the necessity of the multi-stage pipeline: combining sampling and pruning in a single step leads to a measurable drop in PSNR, while intermediate optimization between stages stabilizes training and improves final quality. The merging stage further reduces the number of Gaussians without significant loss in fidelity.

Figure 6: Effect of SD-Score-based sampling versus $\Sigma_t$-based sampling, showing superior distribution of Gaussians across dynamic regions.

OMG4 is also shown to be compatible with alternative 4DGS backbones such as FreeTimeGS, achieving 90% storage reduction with minimal impact on reconstruction quality, underscoring the generality of the approach.

Figure 7: Additional qualitative results on N3DV, demonstrating robustness across diverse scenes.

Implementation Considerations

The pipeline is designed for efficiency and scalability. All stages are differentiable and can be implemented using standard deep learning frameworks. Hyperparameters such as sampling ratios, pruning quantiles, and codebook sizes are empirically tuned to balance storage and fidelity. The method is GPU-friendly and can be executed on a single RTX 3090, with all experiments using consistent settings across datasets.

The staged SVQ approach is critical for stable optimization, especially when quantizing attributes that affect both static and dynamic properties. The merging algorithm leverages locality in spatio-temporal grids to avoid merging temporally mismatched Gaussians, preserving coherence.

Implications and Future Directions

OMG4 sets a new benchmark for compact dynamic scene representation, enabling real-time rendering and streaming of high-fidelity 4D content on resource-constrained devices. The framework's modularity allows integration with various 4DGS architectures and potential extension to other explicit scene representations.

The strong numerical results—99% reduction in model size with negligible perceptual loss—challenge the prevailing assumption that millions of primitives are necessary for high-quality dynamic scene synthesis. This opens avenues for further research in adaptive compression, online streaming, and hardware-aware deployment of dynamic radiance fields.

Future work may explore joint optimization of visibility masks for further FPS improvements, integration with neural rendering pipelines, and application to long-duration, high-resolution dynamic content in VR/AR and autonomous driving.

Conclusion

OMG4 introduces a principled, multi-stage framework for minimal 4D Gaussian Splatting, combining gradient-based importance sampling, redundancy-aware pruning, correlation-driven merging, and advanced attribute compression. The approach achieves dramatic reductions in storage and computational requirements while maintaining state-of-the-art reconstruction quality, with broad applicability to dynamic scene rendering and streaming. The results substantiate the feasibility of highly compact, high-fidelity 4D scene representations and motivate continued research in efficient explicit modeling for dynamic environments.