Trimming the Fat: Efficient Compression of 3D Gaussian Splats through Pruning (2406.18214v2)

Abstract: In recent times, the utilization of 3D models has gained traction, owing to the capacity for end-to-end training initially offered by Neural Radiance Fields and more recently by 3D Gaussian Splatting (3DGS) models. The latter holds a significant advantage by inherently easing rapid convergence during training and offering extensive editability. However, despite rapid advancements, the literature still lives in its infancy regarding the scalability of these models. In this study, we take some initial steps in addressing this gap, showing an approach that enables both the memory and computational scalability of such models. Specifically, we propose "Trimming the fat", a post-hoc gradient-informed iterative pruning technique to eliminate redundant information encoded in the model. Our experimental findings on widely acknowledged benchmarks attest to the effectiveness of our approach, revealing that up to 75% of the Gaussians can be removed while maintaining or even improving upon baseline performance. Our approach achieves around 50$\times$ compression while preserving performance similar to the baseline model, and is able to speed-up computation up to 600 FPS.

• The paper presents an iterative gradient-informed pruning technique that selectively removes up to 75% of Gaussians to maintain model performance.
• It utilizes post-hoc pruning based on opacity and gradient information, followed by fine-tuning to ensure accurate 3D scene reconstruction.
• The approach achieves remarkable compression ratios (up to 25x or 50x in integrated pipelines), enabling real-time rendering in resource-constrained environments.

Efficient Compression of 3D Gaussian Splats through Pruning

The paper "Trimming the Fat: Efficient Compression of 3D Gaussian Splats through Pruning" by Muhammad Salman Ali et al. proposes a methodology to address the memory and computational scalability issues of 3D Gaussian Splatting (3DGS) models. The primary contribution of this work is the introduction of a post-hoc gradient-informed iterative pruning technique that achieves substantial compression without compromising performance, thus enhancing the applicability of 3DGS models in resource-constrained environments.

Background and Motivation

3D scene reconstruction techniques, particularly Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), have garnered significant attention for their ability to render photo-realistic novel views from 2D images. While NeRFs have demonstrated success, they suffer from slow training and rendering times. 3DGS offers a promising alternative by employing a sparse adaptive scene representation, leading to rapid convergence during training and real-time rendering capabilities. However, the scalability of 3DGS models remains a challenge due to the substantial computational and memory requirements associated with millions of 3D Gaussian entities.

Methodology

The authors propose a pruning technique that iteratively eliminates redundant Gaussians based on their opacity levels and gradients. The motivation is to exploit the over-parameterization inherent in the 3DGS models, where many Gaussians contribute minimally to the overall scene representation. The key steps in their approach are:

1. Gradient-Informed Pruning: Gaussians with low opacity and gradients are pruned progressively in each iteration. This ensures that only the Gaussians that minimally impact the rendering quality are removed.
2. Iterative Pruning and Fine-Tuning: This involves iterative pruning followed by fine-tuning of the remaining Gaussians to adjust their parameters such that the scene features are captured accurately. The empirical observation indicates that gradual pruning is more effective than one-shot pruning, preventing the optimization algorithm from converging to sub-optimal local minima.

Experimental Results

The proposed method was evaluated on benchmark datasets, including Mip-NeRF360, Tanks and Temples, and Deep Blending. The results demonstrate that up to 75\% of the Gaussians can be pruned while maintaining or even improving baseline performance. Specifically, the method achieved compression rates up to 25$\times$ without compromising visual quality, as quantified by SSIM, PSNR, and LPIPS metrics. Furthermore, the technique was integrated with an existing compression pipeline, achieving a compression ratio of approximately 50$\times$ while preserving baseline performance.

Implications and Future Work

The pruning technique demonstrates significant implications for the practical use of 3DGS models in various applications, such as VR/AR and mobile gaming, where resources are often limited. By drastically reducing the memory footprint and computational requirements, this approach facilitates deployment on edge devices. The ability to maintain real-time performance with such high compression ratios suggests a feasible path for integrating 3DGS in bandwidth-constrained environments.

Theoretically, the paper hints at further optimization opportunities through more sophisticated pruning criteria and potential integration with quantization-aware training techniques. Future work may include extending the pruning methodology to other forms of scene representations and exploring advanced neural network-based approaches to further enhance the compressibility and efficiency of 3D scene rendering techniques.

In summary, the authors provide a robust solution to the scalability issues of 3DGS models, offering a practical method that balances compression and performance effectively. The iterative gradient-aware pruning approach not only strengthens the real-time rendering capabilities of 3DGS models but also opens avenues for their application in resource-constrained scenarios, aligning well with the broader objectives of scalable and efficient 3D scene reconstruction technologies.