3DGS.zip: A survey on 3D Gaussian Splatting Compression Methods (2407.09510v4)

Abstract: 3D Gaussian Splatting (3DGS) has emerged as a cutting-edge technique for real-time radiance field rendering, offering state-of-the-art performance in terms of both quality and speed. 3DGS models a scene as a collection of three-dimensional Gaussians, or splats, with additional attributes optimized to conform to the scene's geometric and visual properties. Despite its advantages in rendering speed and image fidelity, 3DGS is limited by its significant storage and memory demands. These high demands make 3DGS impractical for mobile devices or headsets, reducing its applicability in important areas of computer graphics. To address these challenges and advance the practicality of 3DGS, this survey provides a comprehensive and detailed examination of compression and compaction techniques developed to make 3DGS more efficient. We categorize current approaches into compression techniques, which aim at achieving the highest quality at minimal data size, and compaction techniques, which aim for optimal quality with the fewest Gaussians. We introduce the basic mathematical concepts underlying the analyzed methods, as well as key implementation details and design choices. Our report thoroughly discusses similarities and differences among the methods, as well as their respective advantages and disadvantages. We establish a consistent standard for comparing these methods based on key performance metrics and datasets. Specifically, since these methods have been developed in parallel and over a short period of time, currently, no comprehensive comparison exists. This survey, for the first time, presents a unified standard to evaluate 3DGS compression techniques. To facilitate the continuous monitoring of emerging methodologies, we maintain a dedicated website that will be regularly updated with new techniques and revisions of existing findings https://w-m.github.io/3dgs-compression-survey/ .

• The paper surveys diverse compression methods for 3D Gaussian Splatting, providing metrics and dataset comparisons to evaluate techniques for reducing memory and improving rendering speed.
• Techniques surveyed include reducing memory footprint (up to 27x), LightGaussian (15x reduction, >200 FPS), and using methods like vector quantization and grid-based organization to enhance compactness and speed.
• Integrating these compression strategies aims to enable more efficient real-time 3DGS rendering, especially on limited devices, and the survey is open for community collaboration for continued advancements.

Survey on 3D Gaussian Splatting Compression Methods

The paper "3DGS.zip: A survey on 3D Gaussian Splatting Compression Methods" provides an extensive yet ongoing survey of 3D Gaussian Splatting (3DGS) compression techniques. The authors seek to provide a comprehensive summary to aid in the comparison of diverse methods by listing key metrics such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Learned Perceptual Image Patch Similarity (LPIPS), and storage size in megabytes across various datasets, including TanksAndTemples, MipNeRF360, DeepBlending, and SyntheticNeRF.

Contributions Focused on Memory Footprint Reduction and Performance Enhancements

The paper presents several distinct approaches to optimizing 3DGS, each aiming to improve efficiency in both storage and computation:

1. Reducing 3DGS Memory Footprint: This method targets the substantial memory requirements of 3D Gaussian Splatting used for novel view synthesis. By applying techniques such as resolution-aware primitive pruning and adaptive adjustment of spherical harmonics coefficients, the authors report a 27-fold reduction in memory size alongside a 1.7-fold increase in rendering speed without substantially impacting visual quality.
2. LightGaussian Compression: LightGaussian advances the state-of-the-art by achieving a 15x reduction in 3D Gaussian representations and enabling frame rates over 200 FPS. This approach capitalizes on network pruning ideas to remove redundant Gaussians while maintaining the fidelity of visual reconstruction.
3. Hash-grid Assisted Context (HAC): The HAC method leverages mutual information between Gaussian attributes and hash grid features, utilizing an adaptive quantization module to optimize compression without sacrificing rendering quality. It demonstrates notable compression ratios, further bolstering 3DGS model efficiency.
4. Compact 3D Gaussian Representation: Aiming to streamline Gaussian point requirements, this method incorporates volume-based masking and a grid-based approach to optimize spatial redundancy. Additionally, it uses a compact codebook-based representation that minimizes duplicate geometric patterns.
5. Vector Quantization for Compact3D: Utilizing K-means to cluster similar Gaussian parameters, this technique significantly reduces the memory footprint of learned radiance fields through the application of vector quantization and run-length encoding.
6. Self-Organizing Gaussian Grids: A novel aspect of this method is the organization of 3DGS parameters into a 2D grid, significantly simplifying data storage while ensuring high visual quality through enforced smoothness in the grid structure.
7. Compressed 3D Gaussian Splatting for Synthesis: Focusing on a new compressed representation, this contribution introduces steps like sensitivity-aware clustering and quantization-aware fine-tuning to enhance rendering efficiency and memory use.

Implications and Future Developments

The survey highlights multiple innovative strategies to mitigate the memory and computational strains associated with 3DGS models. By integrating these varied compression techniques, future approaches can achieve enhanced compactness and efficiency, facilitating real-time rendering applications, especially for limited-capacity devices. The work-in-progress nature of this survey, with its open invitation for community collaboration, promises continued advancement in 3DGS compression methodologies.

This evolving field of paper not only underscores the growing importance of effective data compression in scalable 3D visualization tasks but also foresees future developments where continued research can incorporate these advancements into more holistic and efficient machine vision frameworks. As researchers push the boundaries in optimizing and compressing 3D data, significant enhancements in performance and new applications within artificial intelligence and graphics are anticipated.