Compact 3D Gaussian Splatting For Dense Visual SLAM (2403.11247v2)

Abstract: Recent work has shown that 3D Gaussian-based SLAM enables high-quality reconstruction, accurate pose estimation, and real-time rendering of scenes. However, these approaches are built on a tremendous number of redundant 3D Gaussian ellipsoids, leading to high memory and storage costs, and slow training speed. To address the limitation, we propose a compact 3D Gaussian Splatting SLAM system that reduces the number and the parameter size of Gaussian ellipsoids. A sliding window-based masking strategy is first proposed to reduce the redundant ellipsoids. Then we observe that the covariance matrix (geometry) of most 3D Gaussian ellipsoids are extremely similar, which motivates a novel geometry codebook to compress 3D Gaussian geometric attributes, i.e., the parameters. Robust and accurate pose estimation is achieved by a global bundle adjustment method with reprojection loss. Extensive experiments demonstrate that our method achieves faster training and rendering speed while maintaining the state-of-the-art (SOTA) quality of the scene representation.

• The paper presents a novel 3D Gaussian Splatting method that compresses scene representation using online masking and a geometry codebook.
• The approach significantly improves rendering and training speeds while reducing memory usage for high-fidelity SLAM systems.
• Experimental validation on datasets like Replica and ScanNet demonstrates state-of-the-art performance in pose estimation and scene quality.

Enhancing Dense Visual SLAM with Compact 3D Gaussian Splatting

Introduction

Simultaneous Localization and Mapping (SLAM) serves as a cornerstone in the domain of computer vision, facilitating a plethora of applications across autonomous driving, robotics, and virtual/augmented reality. The advent of Neural Radiance Fields (NeRF) has marked a significant shift towards dense scene reconstruction within the SLAM ecosystem, integrating implicit scene representation with SLAM systems to achieve high levels of accuracy in rendering and pose estimation. This paper introduces a novel approach to 3D Gaussian Splatting (GS) within the milieu of Dense Visual SLAM systems, aimed at mitigating the limitations posed by storage inefficiencies and slow processing speeds inherent in existing methodologies.

Compact 3D Gaussian Scene Representation

The essence of our method lies in its ability to significantly reduce the redundancy of 3D Gaussian ellipsoids, which are traditionally employed in voluminous quantities for high-fidelity scene reconstruction. Our approach features a two-pronged strategy:

• Sliding Window-based Masking: We propose an online masking method utilizing a sliding window approach that effectively discerns and eliminates superfluous 3D Gaussian ellipsoids without compromising the quality of scene representation. This achievement is facilitated by the introduction of a learnable mask parameter that dynamically adjusts to the evolving scene complexity, ensuring efficient memory utilization.
• Geometry Codebook: A novel element of our methodology is the observation of inherent geometric similarities across the majority of Gaussian points within a scene. By developing a codebook-based method for geometric attribute compression, we enforce a more compact representation of 3D Gaussian ellipsoids. This encoding mechanism not only achieves a commendable reduction in memory footprint but also enhances the rendering and training speed of the SLAM system.

Robust and Accurate Pose Estimation

A significant contribution of this paper is the integration of a global bundle adjustment method with reprojection loss, aimed at refining pose estimation accuracy. This approach consolidates the robustness of camera tracking, leveraging historical observations to mitigate cumulative errors and enhance system reliability. The efficient management of a global keyframe database further underpins our contribution to improving the overall performance of GS-based SLAM systems.

Experimental Validation

The efficacy of our proposed system is rigorously tested across various datasets, including Replica, ScanNet, and TUM-RGBD, encompassing both synthetic and real-world environments. Our experiments highlight the system's capability to provide state-of-the-art performance in terms of scene representation quality, execution speed, and memory efficiency. Key findings from our experiments include:

• Performance Metrics: The system demonstrates an outstanding balance between high-quality scene representation (as observed through metrics such as PSNR, SSIM, and LPIPS) and the efficiency of pose estimation (evidenced by reduced ATE RMSE).
• Enhanced Speed and Efficiency: Our approach facilitates a near 176% increase in rendering speed, accompanied by a notable improvement in training speed and a reduction in memory usage by over 1.97×.

Theoretical and Practical Implications

The introduction of a compact 3D Gaussian scene representation method in Dense Visual SLAM systems presents profound implications for both theoretical advancement and practical application. From a theoretical standpoint, our work elucidates the potential of geometry compression via a codebook approach and highlights the efficacy of online masking for dynamic scene representation. Practically, the system paves the way for deploying high-fidelity SLAM on resource-constrained devices, thereby expanding the horizon for real-time applications in navigation, immersive technologies, and autonomous robotics.

Future Prospects

The development of our novel GS-based SLAM system opens avenues for future investigation, particularly in exploring adaptive mechanisms for codebook optimization, enhancing the robustness of reprojection loss in varied environmental conditions, and further compressing scene representation for ultra-efficient SLAM applications.

In conclusion, our work contributes significantly to the enhancement of dense visual SLAM by addressing critical challenges related to memory and processing efficiency. The proposed compact 3D Gaussian Splatting method not only achieves state-of-the-art reconstruction and pose estimation accuracy but also sets a new benchmark for the real-time execution of SLAM systems.