DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes (2312.07920v3)

Abstract: We present DrivingGaussian, an efficient and effective framework for surrounding dynamic autonomous driving scenes. For complex scenes with moving objects, we first sequentially and progressively model the static background of the entire scene with incremental static 3D Gaussians. We then leverage a composite dynamic Gaussian graph to handle multiple moving objects, individually reconstructing each object and restoring their accurate positions and occlusion relationships within the scene. We further use a LiDAR prior for Gaussian Splatting to reconstruct scenes with greater details and maintain panoramic consistency. DrivingGaussian outperforms existing methods in dynamic driving scene reconstruction and enables photorealistic surround-view synthesis with high-fidelity and multi-camera consistency. Our project page is at: https://github.com/VDIGPKU/DrivingGaussian.

• The paper introduces a novel Composite Gaussian Splatting framework that reconstructs both static and dynamic elements in autonomous driving scenes.
• It employs incremental static 3D Gaussians and dynamic Gaussian graphs to achieve detailed, multi-view consistent scene modeling.
• LiDAR priors enhance reconstruction accuracy by providing robust geometric cues, boosting fidelity in large-scale dynamic environments.

Composite Gaussian Splatting Enhances Autonomous Driving Scene Reconstruction

Introduction to DrivingGaussian

The field of autonomous driving demands precise and comprehensive 3D scene understanding, motivating the development of sophisticated methods for large-scale dynamic scene reconstruction. In this context, the paper introduces DrivingGaussian, an innovative approach that capitalizes on Composite Gaussian Splatting to effectively and efficiently represent intricate driving scenes involving both static backgrounds and dynamic objects. Unlike prior methods that rely heavily on neural radiance fields (NeRF) and face challenges with dynamic content and computational efficiency, DrivingGaussian presents a hierarchical modeling strategy that adeptly handles large-scale scenes captured from multiple sensors. This approach not only surpasses existing methods in terms of reconstruction fidelity but also promises practical implications for enhancing the validation frameworks for autonomous driving systems.

Key Contributions

DrivingGaussian's foremost contribution lies in its novel application of Composite Gaussian Splatting, marking a first in the field of dynamic driving scene representation. The methodology is built upon two primary modules:

• Incremental Static 3D Gaussians, which reconstruct the static background by embracing the temporal and spatial variations across sequential frames from surrounding multi-camera setups.
• Composite Dynamic Gaussian Graphs, dedicated to precisely modeling and integrating multiple dynamic objects within the static background, leveraging a Gaussian graph for dynamic object integration.

Moreover, the integration of a LiDAR prior within the Gaussian Splatting process is another significant contribution, providing a robust geometric prior that ensures enhanced detail recovery and multi-view consistency without reliance on dense point clouds.

Methodological Insights

Composite Gaussian Splatting

The essence of the proposed Composite Gaussian Splatting lies in its bifurcated approach towards scene reconstruction:

1. Incremental Construction for Static Backgrounds: To tackle the variability and complexity of static scenes, DrivingGaussian incrementally constructs the scene using static 3D Gaussians. This method not only accommodates the variability introduced by the ego vehicle's movement but also ensures the integrity and continuity of the static background.
2. Dynamic Object Modeling: For dynamic entities in the scene, the method employs a composite dynamic Gaussian graph. This unique construction allows for the individual and collective modeling of dynamic objects, ensuring their accurate representation over time.

LiDAR Prior

A distinctive aspect of DrivingGaussian is its utilization of LiDAR data as a geometric prior, a strategy that significantly augments the precision of the reconstruction. Unlike methods that solely depend on image data, the inclusion of LiDAR information aids in overcoming the limitations associated with sparse sensor data, enabling the recovery of finer details and sustaining panoramic consistency across the reconstructed scene.

Empirical Validation

Extensive experiments demonstrate that DrivingGaussian achieves state-of-the-art performance in reconstructing driving scenes. The method exhibits remarkable enhancements over existing techniques on public autonomous driving datasets, showcasing its capability in synthesizing photorealistic surround views and accurately reconstructing dynamic scenes. Furthermore, the proposed framework enables the effective simulation of corner cases, paving the way for more comprehensive safety validations of autonomous driving systems.

Future Directions in AI and Autonomous Driving

DrivingGaussian's introduction of Composite Gaussian Splatting opens up new avenues for research and development in the field of autonomous driving. The method's ability to efficiently handle large-scale, dynamic environments suggests potential applications beyond autonomous driving, including virtual reality, augmented reality, and the broader domain of machine perception. Future research may explore the extension of this methodology to other types of dynamic scenes, enhancing the generalizability and utility of the approach.

In conclusion, DrivingGaussian represents a significant advancement in the modeling and reconstruction of large-scale dynamic scenes for autonomous driving, offering both theoretical insights and practical implications for future developments in AI and autonomous systems.