GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering (2403.11324v2)

Abstract: During the Gaussian Splatting optimization process, the scene's geometry can gradually deteriorate if its structure is not deliberately preserved, especially in non-textured regions such as walls, ceilings, and furniture surfaces. This degradation significantly affects the rendering quality of novel views that deviate significantly from the viewpoints in the training data. To mitigate this issue, we propose a novel approach called GeoGaussian. Based on the smoothly connected areas observed from point clouds, this method introduces a novel pipeline to initialize thin Gaussians aligned with the surfaces, where the characteristic can be transferred to new generations through a carefully designed densification strategy. Finally, the pipeline ensures that the scene's geometry and texture are maintained through constrained optimization processes with explicit geometry constraints. Benefiting from the proposed architecture, the generative ability of 3D Gaussians is enhanced, especially in structured regions. Our proposed pipeline achieves state-of-the-art performance in novel view synthesis and geometric reconstruction, as evaluated qualitatively and quantitatively on public datasets.

• The paper introduces GeoGaussian, which integrates explicit geometry constraints into Gaussian splatting to enhance rendering quality and preserve structural integrity.
• The paper employs a twofold strategy using initialization with thin ellipsoids and densification to ensure co-planarity and accurate scene modeling.
• The paper demonstrates superior performance in novel view synthesis and geometric reconstruction on public datasets compared to traditional methods.

GeoGaussian: Enhanced Scene Rendering through Geometry-aware Gaussian Splatting

Introduction

Recent advancements in Neural Radiance Fields (NeRF) have significantly boosted photo-realistic novel view synthesis (NVS), attracting a surge of interest from computer vision, graphics, and robotics communities. Though NeRF offers impressive rendering quality, the emergence of 3D Gaussian Splatting has shown promising avenues, especially in terms of training speed and rendering quality. A pivotal feature of Gaussian Splatting is its parametric representation that encompasses position, orientation, and appearance parameters, subsequently re-projected to training images through alpha-blending. Despite its advantages, the lack of explicit geometric constraints in Gaussian Splatting often leads to subpar geometry preservation, markedly in non-textured regions, thus, compromising rendering quality of novel views significantly deviating from training perspectives.

Methodology

To address the aforementioned limitations, we introduce GeoGaussian, a novel adaptation of Gaussian Splatting, primarily focused on preserving scene geometry. GeoGaussian employs a twofold strategy: Initialization and Densification, underpinned by explicit geometry constraints to preserve scene structure and enhance rendering fidelity.

• Initialization: GeoGaussian employs a unique strategy for initializing Gaussians, especially in smoothly connected areas. It differentiates between general regions and smooth surfaces, parameterizing each point in smooth regions as a thin ellipsoid harboring explicit geometric information. The scale vector and rotation matrix are initialized to reflect the surface geometry accurately.
• Densification: The proposed method further embarks on a densification strategy that ensures geometrically consistent generation of new Gaussians. Here, the process encourages the co-planarity of newly formed Gaussians with the original surfaces, aligning new generations within the same geometric plane.
• Optimization: GeoGaussian integrates a novel set of constraints focusing on geometric consistency alongside traditional photometric losses. This amalgamation allows for the optimized balance between appearance fidelity and geometric accuracy, ensuring harmonized learning from varying perspectives.

Experimental Evaluation

The model's efficacy is substantiated through qualitative and quantitative benchmarks on public datasets, showcasing superiority over existing Gaussian Splatting methods particularly in geometric reconstruction and novel view synthesis. The proposed pipeline achieved significant improvements in preserving structure in non-textured regions, thereby, resolving blurring issues prevalent in other methods.

• Geometric Reconstruction: GeoGaussian's ability to maintain scene geometry was evaluated against state-of-the-art methods, demonstrating superior performance. Moreover, its effectiveness in handling non-textured regions significantly contributed to enhanced rendering quality.
• Novel View Synthesis: Comparative analysis on public datasets revealed GeoGaussian's proficiency in novel view rendering. Especially in structured environments, the method exhibited notable improvements across various metrics, consolidating its adaptability and robustness in rendering tasks.

Conclusion and Outlook

GeoGaussian represents a significant stride towards achieving geometrically consistent scene rendering in Gaussian Splatting frameworks. By integrating geometry-aware strategies, the method not only boosts the rendering fidelity but also markedly improves the structural integrity of 3D models. This approach unlocks new potentials for applications requiring real-time rendering and high-quality visualization.

The benefits of GeoGaussian underscore the importance of incorporating explicit geometry constraints in Gaussian Splatting, opening pathways for future research in optimizing geometric fidelity further. Exploring comprehensive solutions for improving Gaussian models through additional data inputs like depth and normals represents a promising avenue for enhancing rendering quality and model generalizability further.