Incremental Online Scene Reconstruction by 3D Gaussian Triangulation

Abstract: Incremental scene reconstruction is essential for real-world applications. Although 3D Gaussian Splatting shows strong potential, most existing approaches require offline conversion of the optimized Gaussians into an intermediate implicit field for explicit mesh extraction, which hinders seamless integration with downstream tasks. To address this limitation, we propose a novel online framework that incrementally reconstructs and updates high-fidelity explicit meshes by directly triangulating a dense geometric Gaussian representation, which supports both high-quality rendering and incremental surface reconstruction. Moreover, we present a direct meshing algorithm that efficiently extracts and updates the mesh from the Gaussian set. To ensure mesh accuracy, we enforce a plane-based pulling constraint that dynamically aligns 3D Gaussian primitives to the approximated local surface. Furthermore, our framework significantly reduces memory and computational overhead during long-sequence processing by dynamically freezing fully optimized historical regions. Experiments on public datasets demonstrate that our method outperforms conventional Gaussian-based methods on both rendering quality and reconstruction accuracy.

• The paper introduces a framework that incrementally integrates RGB-D data with direct Gaussian triangulation to produce watertight 3D meshes.
• It employs a dense geometric Gaussian representation optimized with photometric and depth constraints, ensuring high fidelity even in noisy environments.
• Experimental results show superior geometric accuracy and efficiency, with enhanced mapping speeds and lower memory usage compared to conventional methods.

Incremental Online Scene Reconstruction by 3D Gaussian Triangulation

Introduction

This paper presents a framework for incremental online 3D scene reconstruction using direct Gaussian triangulation. Traditional 3D reconstruction methods, whether explicit or implicit, face significant challenges when deployed in online or streaming scenarios. Conventional approaches either require full-scene optimization before mesh extraction or necessitate the construction of heavy global data structures, impeding their integration into memory- and computation-constrained environments. The introduced system incrementally integrates incoming RGB-D streams, optimizes a dense geometric Gaussian representation in an online fashion, and produces triangular meshes through a novel fast triangulation algorithm.

Figure 1: The proposed incremental framework enables progressive updates to both the Gaussian scene representation and mesh, unlike traditional batch pipelines that require global re-optimization for each new frame.

Dense Geometric Gaussian Representation

The foundation of the method is an explicit dense set of geometric surfel-like 3D Gaussians, initialized from oriented point clouds computed directly from RGB-D data. Each Gaussian encodes position, normal, local tangent orientation (enforced through a planar constraint), color, opacity, and scale. Contrary to standard radiance field-optimized Gaussians, these are strictly regularized to be planar and fully opaque ($\alpha \approx 1$).

A dual-branch rendering pipeline is used for optimizing both photometric (RGB) and geometric (depth) consistency. A set of geometric and photometric constraints (including an explicit plane-pulling loss and normal alignment) is applied iteratively to local windows of Gaussians. This ensures spatially accurate surface adherence and robustness against sensor noise and missing observations, compensating for practical imperfections in the RGB-D input.

Figure 2: The end-to-end architecture: oriented point clouds yield planar Gaussians, which undergo loss-driven densification and geometric constraint optimization. Final meshes are generated by selecting geometric Gaussians, pruning, and direct triangulation.

Incremental Meshing: Gaussian Triangulation

A core contribution is a direct triangulation method over the Gaussian set, which combines local neighbor identification, geometric filtering, and adaptive connectivity. For meshing, only a subset of highly accurate, fully opaque Gaussians is retained, enforcing both opacity and strict depth agreement w.r.t. the observed geometry.

The triangulation proceeds by projecting local neighbors onto the tangent plane of each Gaussian, pruning based on mutual visibility and normal consistency, fusing for local smoothing, and angularly sorting neighbors. Mesh connectivity is established incrementally with an angle-based greedy algorithm, and geometric quality controls (e.g., minimum angle) prevent degenerate triangles. This pipeline ensures watertight, locally-adapted meshes compatible with online scene growth.

Figure 3: The Gaussian triangulation pipeline: neighbor identification, geometric pruning, position refinement, angle-based ordering, and triangulation.

To address the dynamics of scene expansion and changes, a local remeshing strategy is embedded: edge splits/collapses, flips, and Laplacian smoothing guarantee seamless mesh merging across incrementally processed regions. Crucially, "freezing" occurs whereby fully optimized regions (gauged by multi-view observation count and convergence) are permanently excluded from further optimization, capping both memory overhead and compute time per frame.

Experimental Results

Mesh Quality and Rendering Fidelity

The method is evaluated on both the Replica and ScanNet++ datasets. Compared to explicit (KinectFusion), neural implicit (NICE-SLAM), and Gaussian-based (MonoGS, RTG-SLAM) baselines, the proposed framework achieves superior geometric accuracy (average mesh error of 1.34 cm on Replica, outperforming all published alternatives) and mesh completeness. It also achieves state-of-the-art novel-view rendering—demonstrating PSNR up to 42.65 and SSIM of 0.99—while preserving surface details and suppressing artifacts.

Figure 4: Qualitative mesh reconstruction comparison on Replica; the proposed method recovers finer details and smoother surfaces than alternatives.

Figure 5: Direct comparison to MonoGS on Replica, highlighting sharper geometry and reduced artifacts.

Figure 6: Influence of geometric constraints on reconstruction: omitting plane- and normal-based losses leads to rough, incomplete, and noisy surfaces.

Efficiency and Scalability

The incremental design coupled with freezing yields significant improvements in computational efficiency and memory consumption. Mapping speeds surpass MonoGS and RTG-SLAM by a 2×–7× margin, with peak memory reduced to less than 2.5 GB on challenging sequences. Mesh extraction is 80×–100× faster than global implicit methods: complete mesh updates take ≈5s versus ≈400–500s for alternatives that must (re-)process global fields.

Ablation Studies

Critical components are validated through systematic removal and analysis. Disabling direct triangulation or geometric constraints degrades accuracy and increases surface roughness, while omission of remeshing or freezing impairs mesh quality and system stability for long sequences. The explicit combination of all proposed losses and triangulation modules is required for optimal detail recovery and artifact suppression.

Parameter sweeps on the mesh candidate selection threshold $\tau$ illustrate the trade-off between mesh completeness and precision, informing hyperparameter choices for practical deployments.

Figure 7: Reconstruction performance across different values of the candidate selection threshold $\tau$; larger $\tau$ increases completion but deteriorates precision.

Discussion and Future Directions

This framework demonstrates that treating dense, planar, geometric Gaussians as primary scene elements—with direct, constraint-driven surfel-to-mesh triangulation—enables scalable, online, and memory-bounded 3D reconstruction. Memory and compute costs scale with currently active scene regions, enabling application to extended or large-scale datasets without the prohibitive growth typical of global approaches.

One explicit limitation is the dependence on dense, accurate depth input. Incomplete, low-quality, or missing geometric cues lead to loss of surface fidelity in unobserved regions, as the method does not hallucinate or extrapolate unseen structure. Extending this framework to RGB-only or incomplete-sensor input involves open questions around uncertainty-aware surfel estimation and learned priors for unobserved geometry. Further, generalizing the freezing and region selection regime could push efficiency for unbounded scene capture in dense SLAM or AR/VR settings.

Conclusion

The paper introduces a practical and technically rigorous framework for incremental online scene reconstruction through 3D Gaussian triangulation, circumventing the memory and recomputation bottlenecks of contemporary approaches. The unified treatment of surfel-based dense Gaussians, direct constraint-driven meshing, and progressive resource freezing achieves new best-in-class performance in accuracy, rendering quality, and computational efficiency. The design paradigm and algorithmic contributions are widely applicable for scalable volumetric mapping and online 3D scene understanding (2607.10690).