SurfelMeshing: Online Surfel-Based Mesh Reconstruction (1810.00729v2)

Abstract: We address the problem of mesh reconstruction from live RGB-D video, assuming a calibrated camera and poses provided externally (e.g., by a SLAM system). In contrast to most existing approaches, we do not fuse depth measurements in a volume but in a dense surfel cloud. We asynchronously (re)triangulate the smoothed surfels to reconstruct a surface mesh. This novel approach enables to maintain a dense surface representation of the scene during SLAM which can quickly adapt to loop closures. This is possible by deforming the surfel cloud and asynchronously remeshing the surface where necessary. The surfel-based representation also naturally supports strongly varying scan resolution. In particular, it reconstructs colors at the input camera's resolution. Moreover, in contrast to many volumetric approaches, ours can reconstruct thin objects since objects do not need to enclose a volume. We demonstrate our approach in a number of experiments, showing that it produces reconstructions that are competitive with the state-of-the-art, and we discuss its advantages and limitations. The algorithm (excluding loop closure functionality) is available as open source at https://github.com/puzzlepaint/surfelmeshing .

• The paper introduces a novel surfel-based method for online mesh reconstruction that efficiently adapts to SLAM pose updates including loop closures.
• The method employs iterative remeshing and surfel denoising to produce smoother, more complete meshes, outperforming traditional volumetric techniques.
• The approach operates in real-time on typical GPU setups, making it ideal for applications in augmented reality and robotics requiring dynamic scene updates.

SurfelMeshing: Online Surfel-Based Mesh Reconstruction

The paper introduces SurfelMeshing, a novel approach for online mesh reconstruction from live RGB-D video utilizing surfel-based techniques. Unlike previous methods that rely heavily on volumetric integration, SurfelMeshing implements a dense surfel cloud for surface representation, enabling efficient adaptation to SLAM-derived pose updates such as loop closures.

Technical Approach

SurfelMeshing leverages surfel clouds to continuously update and triangulate surface meshes. A surfel, characterized by its position, normal, color, and confidence, supports high adaptability to varying scan resolutions, facilitating the accurate reconstruction of detailed textures. Upon capturing new frames, the system refines surfel positions through data association, with integration performed using weighted averages of observed data.

The method employs surfel denoising to mitigate noise from depth sensor inaccuracies and calibration errors. This involves regularization across transitive neighbors, ensuring smoothness in reconstructed surfaces, and blending at observation boundaries to prevent discontinuities. These techniques contribute significantly to the mesh quality, reducing holes and noise present in standalone surfel reconstructions.

For meshing, the method adapts techniques from fast point set triangulation and introduces an octree-based spatial structure to maintain efficient neighbor determination as surfels move. Through a process of iterative remeshing, SurfelMeshing efficiently updates only those regions of the mesh affected by changes, preserving computational efficiency.

Key Results and Evaluation

The paper presents extensive evaluations across various datasets, comparing SurfelMeshing to current state-of-the-art methods like ElasticFusion and volume-based approaches such as InfiniTAM. SurfelMeshing demonstrates superior adaptability and efficiency, particularly in scenarios involving loop closures, which present significant challenges for traditional volumetric methods.

• Accuracy and Completeness: SurfelMeshing exhibits competitive accuracy and higher completeness rates compared to ElasticFusion and volumetric methods on the ICL-NUIM dataset. The reconstructions are notably smoother due to the denoising processes.
• Mesh Quality: The approach achieves a higher quality of mesh with a lower mean curvature compared to volumetric counterparts, owing to its adaptive surfel resolution and effective handling of thin objects.
• Performance Efficiency: SurfelMeshing maintains high performance, operating in real-time without substantial computation delays, particularly during loop closure events. The integration of surfel updates runs efficiently on a typical GPU setup.

Implications and Future Directions

SurfelMeshing offers a flexible and scalable solution for real-time 3D reconstruction, supporting applications in areas like augmented reality and robotics that require continuous environmental interaction. The system's ability to handle complex changes in scene topology through non-rigid deformation opens avenues for further refining SLAM integration, reducing data redundancy, and enhancing surface detail preservation.

Future research directions may explore tighter integration between SLAM algorithms and surfel-based reconstruction for seamless data flow and higher precision. Improvements in depth sensor technology could further enhance the approach's robustness and accuracy.

In summary, SurfelMeshing presents a substantial advancement in online 3D reconstruction methodologies, effectively addressing key limitations of volumetric approaches while offering adaptability and efficiency in dynamic environments.