PSDF: Prior-Driven Neural Implicit Surface Learning for Multi-view Reconstruction

Abstract: Surface reconstruction has traditionally relied on the Multi-View Stereo (MVS)-based pipeline, which often suffers from noisy and incomplete geometry. This is due to that although MVS has been proven to be an effective way to recover the geometry of the scenes, especially for locally detailed areas with rich textures, it struggles to deal with areas with low texture and large variations of illumination where the photometric consistency is unreliable. Recently, Neural Implicit Surface Reconstruction (NISR) combines surface rendering and volume rendering techniques and bypasses the MVS as an intermediate step, which has emerged as a promising alternative to overcome the limitations of traditional pipelines. While NISR has shown impressive results on simple scenes, it remains challenging to recover delicate geometry from uncontrolled real-world scenes which is caused by its underconstrained optimization. To this end, the framework PSDF is proposed which resorts to external geometric priors from a pretrained MVS network and internal geometric priors inherent in the NISR model to facilitate high-quality neural implicit surface learning. Specifically, the visibility-aware feature consistency loss and depth prior-assisted sampling based on external geometric priors are introduced. These proposals provide powerfully geometric consistency constraints and aid in locating surface intersection points, thereby significantly improving the accuracy and delicate reconstruction of NISR. Meanwhile, the internal prior-guided importance rendering is presented to enhance the fidelity of the reconstructed surface mesh by mitigating the biased rendering issue in NISR. Extensive experiments on the Tanks and Temples dataset show that PSDF achieves state-of-the-art performance on complex uncontrolled scenes.

• The paper proposes PSDF, a framework that integrates external and internal priors to enhance neural implicit surface reconstruction.
• It introduces a visibility-aware consistency loss and depth-informed sampling to accurately pinpoint surface intersections.
• Empirical tests on Tanks and Temples and DTU datasets demonstrate significant improvements in reconstruction accuracy.

Introduction

The development of Neural Implicit Surface Reconstruction (NISR) techniques has been a significant advancement in addressing challenges faced by Multi-View Stereo (MVS)-based surface reconstruction pipelines. Traditional MVS approaches struggle with areas of low texture and varying illumination, leading to noisy and incomplete geometry. Recent NISR methods show promise by directly reconstructing surface geometry through a combination of differentiable surface and volume rendering techniques. However, these methods typically fall short when dealing with complex, real-world scenes due to underconstrained optimization that focuses on global structure at the expense of fine detail.

The PSDF Framework

A newly proposed framework, PSDF, seeks to refine neural implicit surface learning by incorporating both external geometric priors from a pretrained MVS network and internal geometric priors inherent within the NISR model. The framework introduces a visibility-aware feature consistency loss and a depth prior-assisted sampling methodology. These additions offer robust geometric consistency and aid in accurately locating surface intersection points.

To combat biased rendering issues inherent in volume rendering, PSDF deploys an internal prior-guided importance rendering strategy. By harnessing densely distributed near-surface points, PSDF can direct the rendering process toward points that confer unbiased rendering effects.

Empirical Evaluation

The effectiveness of PSDF is quantitatively validated on the Tanks and Temples dataset, achieving state-of-the-art performance. Notable improvements are reported compared with other NISR methods like VolSDF, MonoSDF, Geo-Neus, and Neus, with substantial percentage gains in reconstruction accuracy. These experiments emphasize PSDF's capability to reconstruct complex scenes with high fidelity and detail. Moreover, the DTU dataset benchmarking depicts that PSDF is capable of handling detailed object-centric scenes, second only to Geo-Neus in terms of Chamfer Distance.

Contributions

PSDF marks an essential progression in the sphere of surface reconstruction by making several seminal contributions. It elevates the optimization of the geometric field, provides robust geometric consistency constraints, and enhances the fidelity of reconstructed surfaces. These advancements are established through a meticulous integration of external priors for efficient sample generation and internal priors for importance rendering.

Conclusion

The paper posits that the prior-driven neural implicit surface learning framework, PSDF, sets a new benchmark for multi-view reconstruction, especially in complex and dynamic real-world scenes. By fully exploiting external and internal geometric priors, PSDF achieves high-quality surface reconstruction. Future work may explore expediting the training process and overcoming the current limitations associated with the learning of thin structures within scenes.