Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping (2306.01405v1)

Abstract: Learning signed distance functions (SDFs) from 3D point clouds is an important task in 3D computer vision. However, without ground truth signed distances, point normals or clean point clouds, current methods still struggle from learning SDFs from noisy point clouds. To overcome this challenge, we propose to learn SDFs via a noise to noise mapping, which does not require any clean point cloud or ground truth supervision for training. Our novelty lies in the noise to noise mapping which can infer a highly accurate SDF of a single object or scene from its multiple or even single noisy point cloud observations. Our novel learning manner is supported by modern Lidar systems which capture multiple noisy observations per second. We achieve this by a novel loss which enables statistical reasoning on point clouds and maintains geometric consistency although point clouds are irregular, unordered and have no point correspondence among noisy observations. Our evaluation under the widely used benchmarks demonstrates our superiority over the state-of-the-art methods in surface reconstruction, point cloud denoising and upsampling. Our code, data, and pre-trained models are available at https://github.com/mabaorui/Noise2NoiseMapping/

Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping

This paper addresses the challenging task of learning Signed Distance Functions (SDFs) from noisy 3D point clouds in computer vision. The authors present a novel approach that leverages noise to noise mapping, enabling the inference of accurate SDFs without the need for ground truth supervision, clean point clouds, or point normals. This method is particularly relevant given the prevalence of noisy observations in data captured by modern LiDAR systems, which typically generate between 10 to 30 noisy samples per second.

The primary innovation lies in the introduction of a novel loss function that incorporates statistical reasoning and geometric consistency, making it suitable for irregular and unordered point clouds where no point correspondence exists among noisy observations. The methodology leverages Earth Mover’s Distance (EMD) as a pivotal component in performing statistical reasoning, enabling the network to extract the correct geometrical structure from noise-laden data.

Numerical Results and Claims:

The paper asserts strong performance on a variety of benchmarks across multiple applications, including surface reconstruction, point cloud denoising, and upsampling. The results indicate that the proposed method significantly outperforms the current state-of-the-art techniques. For instance, in point cloud denoising tasks, the approach surpasses both traditional and contemporary deep learning-based methods in reducing L2 Chamfer Distance (L2CD) and point-to-mesh (P2M) error metrics. The evaluations in surface reconstruction reveal superior accuracy, as measured by L1 Chamfer Distance (L1CD), Normal Consistency (NC), and F-score, when applied to the ShapeNet dataset.

Methodological Contributions:

1. Noise to Noise Mapping: The method circumvents the need for clean ground truth by directly learning mappings between noisy point clouds, effectively allowing the model to discern the latent, true geometry inherent in the data.
2. Distance Metric and Geometric Regularization: By using EMD, the approach ensures one-to-one correspondence in the statistical evaluation of point clouds, aligning noisy inputs to latent structures with high fidelity. Geometric consistency is enforced through a novel regularization term, ensuring that the learned SDFs are robust and accurate.
3. Broad Applicability: The approach handles a variety of noise distributions and is applicable to both synthetic and real-world datasets, showcasing its versatility in different operational conditions like those found in automotive lidar scans.

Implications and Future Work:

The implications of this research are notable in fields where accurate 3D modeling from noisy data is crucial, such as autonomous driving, augmented reality, and digital fabrication. By reducing reliance on clean datasets, the methodology potentially simplifies the data acquisition process and broadens the applicability of 3D modeling technologies.

Future developments could explore extending this framework to other forms of 3D implicit functions beyond SDFs, potentially incorporating dynamic scenes where temporal coherence can be exploited. Additionally, integration with differentiable rendering techniques might facilitate seamless utilization of SDFs in end-to-end pipelines for rendering and real-time applications.

Overall, the paper contributes significant novel insights and establishes a foundation for efficient learning of 3D geometries from imperfect data, promoting further research and development in this domain.