SAL: Sign Agnostic Learning of Shapes from Raw Data (1911.10414v2)

Abstract: Recently, neural networks have been used as implicit representations for surface reconstruction, modelling, learning, and generation. So far, training neural networks to be implicit representations of surfaces required training data sampled from a ground-truth signed implicit functions such as signed distance or occupancy functions, which are notoriously hard to compute. In this paper we introduce Sign Agnostic Learning (SAL), a deep learning approach for learning implicit shape representations directly from raw, unsigned geometric data, such as point clouds and triangle soups. We have tested SAL on the challenging problem of surface reconstruction from an un-oriented point cloud, as well as end-to-end human shape space learning directly from raw scans dataset, and achieved state of the art reconstructions compared to current approaches. We believe SAL opens the door to many geometric deep learning applications with real-world data, alleviating the usual painstaking, often manual pre-process.

SAL: Sign Agnostic Learning of Shapes from Raw Data

The paper "SAL: Sign Agnostic Learning of Shapes from Raw Data" introduces a novel methodology in geometric deep learning for learning implicit shape representations from raw, unsigned geometric data. Unlike traditional methods that rely on signed data representations, SAL (Sign Agnostic Learning) operates effectively with unsigned input, such as un-oriented point clouds or triangle soups, alleviating the dependency on pre-computed signed distance functions.

Key Contributions

SAL proposes a deep learning framework characterized by a family of loss functions designed to work directly with raw geometric data. The framework aims to reconstruct, learn, and generate 3D shapes without requiring signed implicit functions from the data, thereby streamlining the preprocessing pipeline. This paradigm shift has practical implications in applications involving noisy or incomplete datasets, as found in real-world scenarios.

The primary theoretical contribution of this work is the introduction of a loss function that is sign agnostic. The loss is designed to encourage neural networks to learn the signed implicit surface representation directly from unsigned data. This is achieved by defining the loss function using an unsigned similarity metric that can handle the symmetrical nature of the data.

Numerical Results

Experimental results demonstrate the efficacy of SAL in scenarios requiring surface reconstruction from un-oriented point clouds and end-to-end human shape space learning using the D-Faust dataset. The SAL framework achieves state-of-the-art performance compared to conventional methods. Numerical experiments highlight SAL's ability to handle the challenging task of surface reconstruction, demonstrating high fidelity outputs even in cases of sparse and irregular point clouds.

Theoretical Implications

The paper provides a rigorous theoretical foundation underpinning SAL, including a proof of a plane reproduction property for the proposed loss functions. This guarantees that if the input data lies within a plane, the learned representation will accurately reconstruct this plane, an essential aspect given the piecewise planar nature of surfaces.

Moreover, the paper discusses a geometric initialization strategy for multi-layer perceptrons (MLPs) to approximate signed distance functions to spheres. This initialization is critical for favoring the correct local minima during optimization, guiding the network towards learning meaningful signed representations despite the agnostic nature of the input data.

Future Directions

This research offers several potential future directions. The adaptability of SAL suggests applications in generative models such as GANs, where inferring shape spaces from raw data could enhance diversity and realism. Furthermore, exploring the integration of SAL with multimodal data, such as combining raw geometric input with image data, may unlock further advancements in object recognition and reconstruction pipelines.

Overall, SAL presents a valuable contribution to the field of computer vision and pattern recognition, offering an effective tool for working with raw and noisy 3D data. The elimination of the reliance on signed pre-processing has the potential to greatly enhance the versatility and applicability of deep learning approaches in real-world geometric data processing.