MetaSDF: Meta-learning Signed Distance Functions (2006.09662v1)

Abstract: Neural implicit shape representations are an emerging paradigm that offers many potential benefits over conventional discrete representations, including memory efficiency at a high spatial resolution. Generalizing across shapes with such neural implicit representations amounts to learning priors over the respective function space and enables geometry reconstruction from partial or noisy observations. Existing generalization methods rely on conditioning a neural network on a low-dimensional latent code that is either regressed by an encoder or jointly optimized in the auto-decoder framework. Here, we formalize learning of a shape space as a meta-learning problem and leverage gradient-based meta-learning algorithms to solve this task. We demonstrate that this approach performs on par with auto-decoder based approaches while being an order of magnitude faster at test-time inference. We further demonstrate that the proposed gradient-based method outperforms encoder-decoder based methods that leverage pooling-based set encoders.

• The paper introduces a meta-learning framework for SDFs that achieves an order of magnitude faster inference compared to traditional auto-decoders.
• It reformulates neural implicit shape representation using gradient descent for rapid adaptation without requiring regular grid inputs.
• Experimental results on 2D and 3D benchmarks demonstrate competitive accuracy and potential for real-time 3D reconstruction applications.

MetaSDF: Meta-learning Signed Distance Functions

The paper, "MetaSDF: Meta-learning Signed Distance Functions," introduces a novel method for generalizing neural implicit representations of 3D shapes. Neural implicit shape representations, such as Signed Distance Functions (SDFs), are gaining traction due to their high memory efficiency and capability to reconstruct geometry from incomplete or noisy data. Traditional methods in this domain often rely on either encoder-based or auto-decoder approaches that are conditioned on low-dimensional latent codes. In contrast, the proposed approach leverages meta-learning to efficiently learn a shape space. This method performs comparably to auto-decoders but with significantly faster inference times.

Core Contributions

The key contribution of MetaSDF is its formulation as a meta-learning problem, specifically through gradient-based meta-learning techniques, which provide several advantages over existing strategies:

• Efficiency: At test time, the proposed method exhibits inference speeds that are an order of magnitude faster than auto-decoder models.
• Independence from Regular Grids: The method does not require inputs to be on a regular grid, avoiding the limitations of convolutional encoders.
• Adaptability: The model naturally accommodates varying numbers of observations and performs well even when conditioned only on zero-level set points.
• Dimensional Assumptions: Unlike traditional methods that depend on a low-dimensional latent space assumption, MetaSDF operates directly in the high-dimensional parameter space of the network.

Methodological Insights

MetaSDF reframes the task of learning neural implicit function spaces into a meta-learning framework. Each shape representation task involves finding an instance-specific signed distance function, which is accomplished by quickly adapting a meta-learned model using 'context' observations (e.g., a set of 3D points and their respective distances). The method of model adaptation utilizes gradient descent, iteratively updating its parameters to refine the shape-specific instantiation.

The architecture utilizes a multi-layer perceptron (MLP) to approximate the signed distance function, ensuring that the zero-level sets of these functions accurately represent the surfaces of 3D objects. Additionally, the introduction of a composite multi-task loss function aids in stabilizing training and improving network convergence.

Experimental Evaluation

The experiments conducted demonstrate the efficacy of MetaSDF across several domains:

• 2D Representations: Utilizing MNIST digit data, the approach excelled at reconstructing 2D SDFs from both dense observations and points confined to zero-level sets, outperforming state-of-the-art encoder methods in metric-based evaluations.
• 3D Shape Generalization: In tests using the ShapeNet dataset, MetaSDF maintained competitive quantitative accuracy with existing methods while reducing inference time significantly. Furthermore, it performed robustly even when tasked with uncommon or out-of-distribution shape configurations.
• Interpretation as Unsupervised Representation Learning: The resulting parameters of network instances can encode substantial information about object class, suggesting potential for unsupervised learning applications in other domains.

Implications and Future Directions

The findings indicate MetaSDF's potential as a robust tool for real-time applications requiring fast and versatile 3D object reconstruction, such as autonomous driving and robotic perception. Future explorations might focus on reducing memory complexity using advanced gradient methods or extending the approach towards holistic scene representations involving both shape and appearance using neural rendering techniques.

In conclusion, the work introduces a promising direction in efficiently generalizing neural implicit shape representations, facilitated by meta-learning, that could significantly streamline complex 3D shape reconstruction tasks across various industries.