Geometry-Consistent Neural Shape Representation with Implicit Displacement Fields (2106.05187v3)

Abstract: We present implicit displacement fields, a novel representation for detailed 3D geometry. Inspired by a classic surface deformation technique, displacement mapping, our method represents a complex surface as a smooth base surface plus a displacement along the base's normal directions, resulting in a frequency-based shape decomposition, where the high frequency signal is constrained geometrically by the low frequency signal. Importantly, this disentanglement is unsupervised thanks to a tailored architectural design that has an innate frequency hierarchy by construction. We explore implicit displacement field surface reconstruction and detail transfer and demonstrate superior representational power, training stability and generalizability.

• The paper presents a novel implicit mapping framework that decomposes 3D shapes into a low-frequency base surface and high-frequency displacement fields.
• It leverages a frequency hierarchy using siren networks to capture detailed geometry with improved reconstruction fidelity and training stability.
• The approach offers transferable, unsupervised geometry representation, enabling robust applications in 3D modeling, virtual reality, and graphics.

Geometry-Consistent Neural Shape Representation with Implicit Displacement Fields

The paper "Geometry-Consistent Neural Shape Representation with Implicit Displacement Fields," authored by Yifan Wang, Lukas Rahmann, and Olga Sorkine-Hornung from ETH Zurich, explores a new method for representing detailed 3D geometry. The proposed representation, termed Implicit Displacement Fields (IDF), is a novel approach for reconstructing complex geometry by decomposing a shape into a smooth base surface and high-frequency details that are expressed as displacements along the base's normal directions. This decomposition is driven by a frequency-based shape representation that integrates smoothly into neural implicit modeling.

Summary of the Method

Inspired by the traditional displacement mapping technique, the authors present IDF as a continuous function over the entire 3D domain, rather than confining it to the surface. The methodology involves separating a detailed surface into a low-frequency base representation and a high-frequency displacement field. This separation allows for an unsupervised disentanglement of geometry as the base surface is reconstructed via low-frequency components, while finer details are handled using higher-frequency signals. IDF capitalizes on siren neural networks, with different frequency parameters, to achieve this aim, creating a frequency hierarchy that mimics the inductive bias necessary for effective geometric representation.

Architectural and Theoretical Contributions

The paper introduces several key technical contributions:

1. Implicit Mapping Framework: The extension of the discrete displacement mapping into a continuous implicit mapping setup provides a theoretically grounded alternative that enhances neural shape representation.
2. Frequency Hierarchy: By exploiting the inherent biases of siren networks, the researchers develop a neural architecture capable of distinguishing between low and high-frequency signals for geometric interpretation.
3. Transferable Framework: The proposed methodology also supports transferability by replacing coordinate-based inputs with transferrable feature-based learning, facilitating its application in implicit geometry manipulation and shape modeling tasks.

Results and Analysis

The paper demonstrates, through comprehensive experiments, that the IDF outperforms existing methods in terms of reconstruction fidelity, stability during training, and computational efficiency. Compared to other approaches like NGLOD and different configurations of sirens, the IDF shows significantly enhanced capabilities in handling high-frequency geometric details with fewer model parameters. Furthermore, the paper addresses the challenges associated with training high-frequency networks by implementing a progressive learning paradigm, which mitigates optimization framing issues.

Implications and Future Work

The IDF framework paves the way for more efficient shape representation methodologies, potentially impacting diverse fields such as 3D modeling, virtual reality, and computational graphics. The unsupervised nature of its frequency decomposition allows for flexible adaptation to new tasks without extensive manual parameter tuning. Importantly, the ability to transfer geometrical details across different shapes through transferrable implicit mappings suggests a promising avenue for development in automated geometry synthesis and manipulation. Future work may involve improving the rotational invariance of the feature descriptors or expanding the methods to handle varying alignment scenarios without prior shape adjustment.

The proposed method is set to make significant contributions toward advancing neural shape representations, offering a richly detailed, efficient, and flexible solution to modern modeling challenges.