Neural Dual Contouring (2202.01999v3)

Abstract: We introduce neural dual contouring (NDC), a new data-driven approach to mesh reconstruction based on dual contouring (DC). Like traditional DC, it produces exactly one vertex per grid cell and one quad for each grid edge intersection, a natural and efficient structure for reproducing sharp features. However, rather than computing vertex locations and edge crossings with hand-crafted functions that depend directly on difficult-to-obtain surface gradients, NDC uses a neural network to predict them. As a result, NDC can be trained to produce meshes from signed or unsigned distance fields, binary voxel grids, or point clouds (with or without normals); and it can produce open surfaces in cases where the input represents a sheet or partial surface. During experiments with five prominent datasets, we find that NDC, when trained on one of the datasets, generalizes well to the others. Furthermore, NDC provides better surface reconstruction accuracy, feature preservation, output complexity, triangle quality, and inference time in comparison to previous learned (e.g., neural marching cubes, convolutional occupancy networks) and traditional (e.g., Poisson) methods. Code and data are available at https://github.com/czq142857/NDC.

• The paper introduces a method that combines dual contouring with neural networks to predict vertex positions and reconstruct meshes without relying on gradient information.
• It demonstrates enhanced mesh quality and real-time performance, outperforming traditional techniques like Marching Cubes and Neural Marching Cubes.
• The method effectively handles diverse input types, including signed/unsigned distance fields, voxel grids, and point clouds, broadening its applications in 3D modeling and VR.

Neural Dual Contouring (NDC): A Different Angle on Mesh Reconstruction

A New Approach to Mesh Generation

Mesh reconstruction, the process of creating a polygonal mesh from discrete signals (like voxel grids or point clouds), has been examined for decades in computer graphics. This time, we'll explore a method known as Neural Dual Contouring (NDC), which offers innovative solutions to some persistent problems in the field.

NDC integrates deep learning with dual contouring (DC), a technique recognized for efficiently reconstructing sharp features in meshes. By leveraging convolutional neural networks (CNNs), NDC can adapt to various input types without the need for gradients—a significant departure from traditional DC. Let's unpack how this works and why it matters.

Key Components: How NDC Works

Bridging the Old and the New

Traditional Dual Contouring (DC) works on signed distance functions (SDFs) to produce mesh vertices and quads. However, DC requires accurate gradient information, which is often hard to obtain. NDC sidesteps this by training neural networks to predict vertex locations and edge crossings. This allows it to handle several input forms:

• Signed or Unsigned Distance Fields (SDF/UDF): A grid representation of distances to the nearest surface.
• Binary Voxel Grids: A simple grid where each cell is marked as inside or outside the object.
• Point Clouds: Sets of points sampled from an object's surface, with or without normals.

Addressing Input Variety

NDC operates in two main modes:

1. NDC: Takes SDF as input, predicting vertex positions and cell corner signs to construct a closed mesh.
2. UNDC: An unsigned version that handles UDFs or point clouds, predicting vertex positions and edge crossings, thus able to manage open surfaces and thin features.

Evaluation and Performance

The effectiveness of NDC was rigorously tested on several datasets, including the ABC dataset of CAD models, Thingi10k, FAUST, MGN, and Matterport3D. Key results and observations include:

• Mesh Quality: NDC and UNDC excel in preserving sharp features and produce meshes with better quality than traditional methods like Marching Cubes (MC33) and advanced methods like Neural Marching Cubes (NMC).
• Efficiency: NDC and UNDC are significantly faster than NMC variants, thanks to their simpler tessellation scheme. They can achieve real-time performance on typical hardware like NVIDIA GTX 1080ti GPUs.
• Versatility: UNDC stands out by reconstructing thin sheets and open surfaces from noisy or partial data, an advantage over conventional methods which usually output watertight meshes.

Implications and Future Directions

Practical Impact

For practitioners, NDC offers a tool that requires less manual tweaking and can handle a broader range of input data types. This makes it highly adaptable for applications in fields like 3D modeling, medical imaging, and virtual reality, where varied and sometimes incomplete data are the norms.

Theoretical Contributions

From a theoretical perspective, NDC provides insights into how neural networks can be married with classical meshing techniques to overcome the limitations of both. By predicting geometric properties directly, NDC opens pathways for further research into data-driven mesh generation without demanding precise gradient information.

What’s Next?

Looking ahead, several promising avenues for development could be explored:

1. Robustness to Noisy Data: Enhancing the network's resilience to even noisier inputs or creating more sophisticated noise-handling algorithms.
2. Integration with Neural Radiance Fields: Potentially combining NDC with Neural Radiance Fields (NeRF) could enable more accurate and flexible surface reconstructions from image data.
3. Expanding on Differentiability: Integrating NDC with differentiable rendering techniques to optimize 3D reconstructions from sparse sets of 2D images.

Conclusion

Neural Dual Contouring represents a noteworthy evolution in mesh reconstruction, combining data-driven approaches with the structured elegance of dual contouring. Its ability to generalize across diverse datasets and maintain high-quality mesh outputs, all while operating efficiently, marks it as a valuable addition to the set of tools available to data scientists and graphic artists alike. This balance between maintaining sharp feature preservation and broad input compatibility makes NDC a compelling choice for the future of 3D shape modeling.