BSP-Net: Generating Compact Meshes via Binary Space Partitioning (1911.06971v6)

Abstract: Polygonal meshes are ubiquitous in the digital 3D domain, yet they have only played a minor role in the deep learning revolution. Leading methods for learning generative models of shapes rely on implicit functions, and generate meshes only after expensive iso-surfacing routines. To overcome these challenges, we are inspired by a classical spatial data structure from computer graphics, Binary Space Partitioning (BSP), to facilitate 3D learning. The core ingredient of BSP is an operation for recursive subdivision of space to obtain convex sets. By exploiting this property, we devise BSP-Net, a network that learns to represent a 3D shape via convex decomposition. Importantly, BSP-Net is unsupervised since no convex shape decompositions are needed for training. The network is trained to reconstruct a shape using a set of convexes obtained from a BSP-tree built on a set of planes. The convexes inferred by BSP-Net can be easily extracted to form a polygon mesh, without any need for iso-surfacing. The generated meshes are compact (i.e., low-poly) and well suited to represent sharp geometry; they are guaranteed to be watertight and can be easily parameterized. We also show that the reconstruction quality by BSP-Net is competitive with state-of-the-art methods while using much fewer primitives. Code is available at https://github.com/czq142857/BSP-NET-original.

• The paper introduces BSP-Net, a novel architecture that directly outputs compact, watertight polygonal meshes without needing extensive post-processing.
• It segments 3D shapes into convex primitives using an unsupervised BSP-tree framework, significantly reducing computational complexity.
• Experimental results demonstrate competitive performance in shape auto-encoding and reconstruction by achieving sharper features with fewer primitives.

Overview of BSP-Net: Generating Compact Meshes via Binary Space Partitioning

The paper "BSP-Net: Generating Compact Meshes via Binary Space Partitioning" introduces a novel approach to generating polygonal meshes, a critical representation in 3D digital modeling. This method, termed BSP-Net, leverages the Binary Space Partitioning (BSP) framework to address the inefficiencies in existing 3D shape representations and mesh generation techniques used in deep learning applications.

Motivation and Key Contributions

The authors identify a gap in how polygonal meshes have been underutilized in deep learning, compared to alternatives like voxel grids, point clouds, and implicit functions. The traditional methods often require computationally expensive post-processing steps, such as iso-surfacing, to convert these representations into meshes. BSP-Net addresses this by enabling the direct output of compact, watertight, and parameterizable polygonal meshes.

Key contributions of the paper include:

1. BSP-Net Architecture: This network is the first of its kind to seamlessly generate polygonal meshes. It learns to segment 3D shapes into a collection of convex primitives (or parts) that can be directly assembled into a polygonal mesh. This is achieved without ground truth convex decompositions, making the network unsupervised.
2. Generative Capacity: BSP-Net is designed to handle arbitrary shape topologies and structural variety, emphasizing the ability to produce sharp geometric features while maintaining compactness.
3. Implications for 3D Representation: The architecture allows for implicit field learning that informs whether a point is inside or outside a shape based on learned hyper-parameters of a collection of planes. This approach bypasses the computational intensity of traditional mesh conversion processes.
4. Applications and Adaptability: The network is demonstrated to be versatile, applicable in tasks such as shape auto-encoding, single-view reconstruction, and structured 3D modeling from single images. It makes strides in achieving structured SVR, where a model reconstructs and segments a 3D shape from an unstructured image.

Methodological Insights

BSP-Net operates by inferring a BSP-tree from a dataset of planes, learning convex partitions which aggregate to form complete shapes. This aspect facilitates the creation of watertight meshes which encompass both sharp and smooth features with significantly fewer primitives compared to state-of-the-art methods. The architecture is composed of layers that aggregate hyper-planes to form shape components, which are then merged into a complete representation.

The paper also introduces an Edge Chamfer Distance (ECD) for evaluating the ability of models to represent sharp edges—a notable contribution to the evaluation of 3D reconstruction quality.

Experimental Results

The experiments conducted on BSP-Net demonstrate competitive performance relative to existing methods, achieving superior results in auto-encoding, segmentation, and reconstruction quality while maintaining computational efficiency. Particularly notable are the reconstruction results indicating a favorable trade-off between fidelity and complexity, with significantly reduced mesh size.

Theoretical and Practical Implications

BSP-Net's approach suggests a shift towards more efficient, integrated 3D modeling processes in machine learning, which could influence how generative models handle complex shape and scene representations. The capacity for unsupervised mesh generation underlines a potential future where neural networks autonomously learn and adapt to the intricacies of real-world geometries.

Future Directions

Challenges remain in extending the approach to more complex shapes that require representation as differences rather than unions of convex parts—a task not immediately feasible with the current network formulation. Furthermore, while current training times are significant, adaptive dynamic primitive allocation based on shape complexity could enhance efficiency.

The paper signifies a step forward in structured 3D representation learning, with implications for future developments in computer vision and graphics, potentially improving the integration and utility of machine learning in industry-standard modeling workflows.