MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenization (2408.02555v3)

Abstract: Meshes are the de facto 3D representation in the industry but are labor-intensive to produce. Recently, a line of research has focused on autoregressively generating meshes. This approach processes meshes into a sequence composed of vertices and then generates them vertex by vertex, similar to how a LLM generates text. These methods have achieved some success but still struggle to generate complex meshes. One primary reason for this limitation is their inefficient tokenization methods. To address this issue, we introduce MeshAnything V2, an advanced mesh generation model designed to create Artist-Created Meshes that align precisely with specified shapes. A key innovation behind MeshAnything V2 is our novel Adjacent Mesh Tokenization (AMT) method. Unlike traditional approaches that represent each face using three vertices, AMT optimizes this by employing a single vertex wherever feasible, effectively reducing the token sequence length by about half on average. This not only streamlines the tokenization process but also results in more compact and well-structured sequences, enhancing the efficiency of mesh generation. With these improvements, MeshAnything V2 effectively doubles the face limit compared to previous models, delivering superior performance without increasing computational costs. We will make our code and models publicly available. Project Page: https://buaacyw.github.io/meshanything-v2/

• The paper introduces Adjacent Mesh Tokenization (AMT), which reduces token sequence length and computational load in 3D mesh generation.
• It integrates AMT into MeshAnything V2, doubling the supported mesh face count from 800 to 1600 while maintaining high performance.
• Experimental results demonstrate significant improvements in key metrics such as Chamfer Distance, Edge Chamfer Distance, and Normal Consistency.

MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenization

The paper, "MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenization," introduces significant advancements in the automatic generation of Artist-Created Meshes (AM) using a novel approach dubbed Adjacent Mesh Tokenization (AMT). This method addresses some of the intrinsic inefficiencies observed in previous works by offering a more compact representation of 3D mesh data.

Key Contributions

The primary contributions of this research are multi-fold:

1. Adjunct Mesh Tokenization (AMT): The authors propose a groundbreaking approach for mesh tokenization. Unlike the traditional methods that use three vertices to represent a face, AMT utilizes a single vertex whenever feasible. This reduction in token sequence length results in significant efficiency gains.
2. MeshAnything V2: This new iteration of MeshAnything leverages AMT, allowing the model to handle meshes with up to 1600 faces, a doubling from the previous limit of 800 faces, without compromising on accuracy or efficiency.
3. Experimental Validation: Extensive experiments validate that AMT significantly improves computational efficiency and mesh generation performance.

Theoretical and Practical Implications

Adjacent Mesh Tokenization

AMT innovates by significantly reducing the redundancy in mesh representation. Traditional methods tokenize each face by its three vertices, resulting in a sequence length that scales thrice with the number of faces. In contrast, AMT drastically reduces this by encoding adjacent faces using only one new vertex, leading to a sequence length reduction to nearly one-third in ideal cases. This token sequence is not only shorter but also better structured, which enhances sequence learning.

MeshAnything V2 Integration

MeshAnything V2 integrates AMT to improve both the performance and efficiency of AM generation. By utilizing a point cloud encoder for shape conditions, the transformer-based model is trained to generate high-quality meshes. Notably, the maximum face count supported is scaled up to 1600, thereby broadening the scope of potential applications in 3D asset production pipelines.

Numerical Results

Token Sequence Efficiency

Experiments show that AMT reduces the token sequence length by approximately half on average. This reduction translates to a fourfold decrease in computational load and memory usage, significantly accelerating the training process and inference.

Quality Metrics

Quantitative metrics such as Chamfer Distance (CD), Edge Chamfer Distance (ECD), and Normal Consistency (NC) depict substantial improvements:

• Chamfer Distance (CD): 0.802 (with AMT) vs. 1.454 (without AMT)
• Edge Chamfer Distance (ECD): 4.587 (with AMT) vs. 5.867 (without AMT)
• Normal Consistency (NC): 0.935 (with AMT) vs. 0.913 (without AMT)

These metrics confirm that AMT not only enhances mesh generation efficiency but also preserves or improves quality.

Future Directions

While MeshAnything V2 marks a notable improvement, the authors acknowledge that the current accuracy still falls short for industrial deployment. Future research could delve into refining the algorithm’s stability and resolving any residual inaccuracies. Additionally, the integration of more sophisticated neural architectures and diverse datasets could further bolster model robustness and performance.

Conclusion

The MeshAnything V2 framework, underpinned by Adjacent Mesh Tokenization, represents a significant step forward in the domain of Artist-Created Mesh generation. By reducing the token sequence length dramatically, AMT addresses the computational inefficiencies that have historically plagued this field, enabling the generation of higher-quality meshes at a faster rate. This advancement opens new avenues for integrating AI-driven mesh generation into more complex and demanding 3D asset production environments.