Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GenUDC: High Quality 3D Mesh Generation with Unsigned Dual Contouring Representation (2410.17802v1)

Published 23 Oct 2024 in cs.CV and cs.GR

Abstract: Generating high-quality meshes with complex structures and realistic surfaces is the primary goal of 3D generative models. Existing methods typically employ sequence data or deformable tetrahedral grids for mesh generation. However, sequence-based methods have difficulty producing complex structures with many faces due to memory limits. The deformable tetrahedral grid-based method MeshDiffusion fails to recover realistic surfaces due to the inherent ambiguity in deformable grids. We propose the GenUDC framework to address these challenges by leveraging the Unsigned Dual Contouring (UDC) as the mesh representation. UDC discretizes a mesh in a regular grid and divides it into the face and vertex parts, recovering both complex structures and fine details. As a result, the one-to-one mapping between UDC and mesh resolves the ambiguity problem. In addition, GenUDC adopts a two-stage, coarse-to-fine generative process for 3D mesh generation. It first generates the face part as a rough shape and then the vertex part to craft a detailed shape. Extensive evaluations demonstrate the superiority of UDC as a mesh representation and the favorable performance of GenUDC in mesh generation. The code and trained models are available at https://github.com/TrepangCat/GenUDC.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (78)
  1. Learning representations and generative models for 3D point clouds. In ICML.
  2. PolyDiff: Generating 3D Polygonal Meshes with Diffusion Models. arXiv preprint arXiv:2312.11417 (2023).
  3. Mohammad Samiul Arshad and William J. Beksi. 2020. A Progressive Conditional Generative Adversarial Network for Generating Dense and Colored 3D Point Clouds. In 3DV.
  4. Bounding boxes, segmentations and object coordinates: How important is recognition for 3D scene flow estimation in autonomous driving scenarios?. In ICCV.
  5. ShapeNet: An Information-Rich 3D Model Repository. Technical Report arXiv:1512.03012 [cs.GR].
  6. On visual similarity based 3D model retrieval. In CGF.
  7. LiDAR-Video Driving Dataset: Learning Driving Policies Effectively. In CVPR.
  8. Neural dual contouring. TOG (2022).
  9. Zhiqin Chen and Hao Zhang. 2019. Learning Implicit Fields for Generative Shape Modeling. In CVPR.
  10. Zhiqin Chen and Hao Zhang. 2021. Neural Marching Cubes. TOG (2021).
  11. SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation. In CVPR.
  12. Evgeni Chernyaev. 1995. Marching cubes 33: Construction of topologically correct isosurfaces. Technical Report CN/95-17. CERN. (1995).
  13. Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions. In ICCV.
  14. MRGAN: Multi-Rooted 3D Shape Representation Learning With Unsupervised Part Disentanglement. In ICCVW.
  15. Imperceptible and robust backdoor attack in 3d point cloud. TIFS (2023).
  16. Are we ready for autonomous driving? The KITTI vision benchmark suite. In CVPR.
  17. Generative adversarial nets. In NeurIPS.
  18. Snapnet-r: Consistent 3D multi-view semantic labeling for robotics. In ICCVW.
  19. Sketch2Mesh: Reconstructing and Editing 3D Shapes from Sketches. In ICCV.
  20. Live semantic 3D perception for immersive augmented reality. TVCG (2020).
  21. Hierarchical surface prediction for 3D object reconstruction. In 3DV.
  22. Deep Residual Learning for Image Recognition. In CVPR.
  23. Denoising diffusion probabilistic models. NeurIPS.
  24. Neural wavelet-domain diffusion for 3D shape generation. In SIGGRAPH Asia.
  25. Progressive point cloud deconvolution generation network. In ECCV.
  26. 3D Shape Generation With Grid-Based Implicit Functions. In CVPR.
  27. Dual Contouring of Hermite Data. In TOG.
  28. Softflow: Probabilistic framework for normalizing flow on manifolds. In NeurIPS.
  29. Diederik P Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In ICLR.
  30. Adversarial Generation of Continuous Implicit Shape Representations. In EG.
  31. Discrete point flow networks for efficient point cloud generation. In ECCV.
  32. SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulation. In ICCV.
  33. Learning part generation and assembly for structure-aware shape synthesis. In AAAI.
  34. 3D face point cloud super-resolution network. In IJCB.
  35. Diffusion-SDF: Text-To-Shape via Voxelized Diffusion. In CVPR.
  36. SP-GAN: Sphere-guided 3D shape generation and manipulation. TOG (2021).
  37. Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Process. In CVPR.
  38. Deep Marching Cubes: Learning Explicit Surface Representations. In CVPR.
  39. Multi-dim: A multi-dimensional face database towards the application of 3D technology in real-world scenarios. In IJCB.
  40. Feng Liu and Xiaoming Liu. 2022. 2D GANs Meet Unsupervised Single-View 3D Reconstruction. In ECCV.
  41. Fully Understanding Generic Objects: Modeling, Segmentation, and Reconstruction. In CVPR.
  42. Meshdiffusion: Score-based generative 3D mesh modeling. ICLR.
  43. David Lopez-Paz and Maxime Oquab. 2017. Revisiting classifier two-sample tests. In ICLR.
  44. William E Lorensen and Harvey E Cline. 1987. Marching cubes: A high resolution 3D surface construction algorithm. In SIGGRAPH.
  45. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017).
  46. Shitong Luo and Wei Hu. 2021. Diffusion probabilistic models for 3D point cloud generation. In CVPR.
  47. Controllable Mesh Generation Through Sparse Latent Point Diffusion Models. In CVPR.
  48. Generating 3D House Wireframes with Semantics. In ECCV.
  49. VoroMesh: Learning Watertight Surface Meshes with Voronoi Diagrams. In ICCV.
  50. FullFormer: Generating Shapes Inside Shapes. arXiv preprint arXiv:2303.11235 (2023).
  51. Occupancy Networks: Learning 3D Reconstruction in Function Space. In CVPR.
  52. Polygen: An autoregressive generative model of 3D meshes. In ICML.
  53. T2TD: Text-3D Generation Model based on Prior Knowledge Guidance. arXiv preprint arXiv:2305.15753 (2023).
  54. Gregory M. Nielson. 2004. Dual marching cubes. In IEEE VIS.
  55. Deepsdf: Learning continuous signed distance functions for shape representation. In CVPR.
  56. High-Resolution Image Synthesis With Latent Diffusion Models. In CVPR.
  57. U-Net: Convolutional Networks for Biomedical Image Segmentation. In MICCAI.
  58. CLIP-Sculptor: Zero-Shot Generation of High-Fidelity and Diverse Shapes From Natural Language. In CVPR.
  59. Manifold Dual Contouring. TVCG (2007).
  60. Flexible isosurface extraction for gradient-based mesh optimization. TOG (2023).
  61. Diffusion-Based Signed Distance Fields for 3D Shape Generation. In CVPR.
  62. 3D point cloud generative adversarial network based on tree structured graph convolutions. In ICCV.
  63. 3D Neural Field Generation Using Triplane Diffusion. In CVPR.
  64. Meshgpt: Generating triangle meshes with decoder-only transformers. arXiv preprint arXiv:2311.15475 (2023).
  65. Cnn-slam: Real-time dense monocular slam with learned depth prediction. In CVPR.
  66. Image-to-Video Generation via 3D Facial Dynamics. TCSVT (2022).
  67. Attention is All you Need. In NeurIPS.
  68. Global-to-Local Generative Model for 3D Shapes. TOG (2018).
  69. 3D Semantic Subspace Traverser: Empowering 3D Generative Model with Shape Editing Capability. In ICCV.
  70. TAPS3D: Text-Guided 3D Textured Shape Generation From Pseudo Supervision. In CVPR.
  71. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. In NeurIPS.
  72. Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification. In CVPR.
  73. Learning Descriptor Networks for 3D Shape Synthesis and Analysis. In CVPR.
  74. Pointflow: 3D point cloud generation with continuous normalizing flows. In ICCV.
  75. 3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models. TOG (2023).
  76. Bird Keypoint Detection via Exploiting 2D Texture and 3D Geometric Features. In ICIG.
  77. SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generation. In CGF.
  78. Locally Attentional SDF Diffusion for Controllable 3D Shape Generation. SIGGRAPH.

Summary

  • The paper introduces GenUDC, a novel framework that uses Unsigned Dual Contouring to enhance 3D mesh generation through a structured coarse-to-fine process.
  • It divides mesh representation into face and vertex components to accurately capture complex details and mitigate issues like jagged edges.
  • Empirical results show GenUDC achieves higher accuracy and faster data fitting than methods such as MeshDiffusion, improving metrics like COV, MMD, and 1-NNA.

GenUDC: High Quality 3D Mesh Generation with Unsigned Dual Contouring Representation

The development of effective 3D mesh generation methodologies is a central focus in computer graphics and related fields. The paper "GenUDC: High Quality 3D Mesh Generation with Unsigned Dual Contouring Representation" introduces an innovative approach for 3D mesh generation, leveraging the Unsigned Dual Contouring (UDC) representation. This method aims to overcome the limitations often observed in existing 3D generative models, particularly those using sequence data or deformable tetrahedral grids.

Methodological Contribution

The GenUDC framework capitalizes on the UDC representation to enhance mesh generation. Unlike traditional methods facing memory constraints or surface realism challenges, UDC divides a mesh into face and vertex parts, each discretized within a regular grid. This division facilitates the recovery of complex structures and fine details, addressing the frequent issue of ambiguity found in deformable grids.

The approach is further refined through a two-stage, coarse-to-fine generative process. Initially, the face part is generated to form a rough shape, followed by the vertex part to add detail. This structured process notably resolves issues of jagged edges and enhances the accuracy of mesh interpretation.

Results and Evaluations

The empirical evaluations underscore the superiority of GenUDC in both mesh recovery and data fitting speed. Notably, the framework operates at a significantly reduced processing time and memory requirement compared to methods such as MeshDiffusion, highlighting its efficiency in practical applications.

Quantitative comparisons reveal improvements in metrics such as Coverage (COV), Minimum Matching Distance (MMD), and 1-Nearest Neighbor Accuracy (1-NNA) across multiple categories, including airplanes, cars, and chairs. The data fitting results display GenUDC's ability to model sharp features and intricate geometries effectively, presenting an advancement over conventional techniques limited by over-smoothing and inaccurate surface modeling.

Implications and Future Directions

The implications of adopting UDC for mesh generation extend to various domains, including AR/VR, robotics, and autonomous driving, where high-quality 3D representations are crucial. The compatibility of UDC with deep learning frameworks marks a crucial step in mesh generation, facilitating further development in related areas such as single-view 3D reconstruction and textured mesh synthesis.

Looking forward, the paper hints at numerous possibilities for the expansion of GenUDC, particularly in addressing non-manifold geometry issues and enhancing memory efficiency for higher resolution outputs. The open availability of the code and models suggests a path for collaborative exploration and refinement within the research community.

In conclusion, the GenUDC framework presents a notable advancement in 3D mesh representation and generation, offering a new paradigm that effectively bridges the gap between theoretical complexity and practical application. The focus on precise detail recovery and efficient computation positions it as a valuable tool for researchers and practitioners focused on high-quality 3D model generation.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets