2000 character limit reached
EucliDreamer: Fast and High-Quality Texturing for 3D Models with Stable Diffusion Depth (2311.15573v2)
Published 27 Nov 2023 in cs.CV and cs.GR
Abstract: This paper presents a novel method to generate textures for 3D models given text prompts and 3D meshes. Additional depth information is taken into account to perform the Score Distillation Sampling (SDS) process with depth conditional Stable Diffusion. We ran our model over the open-source dataset Objaverse and conducted a user study to compare the results with those of various 3D texturing methods. We have shown that our model can generate more satisfactory results and produce various art styles for the same object. In addition, we achieved faster time when generating textures of comparable quality. We also conduct thorough ablation studies of how different factors may affect generation quality, including sampling steps, guidance scale, negative prompts, data augmentation, elevation range, and alternatives to SDS.
- Panohead: Geometry-aware 3d full-head synthesis in 360∘{}^{\circ}start_FLOATSUPERSCRIPT ∘ end_FLOATSUPERSCRIPT, 2023.
- Re-imagine the negative prompt algorithm: Transform 2d diffusion into 3d, alleviate janus problem and beyond, 2023.
- Text2tex: Text-driven texture synthesis via diffusion models, 2023a.
- Text2shape: Generating shapes from natural language by learning joint embeddings, 2018.
- Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation, 2023b.
- Vqgan-clip: Open domain image generation and editing with natural language guidance, 2022.
- Objaverse: A universe of annotated 3d objects, 2022.
- Kaolin: A pytorch library for accelerating 3d deep learning research. https://github.com/NVIDIAGameWorks/kaolin, 2022.
- Generative adversarial networks, 2014.
- Delta denoising score, 2023.
- Debiasing scores and prompts of 2d diffusion for robust text-to-3d generation. arXiv preprint arXiv:2303.15413, 2023.
- Mordor Intelligence. 3d mapping and 3d modelling market size & share analysis. https://www.mordorintelligence.com/industry-reports/3d-mapping-and-3d-modelling, 2023.
- Zero-shot text-guided object generation with dream fields, 2022.
- A style-based generator architecture for generative adversarial networks, 2019.
- Modular primitives for high-performance differentiable rendering. ACM Transactions on Graphics, 39(6), 2020.
- Magic3d: High-resolution text-to-3d content creation, 2023.
- Zero-1-to-3: Zero-shot one image to 3d object, 2023a.
- Syncdreamer: Generating multiview-consistent images from a single-view image, 2023b.
- Latent-nerf for shape-guided generation of 3d shapes and textures, 2022.
- Text2mesh: Text-driven neural stylization for meshes, 2021.
- Nerf: Representing scenes as neural radiance fields for view synthesis, 2020.
- Clip-mesh: Generating textured meshes from text using pretrained image-text models. In SIGGRAPH Asia 2022 Conference Papers. ACM, 2022.
- Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph., 41(4):102:1–102:15, 2022.
- Styleclip: Text-driven manipulation of stylegan imagery, 2021.
- Dreamfusion: Text-to-3d using 2d diffusion, 2022.
- Learning transferable visual models from natural language supervision, 2021.
- Dreambooth3d: Subject-driven text-to-3d generation, 2023.
- Accelerating 3d deep learning with pytorch3d, 2020.
- Texture: Text-guided texturing of 3d shapes, 2023.
- High-resolution image synthesis with latent diffusion models, 2021.
- High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, 2022.
- Vox-e: Text-guided voxel editing of 3d objects, 2023.
- Mvdream: Multi-view diffusion for 3d generation, 2023.
- Walt Disney Animation Studios. Physically-based shading at disney. https://disneyanimation.com/publications/physically-based-shading-at-disney/, 2012.
- Mvdiffusion: Enabling holistic multi-view image generation with correspondence-aware diffusion, 2023.
- Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation, 2023.
- Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling, 2017.
- Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
- Vector-quantized image modeling with improved vqgan, 2022.
- Avatarverse: High-quality & stable 3d avatar creation from text and pose, 2023a.
- Adding conditional control to text-to-image diffusion models, 2023b.