Hunyuan3D-Buffalo 1.0: A Unified Multimodal Model for Scalable 3D Generation, Understanding, and Editing

Abstract: Recent advances in image generation have demonstrated the potential of unified multimodal models that integrate understanding, generation, and editing. However, unified 3D modeling remains constrained by scarce multimodal data, particularly the lack of large-scale and geometrically consistent editing data. To address this limitation, we propose Hunyuan3D-Buffalo 1.0, a unified framework supporting 3D understanding, text-to-3D generation, instruction-guided 3D editing, and text-grounded part generation within a single architecture. To enable scalable training, we construct an 87M-scale 3D multimodal corpus, comprising 25M understanding samples, 50M text-to-3D pairs, and 12M editing pairs generated using Nano3D-v2. Architecturally, the framework combines Hunyuan3D-VLM for semantic, structural, and spatial understanding with Hunyuan3D DiT for high-fidelity 3D synthesis. The VLM provides multimodal semantic conditions for generation, while editing and part generation additionally condition the diffusion process on the source object representation to preserve its overall structure and unedited regions. Extensive experiments show that Hunyuan3D-Buffalo 1.0 achieves state-of-the-art or leading performance on text-to-3D generation and 3D editing benchmarks, while exhibiting strong understanding and part-generation capabilities. Our analysis further shows that both generation and understanding improve editing, demonstrating the effectiveness of unified 3D multimodal training. Project Page: https://tencent-hunyuan.github.io/Hunyuan3D-Buffalo1.0/

• The paper introduces a unified multimodal 3D model that integrates understanding, text-to-3D generation, editing, and part segmentation using large-scale, geometrically consistent data.
• It leverages a dedicated 3D vision-language model fused with a diffusion-based generator via an MLP-Connector to ensure precise semantic and structural alignment.
• Experimental results demonstrate significant improvements in fidelity and editing precision, including an 86.7% Chamfer Distance reduction over previous methods.

Hunyuan3D-Buffalo 1.0: Unified Multimodal Large Model for 3D Understanding, Generation, and Editing

Introduction and Motivation

Hunyuan3D-Buffalo 1.0 introduces a unified multimodal 3D model that consolidates understanding, text-to-3D generation, instruction-guided editing, and text-grounded part generation in a single architecture (2608.02711). The work addresses a notable limitation in 3D AI: the lack of comprehensive, large-scale, and geometrically consistent supervision for various 3D tasks, especially editing. By constructing an 87M-scale multimodal 3D corpus spanning 3D understanding, text-to-3D, and editing pairs, the framework aims to achieve versatile and coherent cross-modal competence, analogous to the paradigm shift underway in 2D vision with unified large models.

The architecture synergizes a dedicated 3D vision-LLM (Hunyuan3D-VLM) with a high-capacity diffusion-based 3D generator (3D-DiT), linked via a lightweight MLP-Connector. Uniquely, both 3D generation and editing are tightly conditioned on multimodal semantics and explicit source-shape structure, enabling cross-task synergy and explicit preservation of unedited regions during editing.

Figure 1: Overview of Hunyuan3D-Buffalo 1.0, integrating 3D understanding, generation, editing, and part segmentation within a unified framework.

Multimodal 3D Corpus Construction

The most significant scaling bottleneck in unified 3D modeling is data. The proposed data engine yields:

• 3D Understanding Data: 25M samples, including object/part-level point cloud–text pairs for captioning, QA, grounding, and edit reasoning.
• Text-to-3D Data: 50M high-quality, geometry-grounded text–3D pairs generated via a synthetic prompt pipeline incorporating controlled attribute taxonomies, automated rendering, multi-tier captioning, and stringent VLM-based geometry filtering.

  Figure 2: Comprehensive pipeline for constructing the text-to-3D training corpus via synthetic prompt expansion, rendering, and VLM-based scoring.

• 3D Editing Data: 12M robust (source, edited, instruction) triplets constructed with Nano3D-v2, which combines anchor view selection, learned 3D mask localization, voxel-based editing with structure preservation, geometry and texture refinement (via LATTICE and NaTex), and rigorous multimodal integrity filtering.

  Figure 3: The Nano3D-v2 pipeline for constructing geometrically consistent, large-scale 3D editing supervision.

Examples highlight high-fidelity, multi-stage compositional editing, critical for real-world scalability and reliability.

Figure 4: Representative editing pairs illustrating localized geometric and compositional edits using Nano3D-v2.

Unified Model Architecture

The model adopts a modular, multi-stage architecture:

• Hunyuan3D-VLM: Encodes colored point clouds by combining geometric and semantic representation pathways, fused into token sequences using a Q-Former. This enables explicit 3D perception and language grounding for part-aware captioning, grounding, and instruction synthesis.
• Autoregressive + Diffusion Integration: A pretrained 3D-DiT (from Hunyuan3D-2.1) is conditioned on VLM hidden states aligned through an MLP-Connector. For editing and part segmentation, the denoising process is jointly conditioned on the original asset geometry, ensuring localized edits with robust structure preservation.

  Figure 5: End-to-end pipeline: the Hunyuan3D-VLM backbone and generative DiT modules enable unified 3D understanding, generation, and editing in one architecture.

Training Paradigm

The training protocol is staged:

1. VLM Pretraining for 3D semantic grounding.
2. Text-to-3D Pretraining leveraging the largest corpus to maximize generative prior capacity.
3. Omni Pretraining mixing text-to-3D, editing, and part generation for cross-task capability transfer.
4. Continued Task-Specific Pretraining with mixing to prevent catastrophic forgetting and maximize editability and compositional reasoning.

Human ablations confirm the critical role of scaling the generative corpus; increasing text-to-3D samples yields a marked increase in human preference rates for both fidelity and prompt alignment.

Experimental Results

3D Understanding

Hunyuan3D-VLM achieves the highest scores on UniPart-Bench for part-level QA (SBERT: 85.47; SimCSE: 89.06) and overall object captioning (SBERT: 72.94), outperforming all 3D multimodal baselines in semantic accuracy and lexical fidelity at both part and object levels.

Text-to-3D Generation

In human comparative studies against TRELLIS, Universe3D, and Omni123, Hunyuan3D-Buffalo 1.0 attains a preference rate of 56.6% overall (random: 25%), with strong leads in both geometry and text alignment.

Figure 6: Qualitative text-to-3D generations: accurate structure and prompt alignment across diverse categories.

3D Editing

• Quantitative: On Edit3D-Bench, the model achieves average Chamfer Distance (CD) of 0.0091 and F1 of 0.6515, surpassing Omni123 (CD: 0.0684, F1: 0.2001) and Steer3D (F1: 0.2729) by wide margins—an 86.7% relative CD reduction and 2.39× F1 increase over the best baseline.
• Qualitative: Edits are localized, exhibit precise instruction following, and maximally preserve unedited geometry.

  Figure 7: Shape editing comparison, demonstrating superior geometric consistency and instruction alignment.

Importantly, scaling up only the text-to-3D data—without new editing pairs—directly improves editing capability, confirming that generative prior strength is the key limiting factor for editing in unified paradigms.

Figure 8: Additional text-to-3D data enables the model to correctly execute complex edits, even with sparse editing supervision.

Text-grounded Part Generation

The model supports open-vocabulary, instruction-driven extraction, removal, or generation of semantic parts for arbitrary shapes, unified as a sub-task rather than a standalone pipeline.

Figure 9: Part generation results revealing high geometric fidelity and vocabulary flexibility.

Implications and Future Directions

Hunyuan3D-Buffalo 1.0 establishes unified multimodal pretraining as a pathway to scalable and robust 3D understanding, generation, and editing, with quantitative and qualitative results demonstrating synergistic capability transfer. The architecture shows that strong generative priors derived from massive text-to-3D data are critical for effective editing, lowering the practical barriers for scalable and cost-effective multi-capability 3D models.

Several open research problems remain:

• Single-stage high-quality geometry and texture representation: The reliance on multi-stage pipelines complicates end-to-end editing; a unified stage analog to 2D vision models is unachieved.
• Captioning quality for text-to-3D supervision and efficient, consistent editing pair construction are limiting factors for further scaling.
• End-to-end, joint geometry–texture editing is largely unsolved, restricting fidelity especially for certain categories.
• Architectural exploration: Adoption of “Transfusion”-style, deeply fused networks for 3D remains untested but promising.
• Data scaling—both in volume and annotation quality—continues to be a practical lever for advancing unified 3D AI.

Conclusion

Hunyuan3D-Buffalo 1.0 represents a comprehensive step toward unified, scalable models for multimodal 3D AI, coupling advances in foundation model architecture and large-scale curation of 3D understanding, generation, and editing data. The quantitative superiority on diverse benchmarks and the validated cross-task synergies support unified pretraining as a dominant approach for next-generation 3D foundation models. Continued research into more efficient data construction, single-stage geometry+texture modeling, and architecture-level fusion will be fundamental for future progress in unified 3D AI (2608.02711).