MeshFlow: Efficient Artistic Mesh Generation via MeshVAE and Flow-based Diffusion Transformer

Abstract: We present MeshFlow, a new method for generating artist-like 3D meshes. Current mesh generators often adopt Auto-Regressive (AR) next-token prediction, a natural choice given the discrete nature of mesh topology. However, AR methods scale poorly because the inference cost is quadratic in mesh size. They also require discretizing the vertex coordinates, which introduces quantization errors. To address these challenges, we introduce a Variational Autoencoder (VAE) that, supervised with a contrastive loss, represents both continuous vertex positions and discrete connectivity in a continuous latent space. This latent space is significantly more compact than prior token-based mesh representations. We then build a 3D generator based on a Rectified Flow transformer, generating all mesh vertices and edges in parallel. Our model generates meshes 18x faster than the fastest AR generator while also achieving excellent accuracy across standard mesh-generation metrics. Homepage: https://mesh-flow.github.io/, Code: https://github.com/facebookresearch/meshflow

• The paper demonstrates that MeshFlow achieves high-fidelity mesh synthesis by encoding 3D shapes into a continuous latent space via MeshVAE.
• It introduces a Rectified Flow Diffusion Transformer that generates meshes in parallel, overcoming inefficiencies in autoregressive tokenization.
• Empirical results reveal an 18× faster inference speed and improved reconstruction accuracy compared to traditional AR methods.

MeshFlow: Compact and Efficient Artistic Mesh Generation via MeshVAE and Rectified Flow Diffusion Transformers

Introduction and Motivation

MeshFlow (2606.04621) introduces a paradigm shift in 3D mesh generation by circumventing the inefficiencies and quantization artifacts inherent to traditional autoregressive (AR) approaches. The work targets the direct generation of artist-grade polygonal meshes, which are central to AR/VR/XR, game development, and VFX applications. Existing AR-based mesh generators analogize mesh generation to sequence modeling in NLP, requiring tokenization of mesh components and sequential prediction. This approach leads to quadratic inference cost with mesh size and degrades mesh fidelity through coordinate quantization, as well as frequent early stopping and incomplete geometry.

The authors' central claim is that high-fidelity mesh generation can be achieved with orders-of-magnitude greater computational efficiency and reconstruction precision by encoding the mesh within a continuous, compact latent space. This is realized through their proposed MeshVAE, followed by parallel, conditional mesh synthesis using a Rectified Flow Transformer.

Figure 1: Data-driven insights into mesh size characteristics and the geometric degradation caused by quantization in previous AR approaches.

MeshVAE: Continuous Latent Mesh Encoding

MeshFlow's core innovation is the MeshVAE, a VAE architecture that learns a translation-invariant, compact representation of meshes as a set of continuous latent vectors. Each vector encodes 3D vertex coordinates, vertex normals, and per-vertex edge embeddings. The mesh topology is inferred from pairwise distances between these edge embeddings, eschewing explicit face encoding and discrete adjacency matrices in favor of a differentiable, continuous representation.

This continuous latent representation enables effective downstream denoising via diffusion and avoids quantization-induced collapse of mesh faces and imprecision in geometry.

Figure 2: Overview of mesh encoding and generation pipeline. MeshVAE compresses mesh attributes to latent space, and a Rectified Flow Transformer performs diffusion-based generation, which is then decoded back to the mesh domain.

Encoder/Decoder Architecture

The encoder employs Fourier positional encoding for vertex and normal attributes, augmented with neighbor-aware features based on mesh adjacency. Downsampling is performed via TokenMerge, which agglomerates token vectors, reminiscent of pixel-shuffle methods in vision transformers, followed by a multi-headed cross-attention and self-attention stack. The decoder symmetrically employs TokenSplit for upsampling before reconstructing mesh elements.

Figure 3: TokenMerge/TokenSplit downsampling and upsampling strategy in MeshVAE, critical for information preservation and stable reconstruction.

Contrastive learning is utilized to supervise edge embedding distance, leveraging positive/negative edge samples. The VAE is optimized with a composite loss incorporating Chamfer and mask-based consistency, contrastive adjacency prediction, and a prior KL divergence on the latent code.

Flow-based Mesh Generation

MeshFlow transitions from latent mesh codes to explicit mesh generation using Rectified Flow (RF) diffusion—a straight-line ODE-based process trained with Conditional Flow Matching (CFM) loss. This approach models the flow from noise to the structured latent code in a manner that is robust against path crossing and discretization error. During inference, all mesh vertices and connections are synthesized in parallel rather than sequentially, eliminating the bottleneck of autoregressive token-by-token prediction.

For conditional generation, e.g., mesh from a point cloud, a Diffusion Transformer is used—integrating voxelized point cloud features as input and leveraging 3D rotary positional encodings.

Empirical Analysis

Comprehensive experiments on large-scale proprietary datasets and benchmarks, such as Toys4K, demonstrate robust generalization, mesh quality, and substantial acceleration. MeshFlow achieves mesh synthesis and extraction in approximately one second, outperforming AR baselines by at least $18\times$ in inference speed. Reconstruction accuracy, measured by scaled Chamfer and Hausdorff distances, is strongly superior, with several baselines exhibiting up to $5\times$ higher geometric error.

Qualitative analysis confirms that MeshVAE's continuous representations preserve topological intricacies and geometric detail, unlike AR quantized models which incur face collapse and missing geometry under low quantization regimes.

Figure 4: Comparison between AR mesh tokenizers (requiring quantization) and MeshVAE's continuous representations, highlighting lossless geometric detail on high-resolution assets.

Figure 5: Direct reconstructions from MeshVAE retain both topology and fine geometry across a variety of shapes.

Conditioned mesh generation from point cloud inputs further demonstrates the advantages of parallel, flow-based inference, yielding complete, watertight geometry and fine details—where AR approaches regularly produce fragmented results due to early stopping.

Figure 6: MeshFlow's diffusion paradigm enables fast, complete mesh synthesis in point cloud conditioning tasks; AR baselines suffer from geometric incompleteness under the same conditions.

Architectural Ablations and Robustness

Ablation studies highlight the importance of TokenMerge/TokenSplit operations for both stability and geometric fidelity during VAE training. Downsampling ratios of up to $4\times$ maintain structure and edge F1 above $0.88$, while Q-former and FPS strategies degrade convergence and induce geometric collapse.

Scalability to large mesh sizes is empirically validated: variant datasets with up to $8192$ vertices retain near-lossless reconstruction. The latent space distribution shows a compact, regularized clustering, supporting effective sampling.

Figure 7: Comparative view of downsampling/upsampling strategies; TokenMerge is essential for information-preserving compression.

Figure 8: Visualization of learned mesh latent space distribution, confirming regularization and suitability for diffusion.

Limitations and Future Implications

MeshFlow solely targets triangular mesh topology, whereas quad or n-gon workflows remain prevalent among professionals. Post-processing for small geometric defects persists, particularly in instances of short boundary cycles. Standard geometric error metrics are insufficient proxies for global topology integrity or high-level artifact detection (e.g., surface holes, flipped normals). Enhanced metrics and topology-aware benchmarks will be necessary to push automated mesh generation toward production-grade standards.

The architecture is conceptually extensible to UV/texture coordinate generation and multimodal conditioning (e.g., image or language-driven 3D asset creation), laying foundations for coherent geometry-texture generative pipelines.

Conclusion

MeshFlow establishes a new standard for efficient and high-fidelity 3D mesh generation by leveraging a continuous, contrastively-supervised latent representation (MeshVAE) and a parallel Rectified Flow diffusion process. The method achieves an $18\times$ acceleration in inference with simultaneous improvements in geometric and topological accuracy relative to AR baselines. This approach enables practical, interactive applications in industrial and creative pipelines, and signals a broader trend toward non-autoregressive, diffusion-based 3D asset generation suited for scalable and open-ended downstream synthesis (2606.04621).