Mesh Latent Diffusion Models

• Mesh LDM is a generative framework that encodes meshes into compact latent representations and performs diffusion-based synthesis to decode explicit mesh structures.
• They enable applications such as structured multi-part generation, dynamic 3D+t reconstruction, and intrinsic texture mapping through diverse decoding strategies.
• Models vary in latent representations and denoiser designs—ranging from DDPM to rectified flow—demonstrating robustness and improved performance across specialized domains.

A Mesh Latent Diffusion Model (Mesh LDM) is a generative model that synthesizes meshes, mesh-conditioned fields, or mesh-decoded dynamics by performing stochastic generation in a learned latent space rather than by diffusing directly over raw vertices, faces, or connectivity. In recent literature, the term covers several related but non-identical constructions: latent diffusion over hand-mesh codes from a SpiralNet++ autoencoder (Woo et al., 1 Oct 2025), latent diffusion over left-ventricular mesh embeddings learned by MeshVAE (Mozyrska et al., 18 Aug 2025), conditional diffusion over Fourier-coefficient tokens derived from CoMA latents for 3D+t cardiac meshes (Lan et al., 2 Jun 2026), structured rectified-flow generation over part-wise mesh latent tokens inherited from TripoSG (Lin et al., 5 Jun 2025), diffusion over triplane latents that decode to meshes through DMTet (Gupta et al., 2023), and intrinsic diffusion over tangent-vector texture field latents on a fixed mesh surface (Mitchel et al., 2023). This suggests that Mesh LDM is best treated as a family of mesh-decoding latent generative models rather than as a single canonical architecture.

1. Definition and conceptual boundaries

The strict core of a Mesh LDM is the separation of representation learning from generative modeling: a mesh, or a mesh-derived object, is first encoded into a compact latent representation; a diffusion or diffusion-like prior is then trained in that latent space; finally, a decoder maps latent samples back to an explicit mesh, a mesh sequence, or a mesh-surface field. In this sense, the latent object may be a vector, a token set, a triplane tensor, Fourier coefficient tokens, or an intrinsic surface field, provided that decoding is mesh-relevant and the stochastic process acts on the learned latent rather than on raw mesh coordinates (Woo et al., 1 Oct 2025).

The literature also contains several important boundary cases. “MeshDiffusion” is diffusion for mesh generation, but not latent diffusion, because it operates directly in deformable tetrahedral-grid attribute space and has no learned latent bottleneck (Liu et al., 2023). “LDM: Large Tensorial SDF Model for Textured Mesh Generation” uses diffusion only in an upstream multi-view image stage; its 3D generator is a feed-forward tensorial SDF predictor, so its “LDM” acronym does not mean latent diffusion model (Xie et al., 2024). PolycubeNet uses conditional diffusion on polycube point clouds with dual latent-token bottlenecks for efficiency, but the diffusion variable remains the output point cloud rather than an autoencoded mesh latent (He et al., 19 May 2026). “Controllable Mesh Generation Through Sparse Latent Point Diffusion Models” and “Drive Any Mesh” are likewise indirect or hybrid cases: both generate through latent point or latent-set proxies that are later converted into meshes or mesh deformations, rather than through a mesh-native latent prior in the narrow sense (Lyu et al., 2023, Shi et al., 9 Jun 2025).

These distinctions matter because “mesh diffusion” and “mesh latent diffusion” are not interchangeable. A model may generate explicit meshes without being latent; it may use latent computation without diffusing a mesh latent; or it may be mesh-adjacent while actually modeling an intermediate proxy such as a point cloud, triplane, polycube, or tangent field.

2. Latent representations and decoding pathways

Across the literature, Mesh LDMs differ most sharply in what they choose to encode and what they decode. Some operate on global shape codes, some on structured token sets, and some on surface-attached fields.

Representative latent objects are summarized below.

Model	Latent object	Decoded output
PartCrafter (Lin et al., 5 Jun 2025)	Part token sets $\boldsymbol{z}_i \in \mathbb{R}^{K\times C}$, concatenated as $\boldsymbol{\mathcal{Z}}\in\mathbb{R}^{NK\times C}$	Multiple mesh parts $\mathbf{p}_i=\{\mathcal{V}_i,\mathcal{F}_i\}$
Mesh LDM for hand reconstruction (Woo et al., 1 Oct 2025)	168-dimensional mesh latent vector $x\in\mathbb{R}^{168}$	Hand mesh vertices $\hat V_0\in\mathbb{R}^{778\times 3}$
MeshLDM for LV anatomy (Mozyrska et al., 18 Aug 2025)	MeshVAE latent $z\in\mathbb{R}^{16}$	LV surface mesh on a shared template
4D F-MeshLDM (Lan et al., 2 Jun 2026)	Fourier coefficient tokens $\Theta\in\mathbb{R}^{B\times D}$, with $B=25$, $D=64$	3D+t biventricular mesh sequence
3DGen (Gupta et al., 2023)	Rolled-out triplane latent $z\in\mathbb{R}^{H\times 3W\times C}$	Textured mesh via SDF/deformation and DMTet
FLDM (Mitchel et al., 2023)	Tangent-vector field latents $\boldsymbol{\mathcal{Z}}\in\mathbb{R}^{NK\times C}$0, with $\boldsymbol{\mathcal{Z}}\in\mathbb{R}^{NK\times C}$1	Texture on a fixed or approximately isometric mesh

In the hand-reconstruction Mesh LDM, the latent is defined by a SpiralNet++ Mesh AutoEncoder: $\boldsymbol{\mathcal{Z}}\in\mathbb{R}^{NK\times C}$2, $\boldsymbol{\mathcal{Z}}\in\mathbb{R}^{NK\times C}$3, with $\boldsymbol{\mathcal{Z}}\in\mathbb{R}^{NK\times C}$4 and $\boldsymbol{\mathcal{Z}}\in\mathbb{R}^{NK\times C}$5 (Woo et al., 1 Oct 2025). In the cardiac MeshLDM for left-ventricular anatomy, the encoder-decoder pair is $\boldsymbol{\mathcal{Z}}\in\mathbb{R}^{NK\times C}$6, $\boldsymbol{\mathcal{Z}}\in\mathbb{R}^{NK\times C}$7, with $\boldsymbol{\mathcal{Z}}\in\mathbb{R}^{NK\times C}$8, and the model relies on meshes that share the same number of vertices and consistent ordering across samples (Mozyrska et al., 18 Aug 2025). In 4D F-MeshLDM, the latent is first framewise, $\boldsymbol{\mathcal{Z}}\in\mathbb{R}^{NK\times C}$9, and is then reparameterized over time by a truncated Fourier series, yielding a coefficient matrix $\mathbf{p}_i=\{\mathcal{V}_i,\mathcal{F}_i\}$0 that becomes the actual diffusion object (Lan et al., 2 Jun 2026).

PartCrafter replaces a monolithic whole-object latent set with a structured collection of part-specific token sets. Given a single RGB image $\mathbf{p}_i=\{\mathcal{V}_i,\mathcal{F}_i\}$1 and a user-specified number of parts $\mathbf{p}_i=\{\mathcal{V}_i,\mathcal{F}_i\}$2, it generates $\mathbf{p}_i=\{\mathcal{V}_i,\mathcal{F}_i\}$3, where each $\mathbf{p}_i=\{\mathcal{V}_i,\mathcal{F}_i\}$4, all in the common canonical coordinate system $\mathbf{p}_i=\{\mathcal{V}_i,\mathcal{F}_i\}$5. Each part receives $\mathbf{p}_i=\{\mathcal{V}_i,\mathcal{F}_i\}$6 tokens, and the global latent is $\mathbf{p}_i=\{\mathcal{V}_i,\mathcal{F}_i\}$7 (Lin et al., 5 Jun 2025). In 3DGen, the latent is neither a mesh vector nor a graph token set, but a Gaussian triplane tensor that is rolled out into $\mathbf{p}_i=\{\mathcal{V}_i,\mathcal{F}_i\}$8 for diffusion and later decoded into SDF and deformation values on a tetrahedral grid before DMTet extraction (Gupta et al., 2023). In FLDM, by contrast, the geometry is fixed and the latent object is a vertexwise tangent-vector field $\mathbf{p}_i=\{\mathcal{V}_i,\mathcal{F}_i\}$9 encoding texture intrinsically on the mesh surface (Mitchel et al., 2023).

The decoder stage is correspondingly heterogeneous. Some decoders reconstruct vertex coordinates directly from latent vectors; some decode each part latent into a separate mesh; some decode to SDF fields and then extract meshes; some decode latent trajectories framewise; and some decode to texture values on the surface. The commonality is not the decoder type, but the fact that the learned latent is the domain of the generative prior.

3. Generative objectives and denoiser architectures

Mesh LDMs do not share a single denoising objective. The most common formulation is DDPM-style latent diffusion, but rectified flow and other variants are also used.

The hand Mesh LDM in the cascaded diffusion framework uses the standard latent DDPM forward process,

$x\in\mathbb{R}^{168}$0

and directly predicts the clean latent with

$x\in\mathbb{R}^{168}$1

Its denoiser is DiT-based. The supplement specifies that $x\in\mathbb{R}^{168}$2 is repeated 21 times and concatenated with the 21-joint 3D hand skeleton to produce a tensor of shape $x\in\mathbb{R}^{168}$3, projected to hidden dimension 512, with cross-attention to four levels of image features and Adaptive Layer Normalization (Woo et al., 1 Oct 2025).

The left-ventricular MeshLDM also adopts a DDPM-style latent formulation, but with a much smaller latent and a much lighter denoiser. Its diffusion variable is $x\in\mathbb{R}^{168}$4, the scheduler uses $x\in\mathbb{R}^{168}$5 steps with a linear schedule from $x\in\mathbb{R}^{168}$6 to $x\in\mathbb{R}^{168}$7, and the denoiser is a 6-layer fully connected network with Swish-1 activations and sinusoidal timestep embeddings (Mozyrska et al., 18 Aug 2025). This is an important counterexample to the assumption that Mesh LDMs require spatial UNets or large transformers; in template-aligned medical anatomy, a compact latent and an MLP prior were sufficient to generate clinically plausible meshes.

PartCrafter is architecturally in the Mesh LDM family but does not use standard DDPM noise prediction. It inherits TripoSG’s rectified-flow formulation, with perturbation

$x\in\mathbb{R}^{168}$8

target velocity $x\in\mathbb{R}^{168}$9, and flow-matching loss

$\hat V_0\in\mathbb{R}^{778\times 3}$0

Its denoiser is a hierarchical “local-global denoising transformer” that alternates part-local attention and global attention across the 21 DiT blocks, with DINOv2 image features injected by cross-attention at both levels (Lin et al., 5 Jun 2025).

4D F-MeshLDM returns to a DDPM prior, but its diffusion object is the Fourier coefficient matrix $\hat V_0\in\mathbb{R}^{778\times 3}$1, treated as a sequence of frequency tokens. The training loss is

$\hat V_0\in\mathbb{R}^{778\times 3}$2

and the denoiser is a Transformer with width 256, depth 8, 8 heads, 500 diffusion steps, learnable Fourier-index embeddings, and AdaLN conditioning on age and gender (Lan et al., 2 Jun 2026).

Other variants show that the latent denoiser can also be a 3D-aware UNet over rolled-out triplanes, as in 3DGen, which trains a 1000-step latent diffusion model with a cosine schedule and $\hat V_0\in\mathbb{R}^{778\times 3}$3-prediction over triplane latents (Gupta et al., 2023), or a fully intrinsic surface network built from field convolutions, as in FLDM, which performs $\hat V_0\in\mathbb{R}^{778\times 3}$4-prediction DDPM directly in tangent-bundle latent space on the mesh surface (Mitchel et al., 2023). The architectural design space is therefore broad: transformer blocks, MLP priors, image-style UNets, and intrinsic field-convolution UNets have all been used successfully.

4. Structured, conditional, and dynamic extensions

A major development in the area is the move from monolithic single-shape generation to structured, conditional, and dynamic mesh latent priors.

PartCrafter is the clearest structured extension. It is described as the first structured 3D generative model that jointly synthesizes multiple semantically meaningful and geometrically distinct 3D meshes from a single RGB image. It does not require segmentation masks, segmented crops, or a separate segmentation network at inference. Instead, decomposition is built into the latent organization itself, with per-part identity embeddings and alternating local/global attention. For objects, the output parts correspond to object subcomponents; for scenes, the same machinery is reused with “parts” effectively becoming object instances (Lin et al., 5 Jun 2025).

The hand-reconstruction Mesh LDM is conditional rather than fully generative in the unconditional sense. It is the second stage of a coarse-to-fine cascade: a first diffusion model samples plausible 3D joints from 2D keypoints, and the Mesh LDM then reconstructs a dense 3D hand mesh conditioned on a sampled joint hypothesis and image features. Its central claim is “distribution-aware joint–mesh relationships”: the mesh model is not trained only on ground-truth joints, but on diverse sampled joints from the first-stage diffusion model, so it learns robustness to upstream ambiguity and self-occlusion (Woo et al., 1 Oct 2025).

4D F-MeshLDM extends the Mesh LDM notion into periodic 3D+t generation. Each frame of a biventricular mesh sequence is encoded by a CoMA-based VAE, the full latent trajectory is parameterized by a truncated Fourier series,

$\hat V_0\in\mathbb{R}^{778\times 3}$5

and diffusion is learned over the coefficient tokens $\hat V_0\in\mathbb{R}^{778\times 3}$6. Because the normalized phase satisfies $\hat V_0\in\mathbb{R}^{778\times 3}$7 and $\hat V_0\in\mathbb{R}^{778\times 3}$8, inverse Fourier synthesis yields exactly periodic latent trajectories by construction (Lan et al., 2 Jun 2026).

DriveAnyMesh occupies a hybrid position. It denoises sequences of latent sets that jointly capture shape and motion information, then decodes them into point cloud trajectory sequences that drive an existing mesh over time. The output is directly compatible with rasterization-based rendering engines because topology is preserved by updating vertex positions while keeping connectivity fixed. The paper explicitly notes, however, that it is not fully mesh-native, since the latent does not explicitly encode mesh connectivity, geodesics, or physical constraints (Shi et al., 9 Jun 2025).

3DGen and FLDM show two other conditional directions. 3DGen performs conditional and unconditional generation of textured or untextured meshes by diffusing triplane latents and decoding them through a tetrahedral SDF/deformation field plus a color field (Gupta et al., 2023). FLDM is narrower in scope but more intrinsic: it is a single-textured-mesh paradigm in which diffusion operates on latent vector fields attached to the mesh surface, enabling unconditional texture variation, label-guided generation, inpainting, and approximately isometric generative transfer to new meshes (Mitchel et al., 2023).

5. Empirical landscape

The empirical record of Mesh LDMs is domain-specific, and results are best read relative to task definitions rather than as a single cross-paper leaderboard.

In structured mesh generation, PartCrafter reports that on Objaverse it achieves CD $\hat V_0\in\mathbb{R}^{778\times 3}$9, F-Score $z\in\mathbb{R}^{16}$0, and IoU $z\in\mathbb{R}^{16}$1, compared with HoloPart’s $z\in\mathbb{R}^{16}$2, $z\in\mathbb{R}^{16}$3, and $z\in\mathbb{R}^{16}$4. On ABO it reports CD $z\in\mathbb{R}^{16}$5, F-Score $z\in\mathbb{R}^{16}$6, and IoU $z\in\mathbb{R}^{16}$7, versus HoloPart’s $z\in\mathbb{R}^{16}$8, $z\in\mathbb{R}^{16}$9, and $\Theta\in\mathbb{R}^{B\times D}$0. Runtime for 4-part objects on an H20 GPU is reported as 34s versus 18 minutes. The paper also notes that on 3D-Front scenes PartCrafter surpasses MIDI in CD and F-Score, especially on occluded subsets, while MIDI retains slightly better IoU because it is given ground-truth segmentation masks (Lin et al., 5 Jun 2025).

In probabilistic hand reconstruction, the Mesh LDM stage is supported by several ablations. Training on diverse sampled joint hypotheses performs better than training on off-the-shelf or averaged joints: on FreiHAND, the proposed setting reports P-MPJPE 5.0 and P-MPVPE 5.2, with a best-of-sampling result of 4.2 and 4.5. The latent-space design also outperforms MANO-space diffusion: on FreiHAND, latent diffusion yields P-MPJPE 5.00 and P-MPVPE 5.23, while MANO-space diffusion gives 5.70 and 5.82; on HO3Dv2, the comparison is 7.50 and 7.52 versus 7.81 and 7.81. The paper further reports Pearson correlations between joint error and mesh error of 0.93 on FreiHAND and 0.80 on HO3Dv2, indicating that mesh quality remains strongly dependent on the conditioning joint (Woo et al., 1 Oct 2025).

In medical anatomy synthesis, the static LV MeshLDM reports that generated meshes have an average difference in population mean of 2.4% across all clinical scores and both phases relative to the gold standard. At end-diastole, generated cavity volume is $\Theta\in\mathbb{R}^{B\times D}$1 ml versus $\Theta\in\mathbb{R}^{B\times D}$2 ml in the test set, and generated LV mass is $\Theta\in\mathbb{R}^{B\times D}$3 g versus $\Theta\in\mathbb{R}^{B\times D}$4 g. The same paper also reports limited diversity, particularly at end-systole, with 1-NNA and coverage indicating that generated and real distributions remain distinguishable (Mozyrska et al., 18 Aug 2025). In dynamic cardiac generation, 4D F-MeshLDM reports conditional-generation Specificity $\Theta\in\mathbb{R}^{B\times D}$5 mm, Coverage@5 98.60%, and Seq. RMSE $\Theta\in\mathbb{R}^{B\times D}$6 mm, while its cycle-consistency metrics are markedly smaller than those of recurrent and continuous-time baselines: volume consistency $\Theta\in\mathbb{R}^{B\times D}$7 and mesh consistency $\Theta\in\mathbb{R}^{B\times D}$8 (Lan et al., 2 Jun 2026).

For appearance-focused models, FLDM reports SIFID $\Theta\in\mathbb{R}^{B\times D}$9 and LPIPS $B=25$0 on single-textured-mesh generation, compared with Sin3DM’s SIFID $B=25$1 and LPIPS $B=25$2, and attributes this to intrinsic equivariance and tangent-field latents rather than to geometry generation (Mitchel et al., 2023). For textured mesh generation from triplane latents, 3DGen reports unconditional textured-mesh FID of 36.27 on cars, 11.07 on chairs, and 12.34 on tables, outperforming GET3D variants, and reports image-conditioned geometry improvements in Chamfer-L1 and Shading FiD as well (Gupta et al., 2023).

Taken together, these studies show that Mesh LDMs have been used for at least five distinct problem classes: monolithic mesh synthesis, structured multi-part generation, conditional dense reconstruction from articulated proxies, static medical anatomy synthesis, periodic 3D+t anatomy synthesis, and intrinsic texture generation on meshes.

6. Limitations, misconceptions, and open directions

Several limitations recur across the literature. A first is topology dependence. The medical MeshLDM and 4D F-MeshLDM both assume fixed topology and consistent vertex ordering across subjects; the 3D cardiac datasets are template-aligned, and the VAE backbones depend on that correspondence (Mozyrska et al., 18 Aug 2025, Lan et al., 2 Jun 2026). 3DGen outputs explicit meshes, but topology is still mediated through a tetrahedral grid and DMTet extraction rather than generated as arbitrary connectivity (Gupta et al., 2023). FLDM does not generate geometry at all; it assumes the mesh is given and models only surface appearance (Mitchel et al., 2023).

A second limitation is that latent diffusion does not by itself solve structural ambiguity. In the hand Mesh LDM, most multimodality resides in the first-stage joint distribution, while the second-stage mesh model has limited variance for a fixed joint condition (Woo et al., 1 Oct 2025). In PartCrafter, the number of parts is user-specified rather than inferred, the part semantics depend on mined asset decompositions, and the paper does not provide exact backbone latent dimensions, exact sampling-solver details, or guarantees of watertightness or manifoldness for generated parts (Lin et al., 5 Jun 2025). In the static LV MeshLDM, diversity remains lower than in the real population, especially at end-systole, which the paper partly attributes to the small dataset size of 1034 meshes per phase (Mozyrska et al., 18 Aug 2025).

A third issue is terminological. Not every mesh-producing diffusion system is a Mesh LDM in the strict sense. MeshDiffusion is a strong data-space diffusion baseline for explicit mesh extraction, but not a latent model (Liu et al., 2023). The “LDM” acronym in “Large Tensorial SDF Model” is unrelated to latent diffusion (Xie et al., 2024). PolycubeNet is best described as conditional point-cloud diffusion with latent-token computation, not as a standard latent diffusion model over compressed mesh codes (He et al., 19 May 2026). This suggests that the term Mesh LDM should be used carefully, with explicit reference to what the latent variable actually represents.

Open directions are correspondingly clear in the source papers. PartCrafter raises questions about variable-cardinality prediction, larger structured datasets, and more formal part correspondence across samples (Lin et al., 5 Jun 2025). The hand Mesh LDM points to stronger uncertainty modeling beyond joint-conditioned refinement and beyond dataset-specific autoencoders (Woo et al., 1 Oct 2025). The cardiac models point toward relaxing fixed-topology assumptions, handling severe pathologies, and extending periodic priors beyond single-cycle synthesis (Mozyrska et al., 18 Aug 2025, Lan et al., 2 Jun 2026). Texture-focused intrinsic models raise the possibility of broader surface-native latent diffusion beyond single-mesh regimes, but currently remain tied to fixed or approximately isometric geometry (Mitchel et al., 2023).

In that broader perspective, Mesh LDM denotes not a single recipe but a design principle: learn a mesh-relevant latent space, place the stochastic prior in that space, and use a decoder or extractor that preserves enough geometric or surface structure for the target application. The strongest current systems differ precisely in how they instantiate those three steps.