LADM: Latent Aligned Diffusion Model

• LADM is a framework that structures latent representations to align geometry and semantics with task-specific needs.
• It utilizes both continuous SDE and discrete DDPM formulations to guide stable, controllable, and semantically coherent diffusion processes.
• LADM enhances generative quality and deterministic translation in applications like 3D generation, medical imaging, and LiDAR synthesis.

Searching arXiv for the named LADM-related papers to ground the article in current literature. Latent Aligned Diffusion Model (LADM) denotes a family of diffusion-based generative formulations in which the latent representation is explicitly organized so that its geometry, coordinates, or condition interfaces are compatible with the downstream task before or during diffusion. In recent work, this principle appears as multimodal shape–image–text alignment for 3D generation, semi-paired source–target coupling for diffusion bridges, clinically grounded latent geometry for lung nodule progression, semantic alignment for LiDAR synthesis, spatio-temporal alignment of neural latent dynamics, Gaussian-aligned autoencoding for longitudinal MRI, and prior-aligned tokenizer design for latent diffusion (Zhao et al., 2023, Wang et al., 10 Sep 2025, Song et al., 16 Mar 2026, Xiang et al., 30 Jun 2025, Wang et al., 2023, Dao et al., 17 Apr 2026, Yue et al., 8 May 2026). Across these variants, the common pattern is that diffusion is not asked to discover a usable latent manifold implicitly; instead, the manifold is shaped by alignment objectives that encode cross-modal semantics, domain correspondences, attribute relations, or prior structure.

1. Terminological scope and lineage

The expression “Latent Aligned Diffusion Model” is not confined to a single canonical architecture. In the cited literature, it is used for several related constructions whose shared premise is that latent alignment is a prerequisite for reliable diffusion-based generation or translation. Early 2023 work introduced two distinct but related directions: Michelangelo’s “alignment-before-generation” pipeline for conditional 3D shape synthesis and ERDiff’s diffusion-guided recovery of spatio-temporal neural latent dynamics (Zhao et al., 2023, Wang et al., 2023). By 2025–2026, the same design logic had expanded into semi-supervised domain translation, LiDAR synthesis, medical longitudinal prediction, brain MRI generation, and tokenizer design for DiT-scale latent diffusion (Wang et al., 10 Sep 2025, Xiang et al., 30 Jun 2025, Song et al., 16 Mar 2026, Dao et al., 17 Apr 2026, Yue et al., 8 May 2026).

Work	Aligned entities	Primary use
Michelangelo	shape, image, and text latents	conditional 3D shape generation
ERDiff	source and target latent dynamics	neural manifold alignment
LADB	source and target latent couplings	semi-supervised domain translation
SG-LDM	semantic labels and LiDAR features	semantic-to-LiDAR synthesis
NAMD	nodule attributes, scans, and EHR context	follow-up nodule generation
CLIMB	brain MRI latents and Gaussian prior	longitudinal MRI generation
PAE	tokenizer latents and refined VFM priors	diffusion-friendly latent manifold design

This breadth matters because it distinguishes LADM from a narrow reading in which alignment is merely a conditioning trick. In these works, alignment is instead a structural property of the latent space itself: it determines whether score estimation, conditional control, or deterministic domain mapping remains semantically stable under diffusion.

2. Generative formulations

One mathematically explicit formulation appears in LADB, which defines LADM in a shared latent space $\mathbb{R}^d$ with time variable $t \in [0,1]$. The forward process is the latent-space SDE

$d z_t = f(z_t,t)\,dt + g(t)\,d w_t,$

and deterministic generation or translation uses the probability-flow ODE

$$d z_t = \Bigl[f(z_t,t)-\tfrac12 g(t)^2 \nabla_{z_t}\log q_t(z_t)\Bigr]dt.$$

The intractable score is approximated by a neural network $s_\theta(z_t,t)$ trained by denoising score matching. In this setting, LADM is a target-domain score model trained on a semi-paired latent coupling, and inference concatenates a source PF-ODE solve with a target PF-ODE solve to obtain deterministic one-to-one translation (Wang et al., 10 Sep 2025).

A second, widely used formulation is discrete latent diffusion in DDPM style. Michelangelo, NAMD, and CLIMB all use latent noising of the form

$$z_t = \sqrt{\bar\alpha_t}\,z_0 + \sqrt{1-\bar\alpha_t}\,\epsilon,\qquad \epsilon\sim\mathcal N(0,I),$$

and train a denoiser to predict $\epsilon$ by an $\ell_2$ objective (Zhao et al., 2023, Song et al., 16 Mar 2026, Dao et al., 17 Apr 2026). Michelangelo’s conditional Aligned Shape Latent Diffusion Model (ASLDM) uses a discrete latent diffusion over aligned shape latents and injects image or text conditions through cross-attention; during training the condition is dropped with probability $0.1$, and sampling uses classifier-free guidance with guidance scale $s \approx 7.5$ (Zhao et al., 2023). SG-LDM follows the same DDPM logic on LiDAR range-image representations, with classifier-free guidance and DDIM sampling in 50 steps (Xiang et al., 30 Jun 2025).

PAE shifts attention from the reverse process to the first-stage tokenizer. Its argument is that the quality of downstream latent diffusion depends on three properties of the latent manifold—coherent spatial structure, local manifold continuity, and global manifold semantics—and that these properties are more consistent with downstream generation quality than reconstruction fidelity (Yue et al., 8 May 2026). In that sense, PAE treats “latent alignment” as manifold design for diffusion rather than as a single score-model objective.

3. Alignment mechanisms

The most direct multimodal alignment mechanism appears in Michelangelo’s Shape-Image-Text-Aligned VAE (SITA-VAE). A frozen CLIP image encoder and text encoder, both ViT-L-14, produce token sequences, while a trainable Perceiver-based Transformer encodes 3D point samples into shape tokens. Shape, image, and text embeddings are then aligned by shape–image and shape–text contrastive losses, while the implicit decoder reconstructs occupancy with binary cross-entropy and the VAE includes a KL term. The total loss is

$t \in [0,1]$0

with $t \in [0,1]$1 and $t \in [0,1]$2 in practice (Zhao et al., 2023).

LADB replaces explicit contrastive alignment with latent coupling. A small paired set of source-to-target samples and a larger unpaired target set are combined into a semi-paired latent distribution. For paired samples, the source latent is pushed to the prior end of the bridge by a frozen source PF-ODE; for unpaired targets, the terminal latent is sampled independently from $t \in [0,1]$3. The target-domain score network is trained on this mixture by denoising score matching, and “no explicit KL term is introduced; latent alignment is enforced implicitly by supplying true paired latents to the diffusion bridge loss” (Wang et al., 10 Sep 2025).

NAMD and SG-LDM align latent structure to external semantics in different ways. NAMD learns a “nodule-aligned” latent space in which Euclidean relations between codes mirror clinically meaningful attribute differences derived from the EHR; it adds both an alignment term based on pairwise similarity distributions and a predictive malignancy loss to the VAE objective (Song et al., 16 Mar 2026). SG-LDM, by contrast, introduces a semantic projector $t \in [0,1]$4 that maps bottleneck features to the spatial resolution of the semantic map and optimizes an auxiliary semantic-alignment loss based on cosine similarity, weighted by $t \in [0,1]$5, to prevent unconditional classifier-free training from collapsing semantic structure (Xiang et al., 30 Jun 2025).

ERDiff, CLIMB, and PAE broaden the notion of alignment further. ERDiff fits a diffusion prior to source latent trajectories and aligns target latents by maximum likelihood under that prior, augmented by a Sinkhorn divergence to preserve diversity and prevent mode collapse (Wang et al., 2023). CLIMB’s Gaussian-Aligned Autoencoder (GATE) uses deterministic encoding plus a sliced-CDF alignment loss, perceptual loss, and adversarial loss so that $t \in [0,1]$6 lies close to $t \in [0,1]$7 without conventional VAE reparameterization noise (Dao et al., 17 Apr 2026). PAE aligns deterministic tokenizer latents to refined VFM-derived priors through Spatial Structure Regularization (SSR), Manifold Continuity Regularization (MCR), and Semantic Consistency Regularization (SCR), with RMS normalization producing a unit-sphere-like latent (Yue et al., 8 May 2026).

4. Architectures and conditioning pathways

Michelangelo is architecturally notable for separating alignment, diffusion, and neural-field reconstruction. The shape encoder is a Perceiver-based Transformer that cross-attends from learnable queries into Fourier-encoded point samples and refines the resulting tokens by self-attention; the decoder is a Transformer-based implicit occupancy network. The conditional diffusion model is a 13-block U-Net style Transformer whose blocks contain multi-head self-attention, cross-attention to the condition embedding, and a feed-forward network with 3076 hidden dimensions and GELU under LayerNorm pre-norm, with skip connections between downsampling and upsampling paths (Zhao et al., 2023).

LADB uses a different composition. The source side is a pretrained latent diffusion model with encoder, decoder, and latent-space U-Net score network; the target side has an autoencoder of the same architecture and a target LADM score network trained by Eq. (6). Test-time mapping is the concatenation of source-to-prior and prior-to-target PF-ODE solves. When multiple source modalities are available, the source-side terminal latents are fused by a weighted Fréchet mean,

$t \in [0,1]$8

before the reverse target PF-ODE is applied (Wang et al., 10 Sep 2025).

NAMD, SG-LDM, ERDiff, CLIMB, and PAE illustrate the heterogeneity of LADM backbones. NAMD initializes its VAE and U-Net from StableDiffusion-1.5, concatenates the baseline scan latent with the noised follow-up latent, and injects an EHR-derived context matrix $t \in [0,1]$9 from MedGemma-1.5 via cross-attention in every U-Net block (Song et al., 16 Mar 2026). SG-LDM uses a 2D diffusion U-Net with circular convolutions on $d z_t = f(z_t,t)\,dt + g(t)\,d w_t,$0 range images, a three-layer CNN semantic projector, and bottleneck attention with 512 channels (Xiang et al., 30 Jun 2025). ERDiff employs a Diffusion Transformer with Spatio-Temporal Transformer Blocks, where spatial attention operates across latent dimensions and temporal attention operates across time bins; its encoder and decoder are recurrent, with Poisson likelihood for neural spiking and a ridge-regression behavior head (Wang et al., 2023). CLIMB replaces transformer self-attention with Mamba state-space modules inside a 3D latent diffusion pipeline, motivated by reduced computational overhead while preserving image quality (Dao et al., 17 Apr 2026). PAE uses a frozen VFM encoder, a Detail-Aware Modulator, attn+conv projector and deprojector, RMS-normalized bottleneck latents, and a LightningDiT-XL/1 generator as the downstream diffusion model (Yue et al., 8 May 2026).

5. Applications and empirical evidence

In conditional 3D shape generation, Michelangelo evaluates reconstruction on ShapeNet with 50 K meshes across 55 categories and reports an overall IoU of 0.966, compared with 0.955 for 3DS2V, 0.950 for 3DILG, 0.934 for IF-Net, 0.888 for ConvOccNet, and 0.825 for OccNet (Zhao et al., 2023). For image-conditioned generation it reports SI-S(ULIP) 13.82, SI-S(SITA) 15.21, P-FID 1.59, and P-IS 13.23; for text-conditioned generation it reports P-FID 2.08, P-IS 13.56, ST-S(ULIP) 16.65, and ST-S(SITA) 13.13. Qualitatively, the model recovers wheels and chair backs under image conditioning and, for the prompt “a 3D model of a small wooden airplane,” produces struts and fuselage curvature that the baselines miss (Zhao et al., 2023).

In semi-supervised domain translation and LiDAR synthesis, LADM-style alignment supports both controllability and robustness under scarce supervision. On LSUN-Bedroom and LSUN-Churches depth-to-image translation with 50% paired data, LADB reports FID 33.78, IS 2.35, LPIPS 0.6335, and MSE 0.1125; the runner-up latent DDBM achieves FID 34.23 (Wang et al., 10 Sep 2025). As the paired fraction shrinks to 10%, LADB still achieves FID $d z_t = f(z_t,t)\,dt + g(t)\,d w_t,$1, compared with DDBM $d z_t = f(z_t,t)\,dt + g(t)\,d w_t,$2. In multi-source translation with fused depth and segmentation at 25% paired, LADB reports FID 34.72, versus 37.03 for Uni-ControlNet and 55.43 for DDBM (Wang et al., 10 Sep 2025). SG-LDM, evaluated on SemanticKITTI and SynLiDAR, reports same-domain FRID 4.4, FSVD 10.5, FPVD 7.9, JSD 0.084, and MMD $d z_t = f(z_t,t)\,dt + g(t)\,d w_t,$3, versus LiDM’s 22.9, 20.2, 17.7, 0.072, and $d z_t = f(z_t,t)\,dt + g(t)\,d w_t,$4; it further yields a +1.8 % mIoU gain in data augmentation and a diffusion-based LiDAR translation framework that raises mIoU by +3.4 % over the baseline (Xiang et al., 30 Jun 2025).

Medical imaging and neuroscience supply a different validation regime, where alignment is evaluated not only by sample realism but also by downstream utility. On the NLST cohort, NAMD synthesizes follow-up nodule images whose malignancy-prediction performance reaches AUROC $d z_t = f(z_t,t)\,dt + g(t)\,d w_t,$5 and AUPRC $d z_t = f(z_t,t)\,dt + g(t)\,d w_t,$6, compared with AUROC 0.742 and AUPRC 0.263 for real baseline LDCT, AUROC 0.819 and AUPRC 0.393 for real follow-up LDCT, and AUROC 0.765 and AUPRC 0.305 in an ablation without $d z_t = f(z_t,t)\,dt + g(t)\,d w_t,$7 (Song et al., 16 Mar 2026). CLIMB, trained on 6,306 MRI scans from 1,390 participants in ADNI, reports MSE $d z_t = f(z_t,t)\,dt + g(t)\,d w_t,$8, SSIM $d z_t = f(z_t,t)\,dt + g(t)\,d w_t,$9, PSNR $$d z_t = \Bigl[f(z_t,t)-\tfrac12 g(t)^2 \nabla_{z_t}\log q_t(z_t)\Bigr]dt.$$0 dB, and LPIPS $$d z_t = \Bigl[f(z_t,t)-\tfrac12 g(t)^2 \nabla_{z_t}\log q_t(z_t)\Bigr]dt.$$1 for GATE + Mamba Diffusion, improving on VAE Autoencoder + Mamba Diffusion, which reports MSE $$d z_t = \Bigl[f(z_t,t)-\tfrac12 g(t)^2 \nabla_{z_t}\log q_t(z_t)\Bigr]dt.$$2, SSIM $$d z_t = \Bigl[f(z_t,t)-\tfrac12 g(t)^2 \nabla_{z_t}\log q_t(z_t)\Bigr]dt.$$3, PSNR $$d z_t = \Bigl[f(z_t,t)-\tfrac12 g(t)^2 \nabla_{z_t}\log q_t(z_t)\Bigr]dt.$$4 dB, and LPIPS $$d z_t = \Bigl[f(z_t,t)-\tfrac12 g(t)^2 \nabla_{z_t}\log q_t(z_t)\Bigr]dt.$$5 (Dao et al., 17 Apr 2026). ERDiff, on non-human primate motor cortex data, achieves cross-day $$d z_t = \Bigl[f(z_t,t)-\tfrac12 g(t)^2 \nabla_{z_t}\log q_t(z_t)\Bigr]dt.$$6 versus the best baseline at approximately $$d z_t = \Bigl[f(z_t,t)-\tfrac12 g(t)^2 \nabla_{z_t}\log q_t(z_t)\Bigr]dt.$$7, inter-subject $$d z_t = \Bigl[f(z_t,t)-\tfrac12 g(t)^2 \nabla_{z_t}\log q_t(z_t)\Bigr]dt.$$8 versus approximately $$d z_t = \Bigl[f(z_t,t)-\tfrac12 g(t)^2 \nabla_{z_t}\log q_t(z_t)\Bigr]dt.$$9, and on rat CA1 recordings $s_\theta(z_t,t)$0 versus 21.7% for the next best method; removing either the spatial or temporal transformer hurts performance by 4–6 points (Wang et al., 2023).

At the tokenizer level, PAE provides evidence that latent alignment can dominate end-to-end diffusion performance even when reconstruction is not the main optimization target. On ImageNet $s_\theta(z_t,t)$1, PAE(DINOv2) reaches gFID 1.03 at 800 epochs and unguided gFID 1.43, while also achieving performance comparable to RAE with up to 13x faster convergence under the same training setup (Yue et al., 8 May 2026). In the few-step regime, PAE(DINOv2) reports gFID 1.05 in 45 steps, compared with FAE’s 250-step gFID 1.29. Its ablations show gFID improving from approximately 7.8 without any prior-alignment loss to 1.86 when SSR, MCR, and SCR are used together (Yue et al., 8 May 2026).

6. Distinctions, misconceptions, and open problems

A common misconception is that latent alignment in diffusion is synonymous with ordinary VAE prior matching. The surveyed literature shows otherwise. Michelangelo does include a KL term, but only as one component of a broader contrastive multimodal alignment scheme (Zhao et al., 2023). LADB states that no explicit KL term is introduced and that alignment is enforced implicitly through semi-paired latent couplings (Wang et al., 10 Sep 2025). CLIMB explicitly avoids the sampling noise inherent in conventional variational autoencoders by using a deterministic Gaussian-aligned autoencoder (Dao et al., 17 Apr 2026). PAE does not posit an analytic Gaussian latent prior at all; instead, it treats the empirical distribution of refined VFM priors on the RMS-normalized sphere as the target geometry (Yue et al., 8 May 2026).

A second misconception is that alignment mainly serves reconstruction quality. PAE argues that coherent spatial structure, local manifold continuity, and global manifold semantics are more consistent with downstream generation quality than reconstruction fidelity (Yue et al., 8 May 2026). NAMD’s ablation without $s_\theta(z_t,t)$2 lowers AUROC from 0.805 to 0.765 and AUPRC from 0.346 to 0.305, indicating that the latent geometry itself affects clinical prediction (Song et al., 16 Mar 2026). ERDiff likewise shows that preserving both spatial covariation and temporal evolution is necessary for high-quality alignment and decoding, with 4–6 point drops when either transformer component is removed (Wang et al., 2023).

The main unresolved issues are domain-specific rather than universal. Michelangelo motivates aligned latent diffusion by observing that direct image/text $s_\theta(z_t,t)$3 3D generation is prone to poor semantic consistency because 3D shapes have an additional dimension whose distribution differs significantly from that of 2D images and texts (Zhao et al., 2023). SG-LDM identifies another failure mode: naïve classifier-free training can degrade the unconditional branch because the network over-relies on the semantic condition when it is present (Xiang et al., 30 Jun 2025). ERDiff notes that its current formulation handles alignment from a single source distribution, incurs computational overhead in source diffusion-model training, and has quadratic $s_\theta(z_t,t)$4 Sinkhorn cost; it explicitly proposes multi-source diffusion priors, applications to EEG and fMRI, faster samplers such as DPM-Solver, and richer latent-variable dynamics as extensions (Wang et al., 2023). LADB already moves in that direction by extending the bridge framework to multi-source translation and class-conditioned multi-target translation without architectural change beyond adding the class label via cross-attention (Wang et al., 10 Sep 2025).

Taken together, the literature defines LADM less as a fixed model class than as a design principle for diffusion systems: the latent space should be made structurally compatible with the semantics, correspondences, or dynamics that the reverse process is expected to preserve. Where that compatibility is made explicit, the reported effects are not only better fidelity or diversity, but also more reliable conditional control, deterministic translation, clinically meaningful uncertainty, and improved downstream task performance.