DLVM: Latent Models & Compiler Infrastructure

• DLVM is a context-dependent term referring primarily to deep latent variable models that use variational inference to learn hierarchical and disentangled representations.
• DLVM also denotes a modern compiler infrastructure for deep learning, featuring an SSA-based IR, reverse-mode differentiation, and domain-specific optimizations for efficient GPU code generation.
• DLVM extends to industrial and cognitive applications, integrating approaches like aggregate-posterior matching, importance-weighted imputation, and adaptive latent dimension selection.

Searching arXiv for DLVM-related papers and acronym usages. {"query":"DLVM arXiv deep latent variable models compiler infrastructure distributional latent variable model", "max_results": 10} {"query":"site:arxiv.org DLVM (Wei et al., 2017, Saha et al., 26 Jan 2025, Saha et al., 2023, Mattei et al., 2018, Kong et al., 2022)", "max_results": 5} DLVM is a context-dependent acronym in the arXiv literature. In several machine-learning papers, it denotes a deep latent-variable model, a class of generative models that postulates a joint probability of the form $p_\theta(x,z)=p_\theta(z)\,p_\theta(x\mid z)$ and relies on approximate posterior inference through $q_\phi(z\mid x)$ (Kong et al., 2022). In other contexts, DLVM denotes “DLVM: A modern compiler infrastructure for deep learning systems”, a compiler stack with a linear algebra intermediate representation, algorithmic differentiation by adjoint code generation, domain-specific optimizations, and GPU code generation via LLVM (Wei et al., 2017). In cognitive modeling, DLVM also denotes a Distributional Latent Variable Model that integrates observations across multiple executive-function tasks and individuals (Kasumba et al., 1 Oct 2025). This suggests that the acronym is best interpreted through domain context rather than as a single universally fixed term.

1. Deep latent-variable models as the primary probabilistic usage

A deep latent-variable model (DLVM) posits a joint probability

$p_\theta(x,z)=p_\theta(x\mid z)\,p(z),$

where $x\in\mathbb R^d$ or $x\in\mathbb R^m$ is observed data, $z$ is latent, $p(z)$ is a simple prior such as $N(0,I)$, and $p_\theta(x\mid z)$ is a decoder network (Saha et al., 26 Jan 2025). Exact inference of $p_\theta(z\mid x)$ is generally intractable, so DLVMs introduce an approximate posterior $q_\phi(z\mid x)$0 and optimize a variational lower bound. In the variational autoencoder (VAE), this surrogate posterior is often a factorized Gaussian,

$q_\phi(z\mid x)$1

and training maximizes the ELBO, consisting of a reconstruction term and a KL regularizer (Saha et al., 26 Jan 2025).

The survey literature places VAEs, autoencoders, and deep belief networks within the broader DLVM landscape (Kong et al., 2022). In that survey, DLVMs are contrasted with classic latent-variable models such as PCA, PLS, FA, ICA, GMM, and HMM. The distinction is explicit: classic LVMs have concise principles and good interpretability but limited capacity, whereas neural-network DLVMs offer flexible nonlinear approximation and hierarchical feature extraction at the cost of transparency, hyperparameter sensitivity, computational burden, and larger data requirements (Kong et al., 2022).

Within this usage, disentanglement is a recurrent objective. DLVMs are described as learning meaningful representations in an unsupervised manner, such that hidden explanatory factors are interpretable by independent latent variables (Saha et al., 26 Jan 2025). The VAE is prominent in this literature because the factorized Gaussian posterior encourages alignment of latent factors with latent axes. A central caveat, however, is that this axis-alignment assumption is not generic across all DLVMs; it is especially fragile in models that match only the aggregate posterior rather than the per-instance posterior (Saha et al., 26 Jan 2025).

2. Aggregate-posterior matching, latent geometry, and disentanglement

A major development within DLVM research is the shift from per-instance posterior matching toward aggregate-posterior matching. In the standard VAE, the KL term $q_\phi(z\mid x)$2 can be decomposed as

$q_\phi(z\mid x)$3

where $q_\phi(z\mid x)$4 is the aggregate posterior (Saha et al., 2023). This decomposition motivates methods that match $q_\phi(z\mid x)$5 to the prior directly, rather than penalizing every $q_\phi(z\mid x)$6 against $q_\phi(z\mid x)$7.

The Aggregate-VAE (AVAE) reformulates the VAE objective by replacing the per-instance KL with $q_\phi(z\mid x)$8 and uses kernel density estimate (KDE) to model the aggregate posterior in high dimensions (Saha et al., 2023). In the AVAE formulation, a deterministic encoder is used, and the method is described as curing “holes/pockets” in latent space and mitigating posterior collapse because it does not directly force $q_\phi(z\mid x)$9 toward the prior (Saha et al., 2023). Empirically, the paper reports that AVAE is best on CIFAR-10 in reconstruction and generation FID, best on CelebA generation FID, and lowest in held-out $p_\theta(x,z)=p_\theta(x\mid z)\,p(z),$0 for MNIST and CIFAR-10 (Saha et al., 2023).

This aggregate-posterior perspective changes how disentanglement is evaluated. The paper on disentanglement analysis argues that several recent metrics assume that generative factors align with the cardinal latent axes, but DLVMs such as the AAE and WAE-MMD may encode factors along rotated directions instead (Saha et al., 26 Jan 2025). To address that mismatch, it proposes a statistical method based on PCA that discovers latent directions $p_\theta(x,z)=p_\theta(x\mid z)\,p(z),$1 associated with ground-truth generative factors. Standard metrics such as FactorVAE accuracy and MIG are then applied after projection onto these directions, yielding PCA-FactorVAE and PCA-MIG (Saha et al., 26 Jan 2025).

The empirical consequence is substantial. Models matching the aggregate posterior, especially WAE-MMD and AVAE, are reported to score poorly under axis-aligned metrics but improve markedly under PCA-based metrics, indicating that they did disentangle, albeit along rotated axes (Saha et al., 26 Jan 2025). The same study reports that AVAE attained state-of-the-art disentanglement on 3D Shapes with PCA-FactorVAE $p_\theta(x,z)=p_\theta(x\mid z)\,p(z),$2 and PCA-MIG $p_\theta(x,z)=p_\theta(x\mid z)\,p(z),$3 without sacrificing reconstruction quality, and that pairwise angles among the learned directions were nearly orthogonal for well-disentangled models such as AVAE (Saha et al., 26 Jan 2025). A plausible implication is that latent geometry, not merely marginal independence, is central to evaluating representation quality in DLVMs.

3. Incomplete data, latent-dimension relevance, and model selection

DLVMs have also been adapted to settings with missing observations. MIWAE addresses missing-at-random data by maximizing an importance-weighted lower bound on the observed-data log-likelihood,

$p_\theta(x,z)=p_\theta(x\mid z)\,p(z),$4

without additional computational overhead compared to IWAE (Mattei et al., 2018). The encoder receives a simple imputation $p_\theta(x,z)=p_\theta(x\mid z)\,p(z),$5, such as zero-imputation, while the decoder evaluates $p_\theta(x,z)=p_\theta(x\mid z)\,p(z),$6 under a factorized likelihood. The paper further develops Monte Carlo methods for single and multiple imputation using importance sampling and sampling-importance-resampling (Mattei et al., 2018).

The reported results emphasize that high-quality learning is possible even from substantially incomplete data. On static binarization of MNIST with $p_\theta(x,z)=p_\theta(x\mid z)\,p(z),$7 missing pixels, MIWAE with $p_\theta(x,z)=p_\theta(x\mid z)\,p(z),$8 and zero-imputation achieved test NLL $p_\theta(x,z)=p_\theta(x\mid z)\,p(z),$9 bits, close to MIWAE with oracle imputation at $x\in\mathbb R^d$0 bits and close to IWAE-complete at $x\in\mathbb R^d$1 bits (Mattei et al., 2018). On several UCI continuous benchmarks with $x\in\mathbb R^d$2 randomly dropped features, MIWAE with $x\in\mathbb R^d$3 outperformed MVAE, missForest, $x\in\mathbb R^d$4-NN, and PCA-based imputation in mean RMSE across the listed datasets (Mattei et al., 2018). The paper also reports that a classifier trained on MIWAE multiple imputations of incomplete MNIST reached $x\in\mathbb R^d$5 test accuracy, compared with $x\in\mathbb R^d$6 for training on complete data (Mattei et al., 2018).

A different line of work addresses the choice of latent dimension itself. ARD-VAE replaces the fixed Gaussian prior of the VAE with a hierarchical prior containing one precision parameter $x\in\mathbb R^d$7 per latent axis, with

$x\in\mathbb R^d$8

and then infers a data-dependent marginal prior over $x\in\mathbb R^d$9 (Saha et al., 18 Jan 2025). In practice, when the number of encoded samples is large, each Student’s $x\in\mathbb R^m$0 prior is approximated by a Gaussian with variance $x\in\mathbb R^m$1, yielding a closed-form KL term (Saha et al., 18 Jan 2025).

The empirical findings are explicitly about relevance detection. On dSprites and 3D Shapes, ARD-VAE initialized with $x\in\mathbb R^m$2 identified active dimensions of approximately $x\in\mathbb R^m$3 and $x\in\mathbb R^m$4, respectively, versus ground-truth $x\in\mathbb R^m$5 (Saha et al., 18 Jan 2025). On real datasets including MNIST, CIFAR-10, CelebA, and ImageNet32, it pruned $x\in\mathbb R^m$6–$x\in\mathbb R^m$7 of dimensions while improving FID, with MNIST FID $x\in\mathbb R^m$8 versus VAE $x\in\mathbb R^m$9 and CIFAR-10 FID $z$0 versus VAE $z$1 (Saha et al., 18 Jan 2025). The paper describes collapsed axes as those with very low $z$2 and near-invariant decoder output under perturbation.

4. Specialized DLVMs based on expressive conditional densities

Beyond Gaussian latent-variable formulations, DLVMs have been combined with normalizing flows to obtain exact conditional densities and more expressive latent structure. In FFHFlow-lvm, the grasp-synthesis model introduces a continuous latent vector $z$3 and uses two conditional flows: a latent flow for the prior $z$4 and a grasp flow for the likelihood $z$5 (Feng et al., 2024). Both are RealNVP-style compositions of invertible coupling blocks with Gaussian base densities, and both admit exact likelihood evaluation through change-of-variables formulas (Feng et al., 2024).

Training is based on the ELBO

$z$6

with an amortized Gaussian posterior $z$7 (Feng et al., 2024). The paper explicitly frames the model as a DLVM based on normalizing flows and states that this design counteracts typical pitfalls of cVAEs in generative grasping, notably mode collapse and miss-specified priors. Because both the conditional prior and the generator have exact densities, the model can score a new partial point cloud $z$8 through $z$9 and declare it out-of-distribution when that score exceeds a threshold $p(z)$0 (Feng et al., 2024).

The quantitative results are task-specific but illustrative of what expressive DLVMs enable. In simulation on $p(z)$1 unseen KIT objects, FFHFlow-lvm achieved a mean success rate of $p(z)$2, compared with $p(z)$3 for FFHFlow-cnf and $p(z)$4 for FFHNet (Feng et al., 2024). In real-world tests on $p(z)$5 YCB objects, the reported success rates were $p(z)$6 for FFHFlow-lvm, $p(z)$7 for FFHFlow-cnf, and $p(z)$8 for FFHNet (Feng et al., 2024). In a constrained $p(z)$9-tier shelf scenario, the comparison was $N(0,I)$0 for FFHFlow-lvm versus $N(0,I)$1 for FFHNet. A lightweight variant with $N(0,I)$2 instead of $N(0,I)$3 coupling layers incurred a $N(0,I)$4–$N(0,I)$5 drop in success but halved runtime from approximately $N(0,I)$6 ms to $N(0,I)$7 ms (Feng et al., 2024).

5. DLVMs in industrial data and cognitive assessment

The broad industrial survey treats DLVMs as one half of a larger latent-variable modeling taxonomy. It states that latent variable models can be generally divided into statistical learning-based classic LVM and neural networks-based deep LVM, then reviews DLVM applications in process monitoring and soft sensing (Kong et al., 2022). For process monitoring, the surveyed DLVMs include AE, DAE, SDAE, DBN variants, and VAE variants such as SVAE and $N(0,I)$8VAE; for soft sensing, the survey describes SAE, SVAE, and semi-supervised VAE settings (Kong et al., 2022). The same paper introduces lightweight deep LVM (LDLVM) as a concept meant to combine the virtues of classic LVMs and DLVMs, retaining non-neural building blocks but stacking them hierarchically (Kong et al., 2022).

One LDLVM example is DPI, which repeatedly applies a shallow LVM $N(0,I)$9 across layers,

$p_\theta(x\mid z)$0

followed by Bayesian fault probabilities and a fused Deep Bayesian Statistic (Kong et al., 2022). Another is DPLS, which repeats PLS layers and, in its generalized form, inserts a fixed nonlinear mapping $p_\theta(x\mid z)$1 before each PLS stage (Kong et al., 2022). The survey characterizes these models as aiming for interpretability, smaller parameter count, and effectiveness in small or medium data regimes.

A separate literature uses DLVM to denote a Distributional Latent Variable Model for executive functioning (Kasumba et al., 1 Oct 2025). In that model, each session $p_\theta(x\mid z)$2 has a latent vector $p_\theta(x\mid z)$3, global parameters $p_\theta(x\mid z)$4, and task-specific decoder mappings $p_\theta(x\mid z)$5 that parameterize known observation families such as log-normal reaction-time tasks, binomial accuracy tasks, and psychometric span tasks (Kasumba et al., 1 Oct 2025). The complete-data likelihood factorizes over sessions, tasks, and trials, while the posterior combines a Gaussian prior on $p_\theta(x\mid z)$6 with a regularizing Gaussian prior on $p_\theta(x\mid z)$7 (Kasumba et al., 1 Oct 2025).

The paper emphasizes cross-task and cross-individual sharing. Trained on retrospective data labeled “COLL10,” the model uses gradient-based optimization with Adam for up to $p_\theta(x\mid z)$8 epochs, learning rate $p_\theta(x\mid z)$9, and weight decay $p_\theta(z\mid x)$0, with latent dimensions $p_\theta(z\mid x)$1 corresponding to DLVM-1, DLVM-2, and DLVM-3 (Kasumba et al., 1 Oct 2025). In simulated experiments generated from a DLVM-2 oracle, DLVM-2 achieved $p_\theta(z\mid x)$2 with $p_\theta(z\mid x)$3 observations per task, versus IMLE at approximately $p_\theta(z\mid x)$4, and required approximately $p_\theta(z\mid x)$5 observations per task to reach near-zero KLD, versus IMLE at approximately $p_\theta(z\mid x)$6 (Kasumba et al., 1 Oct 2025). When combined with DALE—Bayesian Distributional Active LEarning based on mutual information—the configuration DLVM-2+DALE+PS2 reached $p_\theta(z\mid x)$7 by about $p_\theta(z\mid x)$8 total trials and then plateaued (Kasumba et al., 1 Oct 2025). The paper states that this demonstrates accurate parameter recovery under data scarcity and faster convergence through adaptive sampling.

6. DLVM as a compiler infrastructure for deep learning systems

In programming languages and systems research, DLVM refers to a compiler framework rather than a probabilistic model. The paper “DLVM: A modern compiler infrastructure for deep learning systems” presents a design inspired by LLVM, centered on a high-level, SSA-based, control-flow graph linear algebra intermediate representation with first-class tensor types and primitive tensor operations (Wei et al., 2017). The IR hierarchy is Module, Function, BasicBlock, and Instruction, and tensor types are written textually as shaped types such as $p_\theta(z\mid x)$9 (Wei et al., 2017).

A defining feature is reverse-mode algorithmic differentiation by adjoint code generation as an IR-level transformation (Wei et al., 2017). A front-end can emit a zero-body gradient declaration, and the differentiation pass generates a concrete gradient function body by copying forward basic blocks, allocating adjoint registers, seeding output adjoints, and traversing forward instructions in reverse order. The paper gives representative rules such as

$q_\phi(z\mid x)$00

for matrix multiplication $q_\phi(z\mid x)$01 (Wei et al., 2017). Because the IR is in SSA form with control flow, the AD transformation can handle loops and branches and inserts phi-nodes for adjoints at merge points (Wei et al., 2017).

The optimization pipeline is domain-specific. Reported passes include algebraic simplification, linear algebra fusion, matrix-multiplication reordering, checkpointing for AD, dead code elimination, common subexpression elimination, and sparse conditional constant propagation (Wei et al., 2017). After optimization, the IR is lowered through ComputeGen and ComputeSched to low-level array operations and scheduled parallel execution, then translated to LLVM IR and GPU kernels through LLVM’s NVPTX backend (Wei et al., 2017). The runtime provides a MemoryTracker for allocation, reuse, and host-device transfers.

The same system includes NNKit, a staged DSL embedded in Swift that emits DLVM IR for tensor computations (Wei et al., 2017). The staging pipeline comprises expression staging, shape specialization, DLVM lowering and optimization, and reification via dynamic linking. The paper characterizes the architecture as modular and generic, with a pass manager, textual IR, plugin-style front-ends, and multiple backends (Wei et al., 2017). It also states that the workshop paper does not include quantitative benchmark tables or charts, though it reports that early experiments achieved performance on par with handwritten CUDA and that the statically typed host language and type-checked IR improve safety by catching many shape and memory errors at compile time (Wei et al., 2017).