Markovian Hierarchical Variational Autoencoders

• Markovian HVAEs are generative models that structure latent spaces into multiple stochastic layers linked by Markov chains for tractable inference.
• They use a bottom-up variational inference approach to optimize the evidence lower bound, revealing redundancy in upper layers and limits in hierarchical feature learning.
• Extensions like Hierarchical Quantized Autoencoders improve discrete representation learning, enhancing lossy compression and preserving semantic features.

Markovian Hierarchical Variational Autoencoders (HVAEs) are a class of generative models that structure the latent variable space into multiple stochastic layers, with each layer forming a Markov chain—each variable depending only on its immediate predecessor or successor. Markovian HVAEs, introduced to leverage deep feature hierarchies within generative modeling frameworks, reveal both representational power and fundamental limitations in terms of hierarchical feature disentanglement. Extensions using quantized discrete codes, such as Hierarchical Quantized Autoencoders (HQA), combine the Markovian structure with powerful discrete representation learning for improved lossy image compression and the retention of semantic features.

1. Model Structure and Markovian Assumptions

A Markovian HVAE defines a generative process with $L$ stochastic layers, with the observation $x$ at the bottom and latent variables $z_1, z_2, \dots, z_L$ hierarchically above. The generative joint factorizes as

$$p_\theta(x,z_{1:L}) = p_\theta(x\mid z_1)\, \prod_{\ell=1}^{L-1} p_\theta(z_\ell\mid z_{\ell+1})\, p_\theta(z_L)$$

where $z_1$ is directly connected to the data, and each $z_\ell$ is conditionally independent of $z_{>\ell+1}$ given $z_{\ell+1}$. The decoder reads only $z_1$, i.e., $x \perp z_{2:L}\mid z_1$. In the discrete code version—exemplified by HQA—the hierarchy is implemented in terms of a Markov chain on discrete code indices $x$0 and associated continuous embeddings $x$1 (Zhao et al., 2017, Williams et al., 2020).

2. Variational Inference and Posterior Approximations

The variational inference model is typically constructed in a "bottom-up" or mean-field form that reverses the arrows of the generative chain:

$x$2

A more general inference model may allow for dependencies on $x$3 or $x$4, but the Markovian case restricts posterior dependencies to mirror the generative structure. In the setting of quantized HVAEs, the approximate posterior is fully factorized over quantized discrete variables at each layer:

$x$5

where $x$6 and stochastic quantization is performed using Gumbel–Softmax annealed to exact sampling at test time (Williams et al., 2020).

3. Evidence Lower Bound and Layer-wise Decomposition

The standard objective is the maximization of the evidence lower bound (ELBO):

$x$7

Substituting the Markov factorization yields a layer-wise decomposition:

\begin{align*} \mathcal{L}\mathrm{ELBO} &= \mathbb{E}_q[\log p\theta(x\mid z_1)] +\sum_{\ell=1}^{L-1}\mathbb{E}_q[\log p_\theta(z_\ell\mid z_{\ell+1})] +\mathbb{E}q[\log p\theta(z_L)] \ &\qquad-\mathbb{E}q[\log q\phi(z_1\mid x)] -\sum_{\ell=2}^L\mathbb{E}_q[\log q_\phi(z_\ell\mid z_{\ell-1}, x)] \end{align*}

In quantized/discrete Markovian HQA, the objective per layer is

$x$8

where $x$9 for the lowest level and $z_1, z_2, \dots, z_L$0 for upper layers (Williams et al., 2020).

4. Theoretical Properties and Limitations

Markovian HVAE architectures are subject to representational redundancy and severe constraints on hierarchical feature learning:

• Redundancy at Optimum: At the global optimum of the ELBO—where $z_1, z_2, \dots, z_L$1 and $z_1, z_2, \dots, z_L$2—the distribution over data can be reconstructed solely from the bottom layer $z_1, z_2, \dots, z_L$3. That is, a two-step Gibbs chain alternately sampling $z_1, z_2, \dots, z_L$4 and $z_1, z_2, \dots, z_L$5 suffices, with upper layers contributing nothing additional [(Zhao et al., 2017), Prop. 1].
• Limited Hierarchical Feature Learning: The conditional priors $z_1, z_2, \dots, z_L$6 are typically chosen from restrictive unimodal families (such as Gaussians). This enforces the approximate posteriors to be unimodal, preventing the emergence of nontrivial, multimodal, part-subpart correlations. As a result, stacking additional latent layers does not yield interpretable or disentangled feature hierarchies—a phenomenon contrary to the compositional representations observed in supervised deep networks (Zhao et al., 2017).

5. Empirical Evaluation and Model Behavior

Experimental studies with Markovian HVAEs using natural image datasets (e.g., MNIST, SVHN) confirm these theoretical limitations:

• Qualitative Sampling: Samples from the full hierarchical ancestral model and the bottom-layer-only Gibbs chain are nearly indistinguishable, demonstrating redundancy of upper layers.
• Layer-wise Noise Perturbation: Varying only the top layer's noise leads to almost all global variation in $z_1, z_2, \dots, z_L$7, whereas perturbing lower layers yields only minimal, local changes—indicating the lack of rich hierarchical structure (Zhao et al., 2017).

In discrete Markov hierarchies such as HQA, additional empirical gains are found in compressed image settings:

• Compression Benchmarks: On CelebA at 576 bits (171× compression), HQA achieves rFID ≈ 39.1, outperforming flat VQ-VAE (rFID ≈ 52.3) and hierarchical autoregressive models (HAMs; rFID ≈ 52.3). At extreme 9-bit rates, HQA maintains rFID ≈ 85.3, superior to HAMs (167.6) and VQ-VAE (196.0) (Williams et al., 2020).
• Ablation Studies: On MNIST at 8 bits, HQA achieves the lowest rFID (22.8) and classification error (~12.4%) compared to non-hierarchical or non-stochastic baselines.

6. Architectural Choices in Markovian Hierarchical Discrete Models

Hierarchical Quantized Autoencoders operationalize the Markov chain via layerwise discrete codes and embeddings:

• Each layer reduces spatial resolution by ×2 via a convolutional encoder, yielding a $z_1, z_2, \dots, z_L$8-dimensional embedding.
• A "codebook" of $z_1, z_2, \dots, z_L$9 vectors provides discrete latent code implementations with Gaussian mixture priors.
• The decoder upsamples from code embeddings to soft targets at the lower layer; at the bottom, reconstructions map back to pixel space via Bernoulli or Gaussian likelihoods.
• Greedy layerwise training—enabled by Markov conditional independence—allows each level to reconstruct the "soft-embedding" representation of the level below (Williams et al., 2020).

These choices—especially the combination of hierarchy, stochastic quantization, and MSE targets—are shown to preserve semantic content and achieve multimodal reconstructions even at very low bitrates.

7. Significance, Context, and Future Directions

Markovian HVAEs clarify both the strengths and inherent bottlenecks of hierarchical stochastic latent modeling in generative frameworks. While conditional independence and Markovian factorization yield tractable inference and modular design, they also induce redundancy and fail to capture deep, disentangled feature hierarchies under most variational training protocols (Zhao et al., 2017). Introduction of discrete latent codes and careful Markovian design—exemplified by HQA—mitigates mode-dropping and delivers empirically superior performance in low-bitrate lossy compression (Williams et al., 2020). A plausible implication is that meaningful hierarchical compositionality may require departing from strict Markovianity, or introducing richer conditional structures and priors at upper layers, to achieve interpretable, disentangled representations.