Multi-Encoder-Decoder VAE (MED-VAE)

• MED-VAE is a class of variational autoencoder architectures featuring multiple encoder and decoder branches interconnected through a shared or hierarchical latent space.
• It enhances model robustness and planning by enabling decoder diversity, hierarchical inference, and flexible multi-modal or multi-source integration.
• The design principle supports improved latent alignment and generalization, proving effective in diverse applications such as text generation, molecular design, and neural data alignment.

Searching arXiv for the cited MED-VAE and closely related multi-encoder/multi-decoder VAE papers. Multi-Encoder-Decoder Variational Autoencoder (MED-VAE) denotes a class of variational autoencoder architectures that replace the canonical single-encoder/single-decoder arrangement with multiple encoder branches, multiple decoder branches, or both, typically coupled through a shared latent variable or a structured latent hierarchy. The designation is used explicitly in "Task-guided cross-subject latent alignment: a multi-encoder-decoder VAE" (Papathanasiou et al., 14 Jun 2026), but closely related ideas appear under other names in hierarchical text generation, multimodal learning, multi-source inference, molecular generation, and alternative variational-bound constructions (Shen et al., 2019, Kwon et al., 2022, Korthals, 2019, Kurle et al., 2018, Cukier, 2022). In that literature, MED-VAE is best understood as an architectural and variational pattern rather than a single canonical model.

1. Terminological scope and lineage

The term has both a narrow and a broad usage. In the narrow sense, it refers to the explicit Multi-Encoder Multi-Decoder Variational Autoencoder proposed for cross-subject neural alignment without shared stimuli (Papathanasiou et al., 14 Jun 2026). In the broader sense, it covers VAE designs in which multiple encoders, multiple decoders, or multiple latent levels are introduced to improve alignment, robustness, planning, or generalization (Shen et al., 2019, Kwon et al., 2022, Korthals, 2019, Kurle et al., 2018, Cukier, 2022).

A recurring source of confusion is that not every relevant paper uses the acronym formally. The long-text model of "Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models" is described as a hierarchical or multi-level VAE rather than a MED-VAE, even though its hierarchical CNN encoder, latent hierarchy, and hierarchical LSTM decoder match a multi-encoder / multi-decoder interpretation (Shen et al., 2019). Conversely, the 2026 neuroscience paper adopts the MED-VAE label explicitly and uses it as the name of a concrete architecture (Papathanasiou et al., 14 Jun 2026).

Paper	MED-VAE interpretation	Distinctive mechanism
"Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models" (Shen et al., 2019)	Multi-level encoder/decoder VAE	Hierarchical planning from sentence vectors to words
"String-based Molecule Generation via Multi-decoder VAE" (Kwon et al., 2022)	Shared-encoder, multi-decoder VAE	Decoder ensemble with different latent samples and collaborative loss
"M$^2$VAE - Derivation of a Multi-Modal Variational Autoencoder Objective from the Marginal Joint Log-Likelihood" (Korthals, 2019)	Multi-encoder / multi-decoder multimodal VAE	All-subset ELBO derivation from marginal joint log-likelihood
"Multi-Source Neural Variational Inference" (Kurle et al., 2018)	Modular multi-source MED-VAE	Source-wise posteriors with MoE/PoE fusion and conflict detection
"Three Variations on Variational Autoencoders" (Cukier, 2022)	Two-encoder/two-decoder variational variants	EUBO and optional fixed P-PCA encoder
"Task-guided cross-subject latent alignment: a multi-encoder-decoder VAE" (Papathanasiou et al., 14 Jun 2026)	Formal MED-VAE	Subject-specific neural branches aligned by a shared ANN scaffold

2. Core architectural motifs

The common structural motif is a shared latent representation coupled to multiple inference and generative mappings. In multi-source and multimodal formulations, the joint model is factorized as

$$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$

or, for two modalities,

$p(a,b,z)=p(z)\,p(a\mid z)\,p(b\mid z),$

so that each decoder is conditionally independent given the latent state (Kurle et al., 2018, Korthals, 2019). Each encoder then approximates a posterior over the same latent variable, either from one source, one modality, or one modality subset.

A second motif is latent hierarchy. In the long-text model, the inference network for the deeper variant factorizes as

$$q_\phi(z_1,z_2\mid x)=q_\phi(z_2\mid x)\,q_\phi(z_1\mid x),$$

while the generative prior is

$$p_\theta(z_1,z_2)=p_\theta(z_2)\,p_\theta(z_1\mid z_2).$$

The top latent carries broad paragraph semantics, and the lower latent refines that information, so the bottom latent prior becomes more flexible and data-dependent than a fixed isotropic Gaussian (Shen et al., 2019).

A third motif is decoder multiplicity without encoder multiplicity. The molecular MD-VAE uses one shared transformer encoder together with $K$ autoregressive transformer decoders $p_{\theta_k}(x\mid y,z)$, and aggregates next-token predictions by averaging logits rather than probabilities (Kwon et al., 2022). By contrast, the cross-subject neural MED-VAE uses multiplicity on both sides: one subject-specific fMRI encoder and decoder per subject, plus one shared ANN encoder and one shared ANN decoder, all coupled through a common latent space of dimension $d_z$ (Papathanasiou et al., 14 Jun 2026).

A fourth motif is parallel variational encoders. In the VAE$_A$, VAE$_B$, and VAE$$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$0 constructions, two learned encoder distributions are introduced, and VAE$$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$1 further adds a fixed encoder $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$2 obtained from probabilistic PCA. In that setting, multiplicity is used not to represent different data modalities, but to alter the probabilistic meaning of the variational approximation and the evidence bounds (Cukier, 2022).

3. Objectives and inference schemes

The baseline objective remains the VAE evidence lower bound. For a standard VAE,

$$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$3

For the two-layer long-text model, the ELBO is written in hierarchical form as

$$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$4

with the KL decomposition

$$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$5

This factorization regularizes the lower latent toward a prior conditioned on the upper latent rather than toward a fixed standard Gaussian (Shen et al., 2019).

In the molecular multi-decoder setting, the central addition is a collaborative ensemble loss. With different latent samples per decoder,

$$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$6

and the reconstruction term is

$$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$7

The total objective adds KL regularization with a dynamically controlled $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$8 term:

$$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$9

The paper’s stated rationale is a bias–variance trade-off: collaborative loss reduces bias, while different latent sampling encourages decoder diversity and reduces variance (Kwon et al., 2022).

In multimodal VAEs derived from marginal joint log-likelihood, the objective is not limited to a single joint ELBO. The M$p(a,b,z)=p(z)\,p(a\mid z)\,p(b\mid z),$0VAE derivation expands $p(a,b,z)=p(z)\,p(a\mid z)\,p(b\mid z),$1 into terms involving full-joint likelihoods, leave-one-out conditionals, and recursively all lower-order subsets, then combines unimodal ELBOs, joint ELBOs, and KL terms that align subset-specific posteriors such as $p(a,b,z)=p(z)\,p(a\mid z)\,p(b\mid z),$2, $p(a,b,z)=p(z)\,p(a\mid z)\,p(b\mid z),$3, and $p(a,b,z)=p(z)\,p(a\mid z)\,p(b\mid z),$4 (Korthals, 2019). In multi-source inference, the integrated posterior can instead be formed as a mixture of experts

$p(a,b,z)=p(z)\,p(a\mid z)\,p(b\mid z),$5

or as a product of experts

$p(a,b,z)=p(z)\,p(a\mid z)\,p(b\mid z),$6

with a hybrid objective

$p(a,b,z)=p(z)\,p(a\mid z)\,p(b\mid z),$7

to keep both source-wise and integrated beliefs meaningful (Kurle et al., 2018).

A more unusual development is the introduction of an Evidence Upper Bound (EUBO) in VAE$p(a,b,z)=p(z)\,p(a\mid z)\,p(b\mid z),$8. By comparing two learned encoder distributions $p(a,b,z)=p(z)\,p(a\mid z)\,p(b\mid z),$9 and $$q_\phi(z_1,z_2\mid x)=q_\phi(z_2\mid x)\,q_\phi(z_1\mid x),$$0, the paper constructs an upper estimate of $$q_\phi(z_1,z_2\mid x)=q_\phi(z_2\mid x)\,q_\phi(z_1\mid x),$$1 that complements the standard ELBO. It further shows, via a second-order expansion, that

$$q_\phi(z_1,z_2\mid x)=q_\phi(z_2\mid x)\,q_\phi(z_1\mid x),$$2

when the two learned encoders become close (Cukier, 2022). This makes multi-encoder structure a tool for variational bracketing rather than only for multimodal fusion.

4. Hierarchical planning for long-form text

In long-form text generation, MED-VAE-style design is instantiated as a hierarchy of encoders, latent variables, and decoders aligned with the linguistic hierarchy of paragraphs, sentences, and words (Shen et al., 2019). The encoder is a hierarchical CNN inference network: sentence-level CNNs encode each sentence into a fixed-length vector, and a paragraph-level CNN aggregates those vectors to parameterize the approximate posterior.

The decoder is explicitly hierarchical. Rather than decode words directly from a paragraph latent, the model first generates intermediate sentence-level plan vectors with a sentence-level LSTM,

$$q_\phi(z_1,z_2\mid x)=q_\phi(z_2\mid x)\,q_\phi(z_1\mid x),$$3

and then realizes each sentence with a word-level LSTM,

$$q_\phi(z_1,z_2\mid x)=q_\phi(z_2\mid x)\,q_\phi(z_1\mid x),$$4

$$q_\phi(z_1,z_2\mid x)=q_\phi(z_2\mid x)\,q_\phi(z_1\mid x),$$5

The sentence-level representation $$q_\phi(z_1,z_2\mid x)=q_\phi(z_2\mid x)\,q_\phi(z_1\mid x),$$6 is concatenated with the previous word embedding at every step, and the word-level decoder’s initial state is inferred from the plan vector through an MLP (Shen et al., 2019).

The paper distinguishes this from merely stacking deeper encoders and decoders. The relevant architectural point is a planning hierarchy: paragraph-level encoding, latent abstraction at one or two stochastic layers, sentence-level plan vectors, and word-level realization. A common misconception is to read the model as only a deeper encoder–decoder VAE; the paper instead emphasizes that the hierarchy is meant to learn a coarse paragraph-level plan and then realize that plan into words (Shen et al., 2019).

Empirically, the hierarchical models are reported to mitigate posterior collapse and improve coherence-related metrics. On Yelp and arXiv, flat-VAE has a KL term near zero, approximately $$q_\phi(z_1,z_2\mid x)=q_\phi(z_2\mid x)\,q_\phi(z_1\mid x),$$7, whereas ml-VAE-S reaches $$q_\phi(z_1,z_2\mid x)=q_\phi(z_2\mid x)\,q_\phi(z_1\mid x),$$8 and $$q_\phi(z_1,z_2\mid x)=q_\phi(z_2\mid x)\,q_\phi(z_1\mid x),$$9, and ml-VAE-D reaches $$p_\theta(z_1,z_2)=p_\theta(z_2)\,p_\theta(z_1\mid z_2).$$0 and $$p_\theta(z_1,z_2)=p_\theta(z_2)\,p_\theta(z_1\mid z_2).$$1. Perplexity also improves from $$p_\theta(z_1,z_2)=p_\theta(z_2)\,p_\theta(z_1\mid z_2).$$2 for flat-VAE to $$p_\theta(z_1,z_2)=p_\theta(z_2)\,p_\theta(z_1\mid z_2).$$3 for ml-VAE-S and $$p_\theta(z_1,z_2)=p_\theta(z_2)\,p_\theta(z_1\mid z_2).$$4 for ml-VAE-D. On Yelp unconditional generation, BLEU-4 rises from $$p_\theta(z_1,z_2)=p_\theta(z_2)\,p_\theta(z_1\mid z_2).$$5 for flat-VAE to $$p_\theta(z_1,z_2)=p_\theta(z_2)\,p_\theta(z_1\mid z_2).$$6 for ml-VAE-D. Human evaluation on Yelp favors ml-VAE over flat-VAE and AAE in grammar, consistency, non-redundancy, and overall quality; latent interpolations are reported as smooth and semantically meaningful; and t-SNE plots show clustering by topic in arXiv abstracts (Shen et al., 2019).

5. Shared-encoder, multi-decoder cVAE for molecular strings

In string-based molecule generation, the MED-VAE idea appears as a multi-decoder VAE for conditional SMILES generation (Kwon et al., 2022). The model maintains one shared encoder approximating

$$p_\theta(z_1,z_2)=p_\theta(z_2)\,p_\theta(z_1\mid z_2).$$7

and multiple decoders

$$p_\theta(z_1,z_2)=p_\theta(z_2)\,p_\theta(z_1\mid z_2).$$8

In the reported experiments, the architecture uses a shared transformer encoder and $$p_\theta(z_1,z_2)=p_\theta(z_2)\,p_\theta(z_1\mid z_2).$$9 autoregressive transformer decoders, typically $K$0 decoders, with decoder outputs aggregated by averaging logits for next-token prediction (Kwon et al., 2022).

The defining training mechanism is that each decoder receives a different latent sample,

$K$1

rather than all decoders sharing one $K$2. The paper states that training decoders independently may not be effective because the bias of the ensemble decoder increases severely under autoregressive inference. Its final MD-VAE formulation is therefore joint rather than independent: shared encoder, shared regularization, collaborative ensemble reconstruction loss, and per-decoder latent samples (Kwon et al., 2022).

The experimental setting uses ZINC250K for training and seen evaluation, ZINC310K for unseen or OOD-like evaluation, transformer encoders and decoders, latent dimension $K$3, $K$4 property dimensions, Adam with learning rate $K$5, $K$6 epochs, and batch size $K$7. The regularization target is tuned to around $K$8 in ControlVAE style (Kwon et al., 2022).

The strongest quantitative claim concerns OOD performance. On unseen ZINC310K reconstruction success, ControlVAE achieves $K$9, MD achieves $p_{\theta_k}(x\mid y,z)$0, MD$p_{\theta_k}(x\mid y,z)$1 achieves $p_{\theta_k}(x\mid y,z)$2, MD$p_{\theta_k}(x\mid y,z)$3 achieves $p_{\theta_k}(x\mid y,z)$4, and MD$p_{\theta_k}(x\mid y,z)$5 achieves $p_{\theta_k}(x\mid y,z)$6. For OOD molecular generative efficiency, measured as validity + uniqueness + novelty, ControlVAE obtains $p_{\theta_k}(x\mid y,z)$7 and MD$p_{\theta_k}(x\mid y,z)$8 obtains $p_{\theta_k}(x\mid y,z)$9, which the paper highlights as a $d_z$0 relative improvement. The ablation study further shows that multi-decoder alone is not enough: MD can be weaker than ControlVAE in reconstruction loss, collaborative loss alone helps, multi-$d_z$1 sampling alone helps, and combining both is best (Kwon et al., 2022).

6. Multimodal, multi-source, and multi-bound generalizations

A substantial part of the MED-VAE literature is not tied to one application domain, but to the problem of how multiple encoders and decoders should be trained and related probabilistically.

In multimodal learning, M$d_z$2VAE starts from the marginal joint log-likelihood $d_z$3 or, more generally, $d_z$4, and derives a trainable objective that supports full multimodal input, unimodal input, and arbitrary modality subsets. For two modalities, the architecture includes $d_z$5, $d_z$6, and $d_z$7, together with decoders $d_z$8 and $d_z$9. For three modalities, the framework extends to all subset encoders $_A$0. The paper’s main conceptual claim is that multimodal objectives should be derived from the marginal joint log-likelihood rather than assembled ad hoc, and that reconstruction terms are needed in addition to KL alignment because KL matching alone may underconstrain posterior statistics (Korthals, 2019).

In multi-source learning, each encoder is conditioned on a different information source,

$_A$1

and all such beliefs refer to the same latent variable. This makes it possible to compare source-specific posteriors directly and to compute a conflict statistic

$_A$2

The paper studies complementarity, redundancy, and conflict under MoE, PoE, and hybrid inference. It reports, among other findings, that on MNIST-NO integrating redundant beliefs improves performance by about $_A$3 nats over a single-source IWAE baseline, and that in a pendulum experiment the conflict measure detects a failed sensor so that integrating only non-conflicting sources yields better inference than combining all sensors blindly (Kurle et al., 2018).

A different generalization is to use multiple encoders to modify the evidence bounds themselves. "Three Variations on Variational Autoencoders" introduces VAE$_A$4, VAE$_A$5, and VAE$_A$6, all of which add a second learned encoder/decoder pair, and VAE$_A$7 also adds a fixed P-PCA posterior

$_A$8

VAE$_A$9 is the most distinctive because it yields an EUBO that can be used together with the ELBO to interrogate convergence; when the two learned encoders approach each other, the EUBO and ELBO are expected to meet from above and below. In this line of work, the MED-VAE idea is not primarily about missing modalities or ensemble decoding, but about the optimization geometry induced by multiple approximate posteriors (Cukier, 2022).

7. Cross-subject neural alignment without shared stimuli

The most explicit formal MED-VAE is the cross-subject neural alignment model introduced for fMRI data (Papathanasiou et al., 14 Jun 2026). Its purpose is to align neural activity across subjects without shared stimuli by anchoring representations to a common scaffold provided by a pretrained ANN. The architecture contains, for each subject $_B$0, a subject-specific fMRI encoder

$_B$1

a subject-specific fMRI decoder

$_B$2

one shared ANN encoder

$_B$3

and one shared ANN decoder

$_B$4

The fMRI encoders output Gaussian posterior parameters, latent samples are obtained by the reparameterization trick, and evaluation uses $_B$5. The ANN scaffold is used during training but discarded at inference time (Papathanasiou et al., 14 Jun 2026).

The full loss combines four reconstruction pathways and KL regularization:

$_B$6

with $_B$7 in the main results. The cross-modal fMRI $_B$8 ANN term is the main alignment constraint, because all subjects’ latent codes must reconstruct the same ANN feature space through the same decoder (Papathanasiou et al., 14 Jun 2026).

The experiments use the Natural Scenes Dataset with $_B$9 subjects and about $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$00 images per subject, most of them non-overlapping and about $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$01 shared among some or all subjects. MED-VAE is trained only on subject-specific non-overlapping responses. Cross-subject evaluation focuses on $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$02 images viewed by all participants and $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$03 semi-shared images viewed by subjects $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$04 and $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$05. The fMRI data come from occipitotemporal cortex, about $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$06 voxels per subject, and the ANN scaffold is ResNet-50 pretrained on ImageNet with activations taken from one layer every $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$07 layers and reduced to $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$08 dimensions per selected layer. An appendix also tests CLIP ViT-L/14 (Papathanasiou et al., 14 Jun 2026).

The baselines are deterministic Shared Response Model and Generalized Procrustes analysis, both fit on the $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$09 fully shared images and evaluated on the $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$10 semi-shared images. MED-VAE, by contrast, does not use shared stimuli during training (Papathanasiou et al., 14 Jun 2026).

The reported quantitative results emphasize semantic organization, alignment quality, retrieval, and robustness to held-out stimuli. MED-VAE obtains a multi-label silhouette score of $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$11, versus $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$12 for SRM and $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$13 for Procrustes; category decoding exact match of $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$14, versus $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$15 and $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$16; balanced accuracy of $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$17, versus $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$18 and $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$19; component-wise correlation of $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$20, versus $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$21 and $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$22; RSA of $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$23, versus $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$24 and $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$25; and top-1 cross-subject retrieval on semi-shared images of $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$26, versus $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$27 for SRM and $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$28 for Procrustes (Papathanasiou et al., 14 Jun 2026).

A particularly important interpretive point concerns stimulus-driven signal. Within-trial reconstruction is lower for MED-VAE ($$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$29) than for SRM ($$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$30) and Procrustes ($$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$31), but under cross-trial evaluation all methods are essentially the same, about $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$32. NC-normalized reconstruction yields $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$33 for MED-VAE, $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$34 for SRM, and $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$35 for Procrustes. The paper interprets the within-trial advantage of SRM and Procrustes as noise capture rather than better preservation of stimulus-driven variance. It further reports that MED-VAE is the best method across subject pairs for cross-subject voxel prediction, and that when predicted voxel patterns are passed to a frozen MindEye2 image decoder, MED-VAE substantially outperforms SRM and Procrustes at $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$36D on most metrics and retains an advantage on high-level semantic metrics at $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$37D (Papathanasiou et al., 14 Jun 2026).

Implementation details relevant for reproduction are also explicit: two-layer MLPs with LayerNorm, ReLU, and dropout $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$38; latent size $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$39 in the main results and $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$40 in additional experiments; hidden size $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$41 for $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$42D and $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$43 for $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$44D; Adam with learning rate $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$45; batch size $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$46; and $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$47 training epochs. The paper further states that subjects can have different voxel counts or ROIs, inference does not require the ANN scaffold, and a new subject can be integrated by training only a subject-specific encoder-decoder pair while freezing the shared space; it also reports that a new subject can be added with only about $$p_\theta(x,z)=p(z)\prod_{m=1}^{M} p_{\theta_m}(x_m\mid z)$$48 of that subject’s data and still recover most of the alignment quality (Papathanasiou et al., 14 Jun 2026).

As an overview of the literature, MED-VAE is not one fixed topology but a design principle: multiple encoders and decoders are introduced so that latent structure can support hierarchical planning, decoder diversity, modality-subset inference, source-wise belief fusion, variational bracketing, or subject alignment. The precise role of multiplicity differs across domains, but the underlying probabilistic idea is stable: a richer arrangement of inference and generative mappings can make the latent variable materially more useful than in a flat VAE.