MoE-Health: Robust Multimodal Clinical Prediction

• MoE-Health is a multimodal fusion framework that adapts predictions to available modalities using specialized expert networks.
• The framework employs dynamic gating and learned missingness embeddings to robustly handle incomplete clinical data while ensuring stable input representation.
• Empirical evaluations on MIMIC datasets show improved AUROC and F1 metrics over standard unimodal and fusion baselines in clinical prediction tasks.

Searching arXiv for the cited MoE-Health paper and closely related medical MoE work. MoE-Health is a Mixture of Experts framework for robust multimodal healthcare prediction that is explicitly designed for clinical settings in which Electronic Health Records, clinical notes, and medical images are available in heterogeneous and often incomplete combinations. Its central premise is that multimodal clinical prediction should not assume complete modality coverage for every sample; instead, the model uses specialized expert networks, a dynamic gating mechanism, and modality-specific learnable missingness embeddings so that prediction can adapt to the modalities actually present for a given admission. The framework is evaluated on MIMIC-IV for in-hospital mortality prediction, long length of stay, and hospital readmission prediction, where it is reported to outperform existing multimodal fusion methods while maintaining robustness across modality-availability patterns (Wang et al., 29 Aug 2025).

1. Conceptual definition and problem setting

MoE-Health addresses a recurrent constraint in clinical machine learning: real-world samples often present with varied or incomplete modalities, whereas many multimodal models either require complete modality data or rely on manual selection strategies. In the formulation described for MoE-Health, the relevant modalities are structured EHR, clinical notes, and chest X-ray images, and the objective is robust prediction under heterogeneous data availability rather than only under idealized fully observed inputs (Wang et al., 29 Aug 2025).

The framework is positioned as a multimodal fusion architecture rather than a unimodal specialist. Its design assumes that healthcare systems generate diverse multimodal data and that the operative challenge is not merely representation learning within each modality, but selective combination of representations under missingness. This suggests a shift from static multimodal fusion toward availability-aware routing, in which the effective model differs across patients depending on what data exist at inference time.

A distinguishing feature is that expert specialization is organized by modality combination observed in training data. For three modalities—EHR, Text, and Image—the paper describes experts specializing in E, T, I, E+T, E+I, T+I, and E+T+I. This makes MoE-Health neither a purely modality-specific MoE nor a generic sparse expert stack; it is a combination-specific MoE intended to match the combinatorics of clinical data availability (Wang et al., 29 Aug 2025).

2. Encoder architecture and multimodal representation

MoE-Health uses modality-specific encoders to convert each available input stream into a fixed-dimensional feature embedding. The EHR encoder combines static demographics, which are embedded with normalization or learnable embeddings, with dynamic time-series data processed via a BiLSTM after binning, producing $\mathbf{e}_{\text{EHR}} \in \mathbb{R}^{d_h}$. Clinical notes are encoded with ClinicalBERT using the $[\mathrm{CLS}]$ embedding, producing $\mathbf{e}_{\text{Text}} \in \mathbb{R}^{d_h}$. Chest X-ray images are encoded with a DenseNet-121 pretrained on CheXpert, with the penultimate layer used as $\mathbf{e}_{\text{Image}} \in \mathbb{R}^{d_h}$ (Wang et al., 29 Aug 2025).

These embeddings are then assembled into a multimodal representation. When a modality is present, the encoder output is used directly; when it is absent, the corresponding feature is replaced by a learned missingness token:

$$\mathbf{v}_m^{(i)} = \begin{cases} \text{Encoder}_m(\mathbf{x}_m^{(i)}) & \text{if } m \text{ present} \ \mathbf{e}_{\text{absent},m} & \text{if } m \text{ absent} \end{cases}$$

The complete sample representation is the concatenation

$$\mathcal{R}^{(i)} = \text{Concat}\big(\mathbf{v}_1^{(i)}, \mathbf{v}_2^{(i)}, \dotsc, \mathbf{v}_M^{(i)}\big).$$

This representation scheme serves two functions simultaneously. First, it keeps the model’s input dimensionality stable regardless of modality availability. Second, it allows missingness to be modeled as a learned signal rather than as zero-padding or omission. A plausible implication is that MoE-Health treats absence itself as predictive context, which is often clinically relevant because ordering practices, documentation density, and imaging availability are institution- and patient-dependent phenomena.

3. Expert specialization, gating, and sparse routing

The MoE fusion layer is the architectural core of MoE-Health. Each expert is described as a lightweight MLP that is pretrained specifically on samples with its assigned modality combination. Rather than forcing one shared fusion operator to process all availability cases, the framework learns combination-specific representations and then uses a learned gate to determine which experts are most relevant for an individual sample (Wang et al., 29 Aug 2025).

The gating mechanism is MLP-based and operates on the concatenated multimodal embedding:

$$\mathbf{g}^{(i)} = G(\mathcal{R}^{(i)}) = \text{Softmax}\left(\mathbf{W}_g \cdot \text{ReLU}(\mathbf{W}_{\mathcal{R}}\mathcal{R}^{(i)} + \mathbf{b}_{\mathcal{R}}) + \mathbf{b}_g\right).$$

Here, $\mathbf{g}^{(i)} \in \mathbb{R}^K$ denotes expert routing weights for sample $i$, with $K$ equal to the number of modality-combination experts observed during training.

To enforce sparsity and efficiency, MoE-Health uses top-$[\mathrm{CLS}]$0 routing with $[\mathrm{CLS}]$1. Only the two highest-scoring experts are activated per sample:

$[\mathrm{CLS}]$2

The paper also states the sparse routing form

$[\mathrm{CLS}]$3

with the corresponding output

$[\mathrm{CLS}]$4

Training is end-to-end with a composite loss:

$[\mathrm{CLS}]$5

The prediction objective is Binary Cross Entropy,

$[\mathrm{CLS}]$6

and expert overuse is discouraged with a load-balancing term,

$[\mathrm{CLS}]$7

where $[\mathrm{CLS}]$8 is the frequency with which expert $[\mathrm{CLS}]$9 is selected in a batch, $\mathbf{e}_{\text{Text}} \in \mathbb{R}^{d_h}$0 is the average gating probability, $\mathbf{e}_{\text{Text}} \in \mathbb{R}^{d_h}$1 denotes element-wise multiplication, and $\mathbf{e}_{\text{Text}} \in \mathbb{R}^{d_h}$2 is the regularization strength. This load-balancing formulation is intended to prevent collapse onto a small subset of experts.

4. Dataset construction, tasks, and empirical performance

MoE-Health is evaluated using MIMIC-IV v2.2 for EHR and clinical notes and MIMIC-CXR-JPG v2.0.0 for chest X-ray images. The final cohort contains 31,088 hospital admissions with at least one modality available, and only 37.4% have all three modalities. Modality coverage is reported as 100% for EHR, 76.5% for clinical notes, and 39.3% for CXRs (Wang et al., 29 Aug 2025).

The three downstream tasks are binary classification problems: in-hospital mortality within 48 hours, long length of stay defined as greater than 7 days, and 30-day hospital readmission. Baselines include unimodal models—BiLSTM for EHR, DenseNet-121 for CXR, and ClinicalBERT for notes—as well as traditional multimodal fusion strategies such as early, late, and joint fusion, the HAIM pipeline, and TriMF.

Implementation details reported for MoE-Health include PyTorch, AdamW with learning rate $\mathbf{e}_{\text{Text}} \in \mathbb{R}^{d_h}$3, batch size 32, an 80/10/10 train/validation/test split, a maximum of 50 epochs, early stopping by validation AUROC, top-$\mathbf{e}_{\text{Text}} \in \mathbb{R}^{d_h}$4 routing with $\mathbf{e}_{\text{Text}} \in \mathbb{R}^{d_h}$5, balance-loss scaling $\mathbf{e}_{\text{Text}} \in \mathbb{R}^{d_h}$6, and training on an NVIDIA A40.

Task	Best reported MoE-Health result	Reported advantage
Mortality	AUROC 0.818, F1 0.465	+0.012 AUROC; +0.03 F1 over runner-up
LOS	AUROC 0.794, F1 0.739	+0.012 AUROC over runner-up; best F1 (tie)
Readmission	AUROC 0.643, F1 0.281	+0.005 AUROC over runner-up

The reported pattern is that MoE-Health consistently outperforms unimodal and advanced multimodal baselines, with the strongest practical relevance in settings where modalities are incomplete or inconsistently available. The paper further reports that as more modalities are combined, both AUROC and F1 improve. For the modality study, the mortality AUROC values are reported as 0.770 for EHR, 0.725 for Text, 0.701 for Image, 0.785 for E+T, 0.781 for E+I, 0.740 for T+I, and 0.794 for E+T+I. This indicates that dual-modality combinations already exceed any single modality, and that the full tri-modal configuration achieves the best performance (Wang et al., 29 Aug 2025).

5. Ablation results and robustness claims

The ablation study attributes performance gains to four components: per-combination expert pretraining, dynamic gating, learned missingness embeddings, and top-$\mathbf{e}_{\text{Text}} \in \mathbb{R}^{d_h}$7 routing. Removing per-combination expert pretraining causes the largest AUROC drop, reported as $\mathbf{e}_{\text{Text}} \in \mathbb{R}^{d_h}$8. Replacing dynamic MLP gating with static fusion yields a $\mathbf{e}_{\text{Text}} \in \mathbb{R}^{d_h}$9 AUROC decrease. Replacing learned missing tokens with zero-padding yields a $\mathbf{e}_{\text{Image}} \in \mathbb{R}^{d_h}$0 decrease, and reducing routing from top-2 to top-1 produces a smaller robustness degradation of $\mathbf{e}_{\text{Image}} \in \mathbb{R}^{d_h}$1 AUROC (Wang et al., 29 Aug 2025).

These ablations clarify what MoE-Health claims as its essential design choices. The expert-pretraining result supports the view that modality-combination specialization, not merely sparse ensembling, is central to performance. The missingness ablation suggests that the framework depends materially on learned absence representations rather than on generic placeholder values. The top-2 versus top-1 result indicates that limited multi-expert aggregation is beneficial even after combination-specific specialization.

The paper’s deployment-oriented argument is that MoE-Health can “do the best possible” with whatever data are available per patient. That claim is grounded in the combination of learned absent-modality embeddings, expert specialization by modality subset, and dynamic gating. A plausible implication is that the framework is intended for operational environments in which documentation practices, imaging frequency, and note availability differ across institutions and across admissions within the same institution.

6. Position within the broader MoE-in-healthcare literature

MoE-Health belongs to a broader family of medical MoE systems, but its technical niche is specifically robust multimodal prediction under missing modality patterns. This differentiates it from EHR-only adaptation frameworks such as TAMER, which combines a soft MoE with Test-Time Adaptation to mitigate patient heterogeneity and distribution shifts in EHR representation learning across four real-world EHR datasets (Zhu et al., 10 Jan 2025). It also differs from modality-specialized medical vision-language systems such as MedMoE, where a report-conditioned MoE routes multi-scale image features through specialized expert branches trained to capture modality-specific visual semantics without requiring modality-specific supervision at inference (Chopra et al., 10 Jun 2025).

In the medical language-modeling line, MOELoRA integrates mixture-of-experts and low-rank adaptation for parameter-efficient fine-tuning of LLMs in multi-task medical applications, using task-motivated gating and retaining the same parameter count as standard LoRA (Liu et al., 2023). MING-MOE similarly addresses diverse medical tasks with sparse Mixture of Low-Rank Adapter experts, but its emphasis is multi-task medical language modeling without task-specific annotations at inference rather than incomplete multimodal fusion (Liao et al., 2024). In medical vision-language modeling, Med-MoE targets lightweight multimodal medical question answering and image classification with domain-specific experts and approximately 30%–50% of activated model parameters (Jiang et al., 2024).

Taken together, these systems suggest that “MoE-Health” can denote both a specific framework—MoE-Health for robust multimodal healthcare prediction—and a broader design direction in which expert specialization is aligned with clinical heterogeneity, modality structure, or deployment constraints. Within that broader direction, MoE-Health is most directly characterized by three linked commitments: specialization by observed modality combination, dynamic top-$\mathbf{e}_{\text{Image}} \in \mathbb{R}^{d_h}$2 gating, and explicit modeling of absent modalities as learned embeddings (Wang et al., 29 Aug 2025).