KnowRare: Deep Learning for Rare ICU Outcomes

• KnowRare is a deep learning framework designed to predict clinical outcomes for rare ICU conditions under severe data scarcity and intra-condition heterogeneity.
• It employs a three-stage pipeline combining self-supervised pre-training, condition-agnostic representation learning, and selective domain adaptation using a data-driven condition knowledge graph.
• Empirical evaluations on MIMIC-III and eICU demonstrate that KnowRare outperforms state-of-the-art models and clinical scoring systems like APACHE IV in tasks such as ICU mortality and 30-day readmission.

Searching arXiv for "KnowRare" and the associated ICU rare-condition prediction paper to ground the article in current arXiv metadata. KnowRare is a domain adaptation-based deep learning framework for predicting clinical outcomes for rare conditions in the intensive care unit (ICU). It is designed for settings in which formally recognised rare diseases and low-prevalence ICU conditions are both affected by severe data scarcity and substantial intra-condition heterogeneity. The framework addresses these constraints by first learning condition-agnostic representations from diverse electronic health records through self-supervised pre-training, then selectively adapting knowledge from clinically similar conditions using a condition knowledge graph. Across MIMIC-III and eICU, KnowRare is evaluated on 90-day mortality, 30-day readmission, ICU mortality, remaining length of stay, and phenotyping, and is reported to outperform existing state-of-the-art models as well as APACHE IV and IV-a on ICU mortality (Zhu et al., 8 Jul 2025).

1. Definition and problem setting

KnowRare is formulated around prediction for rare conditions in the ICU, where “rare conditions” include both formally recognised rare diseases and low-prevalence conditions in critical care (Zhu et al., 8 Jul 2025). The framework operates at the condition level: each patient is assigned a primary diagnosis using ICD-9-CM level-3 codes, and prediction is evaluated on cohorts defined by those conditions (Zhu et al., 8 Jul 2025).

Within each dataset, a condition is labelled Rare if its prevalence is < 1 in 2,000 patients and if it has at least one positive label for the relevant prediction task (Zhu et al., 8 Jul 2025). From all such candidates, the study selects the 10 least prevalent conditions satisfying these criteria in each dataset for evaluation (Zhu et al., 8 Jul 2025). The underlying rarity is pervasive at the code level: in MIMIC-III, 383 of 587 level-3 ICD-9 codes meet this definition, and in eICU, 192 of 303 codes do so (Zhu et al., 8 Jul 2025).

The targeted prediction tasks are dataset-specific. On MIMIC-III, KnowRare is used for 90-day mortality after hospital discharge and 30-day readmission after discharge. On eICU, it is used for ICU mortality, remaining ICU length of stay as a 10-class categorical prediction problem, and phenotyping as multi-label prediction of 25 acute care phenotypes (Zhu et al., 8 Jul 2025). This positioning makes KnowRare a framework for outcome prediction rather than a disease-specific diagnostic engine.

2. Motivation: rarity, scarcity, and heterogeneity

The framework is motivated by two coupled statistical difficulties. The first is data scarcity. For many rare conditions, the available cohort size is only 10–15 patients in the entire dataset, which makes standard supervised learning unstable and prone to overfitting (Zhu et al., 8 Jul 2025). Training only on the rare target condition yields high variance, while pooling all conditions can dilute condition-specific signal and bias learning toward common-condition distributions (Zhu et al., 8 Jul 2025).

The second is intra-condition heterogeneity. Rare conditions in ICU data exhibit multimodal manifestations and high variability in laboratory and vital-sign trajectories. The supplementary PCA analysis reported in the paper shows that the standard deviation of PCA components is significantly larger for rare conditions than for common ones, indicating broader within-condition variation (Zhu et al., 8 Jul 2025). This undermines approaches that assume either homogeneous target cohorts or generic transferability across all conditions.

The paper situates existing approaches as insufficient in distinct ways. Standard supervised deep learning fails under extreme sample scarcity; few-shot and meta-learning methods do not explicitly encode clinical similarity between conditions; and standard domain adaptation can introduce negative transfer when it aligns clinically unrelated source and target conditions (Zhu et al., 8 Jul 2025). KnowRare is therefore built around a selective transfer principle: learn representations from all patients, but adapt only from conditions that are clinically similar according to a learned condition knowledge graph (Zhu et al., 8 Jul 2025).

A plausible implication is that KnowRare is best understood as a response to a failure mode of naive transfer learning in critical care: not all extra data are helpful, and domain relevance must itself be modelled rather than assumed.

3. Architecture and learning pipeline

KnowRare has three stages: data extraction and knowledge graph construction, condition-level representation learning, and rare-condition adaptation (Zhu et al., 8 Jul 2025). The pipeline begins with EHR extraction from MIMIC-III and eICU, including demographics, laboratory variables, vital signs, diagnoses, and drug records, followed by preprocessing with time-window aggregation into 24 time steps per admission, imputation, and normalisation (Zhu et al., 8 Jul 2025).

The temporal backbone is an LSTM-based encoder. The primary temporal encoder $f^{\text{temp}}$ is a single-layer LSTM with hidden dimension 128, operating on multivariate time-series $\mathbf{X}_p \in \mathbb{R}^{T \times D_x}$, where $T=24$ (Zhu et al., 8 Jul 2025). A separate contextual encoder $f^{\text{cont}}$ is a two-layer MLP with LeakyReLU and hidden dimension 128, taking demographic variables $\mathbf{C}_p \in \mathbb{R}^{D_c}$ as input (Zhu et al., 8 Jul 2025). At each time step,

$$\mathbf{h}_t = f^{\text{proj}}\bigl(f^{\text{temp}}(\mathbf{x}_t),\; f^{\text{cont}}(\mathbf{C}_p)\bigr),$$

where $f^{\text{proj}}$ is a projection stage combining temporal and contextual features into a latent representation (Zhu et al., 8 Jul 2025).

Condition-agnostic pre-training is performed via a self-supervised next-step prediction task. A decoder $f^{\text{dec}}$, implemented as a two-layer MLP, predicts the next time-step variables: $$\hat{\mathbf{X}}_{t+1} = f^{\text{dec}}(\mathbf{h}_t),$$ with loss

$$\mathcal{L}_{\text{pre-train}} = \frac{1}{T-1} \sum_{t=1}^{T-1} \left\| \mathbf{X}_{t+1} - f^{\text{dec}}(\mathbf{h}_t) \right\|_2^2.$$

This stage is explicitly condition-agnostic, since it uses all patients without diagnosis-specific supervision, and is intended to learn generic ICU temporal dynamics before any rare-condition specialization (Zhu et al., 8 Jul 2025).

For downstream prediction, a classifier $\mathbf{X}_p \in \mathbb{R}^{T \times D_x}$0, also a two-layer MLP, maps the final latent representation $\mathbf{X}_p \in \mathbb{R}^{T \times D_x}$1 to task-specific outputs (Zhu et al., 8 Jul 2025). The supervised loss is cross-entropy,

$\mathbf{X}_p \in \mathbb{R}^{T \times D_x}$2

with inverse-prevalence weighting by condition to mitigate imbalance (Zhu et al., 8 Jul 2025).

4. Condition knowledge graph and selective transfer

A central component of KnowRare is the condition knowledge graph $\mathbf{X}_p \in \mathbb{R}^{T \times D_x}$3, in which nodes correspond to ICD-9-CM level-3 conditions (Zhu et al., 8 Jul 2025). The graph is not based on external ontologies such as UMLS or HPO; instead, it is data-driven and constructed from EHR-derived relations (Zhu et al., 8 Jul 2025).

Three relation types define the graph. The first is diagnosis co-occurrence. For conditions $\mathbf{X}_p \in \mathbb{R}^{T \times D_x}$4 and $\mathbf{X}_p \in \mathbb{R}^{T \times D_x}$5, the corresponding edge weight is

$\mathbf{X}_p \in \mathbb{R}^{T \times D_x}$6

which behaves as a conditional co-occurrence probability (Zhu et al., 8 Jul 2025). The second is record similarity, based on summary statistics of patient trajectories. If $\mathbf{X}_p \in \mathbb{R}^{T \times D_x}$7 is the condition-level profile, then

$\mathbf{X}_p \in \mathbb{R}^{T \times D_x}$8

Only the top 50% of these record-similarity edges are retained (Zhu et al., 8 Jul 2025). The third is drug similarity, computed as Jaccard overlap: $\mathbf{X}_p \in \mathbb{R}^{T \times D_x}$9 again retaining only the top 50% of edges (Zhu et al., 8 Jul 2025).

The graph is embedded using TuckER, which factorises the multi-relational tensor over conditions and relation types (Zhu et al., 8 Jul 2025). For a target rare condition $T=24$0, KnowRare computes cosine similarity in the learned embedding space,

$T=24$1

and selects the top-$T=24$2 most similar source conditions $T=24$3 (Zhu et al., 8 Jul 2025). The source set is thus not predefined by clinical taxonomy; it is learned from co-occurrence, temporal profile similarity, and therapeutic overlap.

The resulting source-selection behavior is clinically nontrivial. In the paper’s case studies, more than 95% of selected source conditions lie in different ICD-9 chapters than the target condition (Zhu et al., 8 Jul 2025). For Mycoses in MIMIC-III, many selected sources belong to circulatory system diseases; for Aplastic anaemia in eICU, many come from injury and poisoning (Zhu et al., 8 Jul 2025). This suggests that KnowRare privileges latent physiological and treatment-pattern similarity over taxonomic adjacency.

5. Multi-disease adaptation and objective

After pre-training and source selection, KnowRare performs multi-source domain adaptation from $T=24$4 to the rare target condition $T=24$5 (Zhu et al., 8 Jul 2025). The adaptation target is the joint distribution of latent representation and outcome label, $T=24$6, rather than only the marginal feature distribution (Zhu et al., 8 Jul 2025). This design reflects the observation that clinically similar conditions may still differ in how latent features map to risk.

To implement this, the framework introduces a joint discriminator $T=24$7, a two-layer MLP that takes the concatenation of latent representation and predicted outcome $T=24$8 and predicts the originating condition domain (Zhu et al., 8 Jul 2025). The adversarial loss is

$T=24$9

and the optimization problem is

$f^{\text{cont}}$0

so that the encoder and predictor learn to produce condition-invariant joint representations while the discriminator attempts to recover the source condition (Zhu et al., 8 Jul 2025).

The total adaptation objective is

$f^{\text{cont}}$1

where $f^{\text{cont}}$2 controls the balance between predictive supervision and domain alignment (Zhu et al., 8 Jul 2025). The paper searches $f^{\text{cont}}$3 by Bayesian optimisation (Zhu et al., 8 Jul 2025). Per-target training is then performed with early stopping on validation AUPRC (Zhu et al., 8 Jul 2025).

This design distinguishes KnowRare from more generic transfer methods. The adaptation is not one-to-one and not indiscriminate; it is condition-specific, multi-source, and knowledge-guided (Zhu et al., 8 Jul 2025). A plausible implication is that KnowRare can be viewed as learning a task-conditioned local neighborhood in condition space and transferring only within that neighborhood.

6. Empirical results, ablations, and significance

KnowRare is evaluated on MIMIC-III and eICU after preprocessing to 38,360 and 72,536 samples respectively, with train/validation/test splits of 67% / 16% / 17% stratified by condition (Zhu et al., 8 Jul 2025). Across all five tasks, it achieves the best AUPRC among compared methods (Zhu et al., 8 Jul 2025).

For MIMIC-III 90-day mortality, KnowRare reaches AUPRC 0.744 and AUROC 0.797, compared with AUPRC 0.737 for the best baseline, FADA (Zhu et al., 8 Jul 2025). For MIMIC-III 30-day readmission, it achieves AUPRC 0.716 and AUROC 0.749, exceeding AdvDiag and FADA (Zhu et al., 8 Jul 2025). On eICU ICU mortality, it reaches AUPRC 0.709 and AUROC 0.757, compared with AUPRC 0.631 and AUROC 0.691 for the best baseline, Stable-CRP (Zhu et al., 8 Jul 2025). For remaining ICU length of stay, KnowRare obtains AUPRC 0.206, compared with 0.176 for MetaPred (Zhu et al., 8 Jul 2025). For phenotyping, it achieves AUPRC 0.244 and AUROC 0.609, compared with AUPRC 0.237 and AUROC 0.656 for MetaPred, indicating better precision-recall behavior but not uniformly better AUROC (Zhu et al., 8 Jul 2025).

The comparison with ICU scoring systems is particularly explicit for ICU mortality in eICU. APACHE IV achieves AUPRC 0.639 and AUROC 0.701, APACHE IV-a achieves AUPRC 0.627 and AUROC 0.695, and KnowRare reaches AUPRC 0.709 and AUROC 0.757 (Zhu et al., 8 Jul 2025). This situates KnowRare not only against machine-learning baselines but also against established clinical scoring systems.

Ablation analysis shows that all three major components are necessary. Removing knowledge-guided domain selection causes the largest degradation: ICU mortality AUPRC falls from 0.709 to 0.573, remaining length-of-stay AUPRC from 0.206 to 0.065, and phenotyping AUPRC from 0.244 to 0.146 (Zhu et al., 8 Jul 2025). Removing condition-agnostic pre-training reduces 90-day mortality AUPRC from 0.744 to 0.640 and ICU mortality AUPRC from 0.709 to 0.628 (Zhu et al., 8 Jul 2025). Removing joint adversarial domain adaptation sharply harms some tasks, most notably 30-day readmission, where AUPRC drops from 0.716 to 0.481 (Zhu et al., 8 Jul 2025).

The source-selection case study further shows that performance peaks when using only about 10–20% of all conditions as candidate sources, rather than the full condition pool (Zhu et al., 8 Jul 2025). A separate KG sparsity study shows different optimal graph densities across datasets: around 5% edge retention in multi-centre eICU tasks, but 60–70% in single-centre MIMIC-III tasks (Zhu et al., 8 Jul 2025). This suggests that graph density mediates a trade-off between robustness and noise, and that optimal sparsification is dataset-dependent.

Finally, KnowRare is also tested on a common condition under limited data by restricting septicemia training data to 10% of its full size. In that setting, KnowRare can match or exceed an LSTM trained on 100% of septicemia data for several tasks (Zhu et al., 8 Jul 2025). This suggests that the framework’s value extends beyond formally rare conditions to any low-resource clinical prediction setting.

7. Interpretation, limitations, and context

KnowRare’s main interpretability asset is not patient-level feature attribution but condition-level transfer transparency (Zhu et al., 8 Jul 2025). The condition knowledge graph and TuckER embedding space make it possible to inspect which source conditions are selected for each target and how those relations vary by dataset and task (Zhu et al., 8 Jul 2025). The framework thereby offers a global explanation of transfer structure, even though it does not provide SHAP-style feature attributions over individual time steps or variables.

Several limitations are explicit. The system relies on ICD-9-CM level-3 codes, which are relatively coarse and tied to U.S. coding practice (Zhu et al., 8 Jul 2025). The MIMIC-III setting is constrained by discharge-time ICD coding, which restricts some task formulations (Zhu et al., 8 Jul 2025). The condition knowledge graph incorporates only EHR-derived similarity signals and does not include external ontologies such as UMLS or HPO (Zhu et al., 8 Jul 2025). Source conditions are selected once per target condition rather than dynamically per patient (Zhu et al., 8 Jul 2025). The framework is also evaluated only on retrospective U.S. ICU datasets, so broader generalisability remains unresolved (Zhu et al., 8 Jul 2025).

In the wider rare-condition and rare-disease literature, KnowRare occupies a distinct niche. It is not a rare-disease reasoning LLM, a phenotype-to-gene ranker, or a retrieval system. Instead, it is a task-specific outcome prediction framework for rare ICU conditions, built around condition-agnostic pre-training, learned inter-condition similarity, and selective domain adaptation (Zhu et al., 8 Jul 2025). This suggests a different operational role from systems aimed at rare-disease differential diagnosis: KnowRare is designed for risk prediction once a condition label is available, not for identifying the disease itself.

A plausible implication is that KnowRare is best viewed as an infrastructure for “borrowing strength” across clinically related conditions under extreme label sparsity. In that sense, it contributes a general principle to rare-condition modelling: representation sharing should be broad, but adaptation should be sparse, selective, and explicitly knowledge-guided (Zhu et al., 8 Jul 2025).