---
title: 'KnowRare: Deep Learning for Rare ICU Outcomes'
url: https://www.emergentmind.com/topics/knowrare
type: topic
---

# KnowRare: Deep Learning for Rare ICU Outcomes

Searching arXiv for "KnowRare" and the associated ICU rare-condition prediction paper to ground the article in current arXiv metadata.
KnowRare is a domain adaptation-based deep learning framework for predicting clinical outcomes for rare conditions in the intensive care unit (ICU). It is designed for settings in which formally recognised rare diseases and low-prevalence ICU conditions are both affected by severe data scarcity and substantial intra-condition heterogeneity. The framework addresses these constraints by first learning condition-agnostic representations from diverse electronic health records through self-supervised pre-training, then selectively adapting knowledge from clinically similar conditions using a condition knowledge graph. Across MIMIC-III and eICU, KnowRare is evaluated on 90-day mortality, 30-day readmission, ICU mortality, remaining length of stay, and phenotyping, and is reported to outperform existing state-of-the-art models as well as APACHE IV and IV-a on ICU mortality [2507.06432].

## 1. Definition and problem setting

KnowRare is formulated around prediction for **rare conditions in the ICU**, where “rare conditions” include both formally recognised rare diseases and **low-prevalence conditions** in critical care [2507.06432]. The framework operates at the **condition level**: each patient is assigned a primary diagnosis using ICD-9-CM level-3 codes, and prediction is evaluated on cohorts defined by those conditions [2507.06432].

Within each dataset, a condition is labelled *Rare* if its prevalence is **< 1 in 2,000 patients** and if it has at least one positive label for the relevant prediction task [2507.06432]. From all such candidates, the study selects the **10 least prevalent conditions** satisfying these criteria in each dataset for evaluation [2507.06432]. The underlying rarity is pervasive at the code level: in MIMIC-III, **383 of 587** level-3 ICD-9 codes meet this definition, and in eICU, **192 of 303** codes do so [2507.06432].

The targeted prediction tasks are dataset-specific. On **MIMIC-III**, KnowRare is used for **90-day mortality** after hospital discharge and **30-day readmission** after discharge. On **eICU**, it is used for **ICU mortality**, **remaining ICU length of stay** as a 10-class categorical prediction problem, and **phenotyping** as multi-label prediction of 25 acute care phenotypes [2507.06432]. This positioning makes KnowRare a framework for outcome prediction rather than a disease-specific diagnostic engine.

## 2. Motivation: rarity, scarcity, and heterogeneity

The framework is motivated by two coupled statistical difficulties. The first is **data scarcity**. For many rare conditions, the available cohort size is only **10–15 patients** in the entire dataset, which makes standard supervised learning unstable and prone to overfitting [2507.06432]. Training only on the rare target condition yields high variance, while pooling all conditions can dilute condition-specific signal and bias learning toward common-condition distributions [2507.06432].

The second is **intra-condition heterogeneity**. Rare conditions in ICU data exhibit multimodal manifestations and high variability in laboratory and vital-sign trajectories. The supplementary PCA analysis reported in the paper shows that the **standard deviation of PCA components** is significantly larger for rare conditions than for common ones, indicating broader within-condition variation [2507.06432]. This undermines approaches that assume either homogeneous target cohorts or generic transferability across all conditions.

The paper situates existing approaches as insufficient in distinct ways. Standard supervised deep learning fails under extreme sample scarcity; few-shot and meta-learning methods do not explicitly encode clinical similarity between conditions; and standard domain adaptation can introduce **negative transfer** when it aligns clinically unrelated source and target conditions [2507.06432]. KnowRare is therefore built around a selective transfer principle: learn representations from all patients, but adapt only from conditions that are clinically similar according to a learned condition knowledge graph [2507.06432].

A plausible implication is that KnowRare is best understood as a response to a failure mode of naive transfer learning in critical care: not all extra data are helpful, and domain relevance must itself be modelled rather than assumed.

## 3. Architecture and learning pipeline

KnowRare has three stages: **data extraction and knowledge graph construction**, **condition-level representation learning**, and **rare-condition adaptation** [2507.06432]. The pipeline begins with EHR extraction from MIMIC-III and eICU, including demographics, laboratory variables, vital signs, diagnoses, and drug records, followed by preprocessing with time-window aggregation into **24 time steps per admission**, imputation, and normalisation [2507.06432].

The temporal backbone is an **LSTM-based encoder**. The primary temporal encoder \( f^{\text{temp}} \) is a **single-layer LSTM** with hidden dimension **128**, operating on multivariate time-series \( \mathbf{X}_p \in \mathbb{R}^{T \times D_x} \), where \(T=24\) [2507.06432]. A separate contextual encoder \( f^{\text{cont}} \) is a **two-layer MLP** with LeakyReLU and hidden dimension **128**, taking demographic variables \( \mathbf{C}_p \in \mathbb{R}^{D_c} \) as input [2507.06432]. At each time step,
\[
\mathbf{h}_t = f^{\text{proj}}\bigl(f^{\text{temp}}(\mathbf{x}_t),\; f^{\text{cont}}(\mathbf{C}_p)\bigr),
\]
where \( f^{\text{proj}} \) is a projection stage combining temporal and contextual features into a latent representation [2507.06432].

Condition-agnostic pre-training is performed via a **self-supervised next-step prediction task**. A decoder \( f^{\text{dec}} \), implemented as a two-layer MLP, predicts the next time-step variables:
\[
\hat{\mathbf{X}}_{t+1} = f^{\text{dec}}(\mathbf{h}_t),
\]
with loss
\[
\mathcal{L}_{\text{pre-train}} = \frac{1}{T-1} \sum_{t=1}^{T-1} \left\| \mathbf{X}_{t+1} - f^{\text{dec}}(\mathbf{h}_t) \right\|_2^2.
\]
This stage is explicitly **condition-agnostic**, since it uses all patients without diagnosis-specific supervision, and is intended to learn generic ICU temporal dynamics before any rare-condition specialization [2507.06432].

For downstream prediction, a classifier \( f_\theta \), also a **two-layer MLP**, maps the final latent representation \( \mathbf{h}_T \) to task-specific outputs [2507.06432]. The supervised loss is cross-entropy,
\[
\mathcal{L}_{\text{pred}} = \mathcal{L}_{\text{CE}}\bigl(f_\theta(\mathbf{X}, \mathbf{C}), y\bigr),
\]
with inverse-prevalence weighting by condition to mitigate imbalance [2507.06432].

## 4. Condition knowledge graph and selective transfer

A central component of KnowRare is the **condition knowledge graph** \( \mathcal{G} = (\mathcal{V}, \mathcal{E}) \), in which nodes correspond to ICD-9-CM level-3 conditions [2507.06432]. The graph is not based on external ontologies such as UMLS or HPO; instead, it is **data-driven** and constructed from EHR-derived relations [2507.06432].

Three relation types define the graph. The first is **diagnosis co-occurrence**. For conditions \(v_i\) and \(v_j\), the corresponding edge weight is
\[
w_{ij}^{(r_1)} = \frac{\text{CoOcc}(v_i, v_j)}{\sum_k \text{CoOcc}(v_i, v_k)},
\]
which behaves as a conditional co-occurrence probability [2507.06432]. The second is **record similarity**, based on summary statistics of patient trajectories. If \(\mathbf{s}_v = [\operatorname{mean}(\mathcal{X}_v), \operatorname{std}(\mathcal{X}_v)]\) is the condition-level profile, then
\[
w_{ij}^{(r_2)} = \frac{1}{1 + \left\| \mathbf{s}_{v_i} - \mathbf{s}_{v_j} \right\|_2}.
\]
Only the **top 50%** of these record-similarity edges are retained [2507.06432]. The third is **drug similarity**, computed as Jaccard overlap:
\[
w_{ij}^{(r_3)} = \frac{|\mathcal{D}_{v_i} \cap \mathcal{D}_{v_j}|}{|\mathcal{D}_{v_i} \cup \mathcal{D}_{v_j}|},
\]
again retaining only the **top 50%** of edges [2507.06432].

The graph is embedded using **TuckER**, which factorises the multi-relational tensor over conditions and relation types [2507.06432]. For a target rare condition \(v_t\), KnowRare computes cosine similarity in the learned embedding space,
\[
\operatorname{sim}(v_t, v_s) = \frac{\mathbf{E}_{v_t} \cdot \mathbf{E}_{v_s}}{\|\mathbf{E}_{v_t}\|_2 \, \|\mathbf{E}_{v_s}\|_2},
\]
and selects the top-\(k\) most similar source conditions \( \mathcal{S}^{*} \) [2507.06432]. The source set is thus not predefined by clinical taxonomy; it is learned from co-occurrence, temporal profile similarity, and therapeutic overlap.

The resulting source-selection behavior is clinically nontrivial. In the paper’s case studies, **more than 95%** of selected source conditions lie in different ICD-9 chapters than the target condition [2507.06432]. For **Mycoses** in MIMIC-III, many selected sources belong to **circulatory system diseases**; for **Aplastic anaemia** in eICU, many come from **injury and poisoning** [2507.06432]. This suggests that KnowRare privileges latent physiological and treatment-pattern similarity over taxonomic adjacency.

## 5. Multi-disease adaptation and objective

After pre-training and source selection, KnowRare performs **multi-source domain adaptation** from \( \mathcal{S}^{*} \cup \{v_t\} \) to the rare target condition \(v_t\) [2507.06432]. The adaptation target is the **joint distribution** of latent representation and outcome label, \( P(\mathbf{h}_T, y) \), rather than only the marginal feature distribution [2507.06432]. This design reflects the observation that clinically similar conditions may still differ in how latent features map to risk.

To implement this, the framework introduces a **joint discriminator** \( d_\phi \), a two-layer MLP that takes the concatenation of latent representation and predicted outcome \((\mathbf{h}_T, \hat{y})\) and predicts the originating condition domain [2507.06432]. The adversarial loss is
\[
\mathcal{L}_{\text{adv}} = - \mathbb{E}_{(\mathbf{h}_T, \hat{y})} \sum_{i=1}^{|\mathcal{S}^{*}|+1} y_{\mathcal{D}_i} \log d_\phi(\mathbf{h}_T, \hat{y}),
\]
and the optimization problem is
\[
\min_{\theta}\;\max_{\phi}\; \mathcal{L}_{\text{adv}}(\mathbf{h}_T, \hat{y}; \theta, \phi),
\]
so that the encoder and predictor learn to produce condition-invariant joint representations while the discriminator attempts to recover the source condition [2507.06432].

The total adaptation objective is
\[
\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{pred}} + \lambda \, \mathcal{L}_{\text{adv}},
\]
where \( \lambda \) controls the balance between predictive supervision and domain alignment [2507.06432]. The paper searches \( \lambda \in \{0.005, 0.01, 0.02, 0.1\} \) by Bayesian optimisation [2507.06432]. Per-target training is then performed with early stopping on validation AUPRC [2507.06432].

This design distinguishes KnowRare from more generic transfer methods. The adaptation is not one-to-one and not indiscriminate; it is **condition-specific**, **multi-source**, and **knowledge-guided** [2507.06432]. A plausible implication is that KnowRare can be viewed as learning a task-conditioned local neighborhood in condition space and transferring only within that neighborhood.

## 6. Empirical results, ablations, and significance

KnowRare is evaluated on **MIMIC-III** and **eICU** after preprocessing to **38,360** and **72,536** samples respectively, with train/validation/test splits of **67% / 16% / 17%** stratified by condition [2507.06432]. Across all five tasks, it achieves the best AUPRC among compared methods [2507.06432].

For **MIMIC-III 90-day mortality**, KnowRare reaches **AUPRC 0.744** and **AUROC 0.797**, compared with **AUPRC 0.737** for the best baseline, FADA [2507.06432]. For **MIMIC-III 30-day readmission**, it achieves **AUPRC 0.716** and **AUROC 0.749**, exceeding AdvDiag and FADA [2507.06432]. On **eICU ICU mortality**, it reaches **AUPRC 0.709** and **AUROC 0.757**, compared with **AUPRC 0.631** and **AUROC 0.691** for the best baseline, Stable-CRP [2507.06432]. For **remaining ICU length of stay**, KnowRare obtains **AUPRC 0.206**, compared with **0.176** for MetaPred [2507.06432]. For **phenotyping**, it achieves **AUPRC 0.244** and **AUROC 0.609**, compared with **AUPRC 0.237** and **AUROC 0.656** for MetaPred, indicating better precision-recall behavior but not uniformly better AUROC [2507.06432].

The comparison with ICU scoring systems is particularly explicit for ICU mortality in eICU. **APACHE IV** achieves **AUPRC 0.639** and **AUROC 0.701**, **APACHE IV-a** achieves **AUPRC 0.627** and **AUROC 0.695**, and KnowRare reaches **AUPRC 0.709** and **AUROC 0.757** [2507.06432]. This situates KnowRare not only against machine-learning baselines but also against established clinical scoring systems.

Ablation analysis shows that all three major components are necessary. Removing **knowledge-guided domain selection** causes the largest degradation: ICU mortality AUPRC falls from **0.709 to 0.573**, remaining length-of-stay AUPRC from **0.206 to 0.065**, and phenotyping AUPRC from **0.244 to 0.146** [2507.06432]. Removing **condition-agnostic pre-training** reduces 90-day mortality AUPRC from **0.744 to 0.640** and ICU mortality AUPRC from **0.709 to 0.628** [2507.06432]. Removing **joint adversarial domain adaptation** sharply harms some tasks, most notably 30-day readmission, where AUPRC drops from **0.716 to 0.481** [2507.06432].

The source-selection case study further shows that performance peaks when using only about **10–20% of all conditions** as candidate sources, rather than the full condition pool [2507.06432]. A separate KG sparsity study shows different optimal graph densities across datasets: around **5%** edge retention in multi-centre eICU tasks, but **60–70%** in single-centre MIMIC-III tasks [2507.06432]. This suggests that graph density mediates a trade-off between robustness and noise, and that optimal sparsification is dataset-dependent.

Finally, KnowRare is also tested on a common condition under limited data by restricting **septicemia** training data to **10%** of its full size. In that setting, KnowRare can match or exceed an LSTM trained on **100%** of septicemia data for several tasks [2507.06432]. This suggests that the framework’s value extends beyond formally rare conditions to any low-resource clinical prediction setting.

## 7. Interpretation, limitations, and context

KnowRare’s main interpretability asset is not patient-level feature attribution but **condition-level transfer transparency** [2507.06432]. The condition knowledge graph and TuckER embedding space make it possible to inspect which source conditions are selected for each target and how those relations vary by dataset and task [2507.06432]. The framework thereby offers a global explanation of transfer structure, even though it does not provide SHAP-style feature attributions over individual time steps or variables.

Several limitations are explicit. The system relies on **ICD-9-CM level-3 codes**, which are relatively coarse and tied to U.S. coding practice [2507.06432]. The MIMIC-III setting is constrained by discharge-time ICD coding, which restricts some task formulations [2507.06432]. The condition knowledge graph incorporates only **EHR-derived** similarity signals and does **not** include external ontologies such as UMLS or HPO [2507.06432]. Source conditions are selected once per target condition rather than dynamically per patient [2507.06432]. The framework is also evaluated only on retrospective U.S. ICU datasets, so broader generalisability remains unresolved [2507.06432].

In the wider rare-condition and rare-disease literature, KnowRare occupies a distinct niche. It is not a rare-disease reasoning LLM, a phenotype-to-gene ranker, or a retrieval system. Instead, it is a **task-specific outcome prediction framework for rare ICU conditions**, built around condition-agnostic pre-training, learned inter-condition similarity, and selective domain adaptation [2507.06432]. This suggests a different operational role from systems aimed at rare-disease differential diagnosis: KnowRare is designed for **risk prediction once a condition label is available**, not for identifying the disease itself.

A plausible implication is that KnowRare is best viewed as an infrastructure for “borrowing strength” across clinically related conditions under extreme label sparsity. In that sense, it contributes a general principle to rare-condition modelling: representation sharing should be broad, but adaptation should be sparse, selective, and explicitly knowledge-guided [2507.06432].

Source: https://www.emergentmind.com/topics/knowrare