IGNITE: Personalized EHR Data Toolkit

• IGNITE is a deep generative toolkit designed to impute and synthesize sparse, irregular EHR data using patient-specific context.
• It employs a conditional dual-variational autoencoder with dual-stage attention and an individualized missingness mask to capture both observed and missing patterns.
• Validated on major ICU datasets, IGNITE outperforms traditional methods in missing-data reconstruction and downstream mortality prediction.

IGNITE, short for Individualized GeNeration of Imputations in Time-series Electronic health records, is a deep generative toolkit for sparse, irregular, and highly missing longitudinal electronic health records. It is designed for settings in which missingness is clinically informative rather than a nuisance variable, and it generates personalized, realistic values conditioned on an individual’s observed physiology, demographic characteristics, treatments, and missingness pattern. Architecturally, IGNITE combines a conditional dual-variational autoencoder with dual-stage attention, an individualized missingness mask, and additional latent-space constraints; empirically, it was validated on PhysioNet Challenge 2012, eICU, and HiRID, where it outperformed LOCF, MICE, GP-VAE, Transformer, BRITS, and SAITS in missing-data reconstruction and downstream mortality prediction (Ghosheh et al., 2024).

1. Clinical problem setting and representational assumptions

IGNITE is motivated by the observation that longitudinal EHRs are not merely incomplete; they are sparse and highly missing in ways shaped by clinical decision-making, patient severity, cost, feasibility, workflow, and related factors. The paper explicitly argues that missingness can itself reflect the underlying patient’s health status, so successful personalized modeling depends on how physiological data, treatments, and missing values are jointly represented rather than handled by a purely population-level imputer (Ghosheh et al., 2024).

The model therefore treats longitudinal EHR data as a joint structure consisting of continuous physiological time series, discrete treatment or intervention time series, static demographics, and individualized missingness. This formulation is central to the system’s scope. IGNITE is intended both to impute missing entries in an existing patient record and to serve as a personalized data synthesizer that can generate missing EHR values that were never observed prior, or even generate new patients for downstream applications such as risk prediction, digital twins, treatment planning, phenotyping, and related prediction tasks (Ghosheh et al., 2024).

A common misconception is to regard IGNITE as a conventional imputation baseline with a more elaborate neural architecture. The paper instead positions it as a generative model whose core objective is individualized generation of imputations conditioned on patient-specific context. Another misconception is that missingness should be marginalized away as noise. IGNITE is built on the opposite premise: missingness is often informative in ICU EHRs and should be modeled explicitly (Ghosheh et al., 2024).

2. Conditional dual-VAE architecture and dual-stage attention

IGNITE is built around a conditional dual-variational autoencoder architecture augmented with dual-stage attention. The model contains two VAEs. The first, denoted $VAE^{\mathcal{OO}}$, is trained on the observed-data representation. The second, denoted $VAE^{\mathcal{IMM}}$, is trained on data augmented with the individualized missingness mask. Both VAEs use LSTM-based encoders and decoders to map multivariate time series into a latent space and reconstruct them back into sequence space (Ghosheh et al., 2024).

The observed-data branch reconstructs the multivariate series using observed values only, with missing values filled with zero for the ELBO computation. The IMM branch uses a complete-data-style input in which missing entries are first filled by last observation carried forward and then masked with the individualized missingness mask. This dual-view construction is meant to force the latent representation to capture both what was measured and how that patient was measured (Ghosheh et al., 2024).

IGNITE’s attention mechanism is explicitly two-stage. In the encoder, feature-wise attention assigns a time-dependent importance weight to each variable:

$$e_{t}^{k}=\mathbf{v}_{e}^{\top} \tanh \left(\mathbf{W}_{e}\left[\mathbf{h}_{t-1} ; \mathbf{s}_{t-1}\right]+\mathbf{U}_{e} \mathbf{x}^{k}\right)$$

$$\alpha_{t}^{k}=\frac{\exp \left(e_{t}^{k}\right)}{\sum_{i=1}^{n} \exp \left(e_{t}^{i}\right)}$$

and the reweighted input is

$$\tilde{\mathbf{x}}_{t}=\left(\alpha_{t}^{1} x_{t}^{1}, \alpha_{t}^{2} x_{t}^{2}, \cdots, \alpha_{t}^{n} x_{t}^{n}\right)^{\top}.$$

This weighted vector is fed into the LSTM encoder. In the decoder, temporal attention attends over encoder hidden states across time:

$$l_{t}^{i}=\mathbf{v}_{d}^{\top} \tanh \left(\mathbf{W}_{d}\left[\mathbf{d}_{t-1} ; \mathbf{s}_{t-1}^{\prime}\right]+\mathbf{U}_{d} \mathbf{h}_{i}\right), \quad 1 \leq i \leq T$$

$$\beta_{t}^{i}=\frac{\exp \left(l_{t}^{i}\right)}{\sum_{j=1}^{T} \exp \left(l_{t}^{j}\right)}$$

with context vector

$$\mathbf{c}_{t}=\sum_{i=1}^{T} \beta_{t}^{i} \mathbf{h}_{i}.$$

The encoder thus emphasizes which features matter, while the decoder emphasizes which time steps matter (Ghosheh et al., 2024).

Conditioning enters through discrete treatment time series and static demographic information, specifically age and sex. The treatment matrix is concatenated with one-hot encoded demographic features, and these fully observed variables are used as conditioning inputs to the VAEs. The paper’s claim is that generated imputations should depend not only on observed physiology but also on demographic profile and treatment exposure (Ghosheh et al., 2024).

3. The individualized missingness mask

A key novelty in IGNITE is the Individualized Missingness Mask (IMM). The IMM is not a simple binary observed-or-missing indicator. It is designed to capture patient-specific missingness frequency and pattern across time for each feature. The paper’s motivation is that a binary mask cannot distinguish among features that are sometimes measured, never measured, or measured frequently in one patient and rarely in another, even though those regimes may carry distinct clinical meaning (Ghosheh et al., 2024).

The IMM is defined feature-wise for each patient as

$$\mathbf{IMM} = \begin{cases} 1 & \text{if } m_t^f = 1 \ \dfrac{\sum_t^T m_i^f}{T} & \text{if } m_t^f = 0 \end{cases}$$

where $m_t^f$ is the binary mask for feature $VAE^{\mathcal{IMM}}$0 at time $VAE^{\mathcal{IMM}}$1. In words, observed entries are assigned value $VAE^{\mathcal{IMM}}$2, while missing entries are assigned the observation frequency of that feature within that patient record (Ghosheh et al., 2024).

The paper explicitly compares this construction to TF-IDF-like ideas from NLP, in the sense that a signal’s importance depends on relative frequency within an instance. Here the analogy is not lexical but clinical: a feature’s missingness value depends on how often it is measured for that patient. This is especially important for feature-wise missingness, including cases in which a feature is never observed for a patient. LOCF and many standard imputers fail in that regime because there is no previous value to carry forward. IGNITE uses IMM to infer and generate values for such never-observed features from other patient information (Ghosheh et al., 2024).

The ablation study reported in the paper assigns particular significance to the IMM. On PhysioNet 2012, the progression from dual VAE without conditioning to conditioning, then to IMM, MIT loss, and discriminator yielded AUROC values of $VAE^{\mathcal{IMM}}$3, $VAE^{\mathcal{IMM}}$4, $VAE^{\mathcal{IMM}}$5, $VAE^{\mathcal{IMM}}$6, and $VAE^{\mathcal{IMM}}$7, respectively. The largest incremental gain is attributed to adding IMM (Ghosheh et al., 2024).

4. Training objectives and generative behavior

IGNITE is trained jointly with reconstruction, KL divergence, and several auxiliary latent-space losses. The two VAEs are encouraged to produce similar latent representations for the same patient through a matching loss. A semantic loss attaches a classifier to the concatenated latent representations and uses cross-entropy to predict the outcome label. A contrastive loss brings latent codes from the same patient closer and pushes those from different patients farther apart. The model also uses a Masked Imputation Task loss, which hides some observed values during training and asks the system to recover them using mean squared error on the artificially masked entries. In addition, an LSTM-based discriminator is trained adversarially to distinguish real from imputed samples, with the stated goal of making generated imputations indistinguishable from real data (Ghosheh et al., 2024).

The paper describes the total objective as a weighted sum of reconstruction, KL, matching, semantic, contrastive, MIT, and discriminator losses, with dataset-specific coefficients $VAE^{\mathcal{IMM}}$8. It also notes that the notation in the typeset expression is somewhat garbled, but the intended structure of the objective is clear (Ghosheh et al., 2024).

These objectives support two operational modes. In individualized imputation, IGNITE fills in missing entries for an existing patient given observed history, demographics, treatments, and IMM. In personalized data synthesis, the same generative machinery can create complete patient-like records, including features that were never observed for a patient, and can generate entirely new synthetic patients. This dual use is not an auxiliary feature; it is part of the model’s stated identity as both an imputor and a personalized data synthesizer (Ghosheh et al., 2024).

5. Validation datasets, baselines, and quantitative results

IGNITE was evaluated on three large publicly available ICU datasets, all aggregated into hourly bins over 48 time steps. PhysioNet Challenge 2012 contains 12,000 encounters, 35 physiological features, 1 treatment feature, a mortality outcome, and overall missingness greater than $VAE^{\mathcal{IMM}}$9. eICU contains 54,423 patient encounters, 55 physiological features, and 3 treatment features: oxygen therapy, vasopressors, and antibiotics. HiRID is a single-center Swiss ICU dataset with 50 physiological features and 7 treatment features: oxygen therapy, crystalloids, vasopressors, vasodilators, insulin, painkillers, and anticoagulants. Baselines were LOCF, MICE, GP-VAE, Transformer, BRITS, and SAITS (Ghosheh et al., 2024).

Dataset	Structure	Reported full-population mortality performance
PhysioNet 2012	12,000 encounters; 35 physiological; 1 treatment	AUROC 0.835; AUPRC 0.495
eICU	54,423 encounters; 55 physiological; 3 treatments	AUROC 0.774; AUPRC 0.389
HiRID	50 physiological; 7 treatments	AUROC 0.968; AUPRC 0.892

On the full-population mortality prediction task, IGNITE was best overall on all three datasets. For PhysioNet 2012, the next best AUROC values were SAITS at $$e_{t}^{k}=\mathbf{v}_{e}^{\top} \tanh \left(\mathbf{W}_{e}\left[\mathbf{h}_{t-1} ; \mathbf{s}_{t-1}\right]+\mathbf{U}_{e} \mathbf{x}^{k}\right)$$0 and Transformer at $$e_{t}^{k}=\mathbf{v}_{e}^{\top} \tanh \left(\mathbf{W}_{e}\left[\mathbf{h}_{t-1} ; \mathbf{s}_{t-1}\right]+\mathbf{U}_{e} \mathbf{x}^{k}\right)$$1, while LOCF was worst at $$e_{t}^{k}=\mathbf{v}_{e}^{\top} \tanh \left(\mathbf{W}_{e}\left[\mathbf{h}_{t-1} ; \mathbf{s}_{t-1}\right]+\mathbf{U}_{e} \mathbf{x}^{k}\right)$$2. The paper also emphasizes robustness under structured missingness. Under feature-wise missingness, it reports, for example, PhysioNet with $$e_{t}^{k}=\mathbf{v}_{e}^{\top} \tanh \left(\mathbf{W}_{e}\left[\mathbf{h}_{t-1} ; \mathbf{s}_{t-1}\right]+\mathbf{U}_{e} \mathbf{x}^{k}\right)$$3 feature-wise missingness at AUROC $$e_{t}^{k}=\mathbf{v}_{e}^{\top} \tanh \left(\mathbf{W}_{e}\left[\mathbf{h}_{t-1} ; \mathbf{s}_{t-1}\right]+\mathbf{U}_{e} \mathbf{x}^{k}\right)$$4 and AUPRC $$e_{t}^{k}=\mathbf{v}_{e}^{\top} \tanh \left(\mathbf{W}_{e}\left[\mathbf{h}_{t-1} ; \mathbf{s}_{t-1}\right]+\mathbf{U}_{e} \mathbf{x}^{k}\right)$$5, eICU with $$e_{t}^{k}=\mathbf{v}_{e}^{\top} \tanh \left(\mathbf{W}_{e}\left[\mathbf{h}_{t-1} ; \mathbf{s}_{t-1}\right]+\mathbf{U}_{e} \mathbf{x}^{k}\right)$$6 at AUROC $$e_{t}^{k}=\mathbf{v}_{e}^{\top} \tanh \left(\mathbf{W}_{e}\left[\mathbf{h}_{t-1} ; \mathbf{s}_{t-1}\right]+\mathbf{U}_{e} \mathbf{x}^{k}\right)$$7 and AUPRC $$e_{t}^{k}=\mathbf{v}_{e}^{\top} \tanh \left(\mathbf{W}_{e}\left[\mathbf{h}_{t-1} ; \mathbf{s}_{t-1}\right]+\mathbf{U}_{e} \mathbf{x}^{k}\right)$$8, and HiRID with $$e_{t}^{k}=\mathbf{v}_{e}^{\top} \tanh \left(\mathbf{W}_{e}\left[\mathbf{h}_{t-1} ; \mathbf{s}_{t-1}\right]+\mathbf{U}_{e} \mathbf{x}^{k}\right)$$9 at AUROC $$\alpha_{t}^{k}=\frac{\exp \left(e_{t}^{k}\right)}{\sum_{i=1}^{n} \exp \left(e_{t}^{i}\right)}$$0 and AUPRC $$\alpha_{t}^{k}=\frac{\exp \left(e_{t}^{k}\right)}{\sum_{i=1}^{n} \exp \left(e_{t}^{i}\right)}$$1. Under sample-wise missingness, reported examples include PhysioNet at $$\alpha_{t}^{k}=\frac{\exp \left(e_{t}^{k}\right)}{\sum_{i=1}^{n} \exp \left(e_{t}^{i}\right)}$$2 missingness with AUROC $$\alpha_{t}^{k}=\frac{\exp \left(e_{t}^{k}\right)}{\sum_{i=1}^{n} \exp \left(e_{t}^{i}\right)}$$3, HiRID at $$\alpha_{t}^{k}=\frac{\exp \left(e_{t}^{k}\right)}{\sum_{i=1}^{n} \exp \left(e_{t}^{i}\right)}$$4 with AUROC $$\alpha_{t}^{k}=\frac{\exp \left(e_{t}^{k}\right)}{\sum_{i=1}^{n} \exp \left(e_{t}^{i}\right)}$$5, and eICU at $$\alpha_{t}^{k}=\frac{\exp \left(e_{t}^{k}\right)}{\sum_{i=1}^{n} \exp \left(e_{t}^{i}\right)}$$6 with AUROC $$\alpha_{t}^{k}=\frac{\exp \left(e_{t}^{k}\right)}{\sum_{i=1}^{n} \exp \left(e_{t}^{i}\right)}$$7 and AUPRC $$\alpha_{t}^{k}=\frac{\exp \left(e_{t}^{k}\right)}{\sum_{i=1}^{n} \exp \left(e_{t}^{i}\right)}$$8 (Ghosheh et al., 2024).

For the reconstruction task with randomly introduced missingness at $$\alpha_{t}^{k}=\frac{\exp \left(e_{t}^{k}\right)}{\sum_{i=1}^{n} \exp \left(e_{t}^{i}\right)}$$9, $$\tilde{\mathbf{x}}_{t}=\left(\alpha_{t}^{1} x_{t}^{1}, \alpha_{t}^{2} x_{t}^{2}, \cdots, \alpha_{t}^{n} x_{t}^{n}\right)^{\top}.$$0, and $$\tilde{\mathbf{x}}_{t}=\left(\alpha_{t}^{1} x_{t}^{1}, \alpha_{t}^{2} x_{t}^{2}, \cdots, \alpha_{t}^{n} x_{t}^{n}\right)^{\top}.$$1, IGNITE achieved the lowest reconstruction error. Reported PhysioNet values were RMSE/MAE of $$\tilde{\mathbf{x}}_{t}=\left(\alpha_{t}^{1} x_{t}^{1}, \alpha_{t}^{2} x_{t}^{2}, \cdots, \alpha_{t}^{n} x_{t}^{n}\right)^{\top}.$$2, $$\tilde{\mathbf{x}}_{t}=\left(\alpha_{t}^{1} x_{t}^{1}, \alpha_{t}^{2} x_{t}^{2}, \cdots, \alpha_{t}^{n} x_{t}^{n}\right)^{\top}.$$3, and $$\tilde{\mathbf{x}}_{t}=\left(\alpha_{t}^{1} x_{t}^{1}, \alpha_{t}^{2} x_{t}^{2}, \cdots, \alpha_{t}^{n} x_{t}^{n}\right)^{\top}.$$4. HiRID reported $$\tilde{\mathbf{x}}_{t}=\left(\alpha_{t}^{1} x_{t}^{1}, \alpha_{t}^{2} x_{t}^{2}, \cdots, \alpha_{t}^{n} x_{t}^{n}\right)^{\top}.$$5, $$\tilde{\mathbf{x}}_{t}=\left(\alpha_{t}^{1} x_{t}^{1}, \alpha_{t}^{2} x_{t}^{2}, \cdots, \alpha_{t}^{n} x_{t}^{n}\right)^{\top}.$$6, and $$\tilde{\mathbf{x}}_{t}=\left(\alpha_{t}^{1} x_{t}^{1}, \alpha_{t}^{2} x_{t}^{2}, \cdots, \alpha_{t}^{n} x_{t}^{n}\right)^{\top}.$$7. eICU reported $$\tilde{\mathbf{x}}_{t}=\left(\alpha_{t}^{1} x_{t}^{1}, \alpha_{t}^{2} x_{t}^{2}, \cdots, \alpha_{t}^{n} x_{t}^{n}\right)^{\top}.$$8, $$\tilde{\mathbf{x}}_{t}=\left(\alpha_{t}^{1} x_{t}^{1}, \alpha_{t}^{2} x_{t}^{2}, \cdots, \alpha_{t}^{n} x_{t}^{n}\right)^{\top}.$$9, and $$l_{t}^{i}=\mathbf{v}_{d}^{\top} \tanh \left(\mathbf{W}_{d}\left[\mathbf{d}_{t-1} ; \mathbf{s}_{t-1}^{\prime}\right]+\mathbf{U}_{d} \mathbf{h}_{i}\right), \quad 1 \leq i \leq T$$0. The paper highlights the stability of these errors as missingness increases as evidence of robustness (Ghosheh et al., 2024).

6. Significance, limitations, and relation to the broader toolkit literature

The paper presents IGNITE as a step toward individualized clinical modeling and digital twins because it generates patient-specific imputations conditioned on observed history, treatments, demographics, and missingness. Its stated practical implications include more accurate risk prediction, patient-level phenotyping, treatment optimization, synthetic data generation, filling in never-measured features, and better use of incomplete EHRs in precision medicine. At the same time, the study’s limitations are clearly stated: evaluation was restricted to retrospective ICU datasets; downstream assessment was limited to mortality prediction; and the model was not tested on primary care or wearable data. Future work is said to include treatment recommendation, phenotyping, and deeper theoretical analysis of missingness patterns (Ghosheh et al., 2024).

Within the broader literature on research data toolkits, IGNITE occupies a distinct niche. Other recent systems center on retrieval, integration, metadata governance, or scalable preprocessing rather than individualized generative modeling. FireDataForge is an event-centered Python framework that automates retrieval and harmonization of 11 wildfire-related sources into analysis-ready NumPy arrays with embedded metadata (Xia et al., 19 Jun 2026). DIALITE is a pipeline for discovering, aligning, integrating, and analyzing open data tables through discovery methods such as SANTOS and LSH Ensemble together with ALITE-based full disjunction (Khatiwada et al., 2023). “Data Lake Ingestion Management” describes a threefold solution consisting of a metadata model, ingestion algorithms, and a metadata management system implemented with a web application and a Neo4j graph database (Zhao et al., 2021). The echemdb toolkit is a lightweight, file-system-based approach that annotates raw research data with YAML sidecar metadata and converts them into Frictionless Data Packages (Engstfeld et al., 2024). Data Prep Kit separates data access, transformation, and runtime, supports Pure Python, Ray, and Spark, and is intended for LLM pretraining, fine-tuning, and RAG workflows (Wood et al., 2024). “Extensible Data Skipping” focuses on metadata-driven pruning for arbitrary data types and UDF-based predicates in Apache Spark (Ta-Shma et al., 2020).

These systems clarify what “data toolkit” can mean across domains. Some toolkits standardize ingestion and metadata; others automate harmonization or distributed preprocessing. IGNITE differs because its principal transformation is patient-specific generative inference over missing longitudinal EHRs rather than dataset onboarding, schema integration, or data-lake acceleration. This suggests that, in the EHR setting, a “toolkit” may designate not merely infrastructure for moving data, but an operational framework for representing patient trajectories, individualized missingness, and downstream predictive utility within a single learned model (Ghosheh et al., 2024).