Informative Missingness to Generate Irregular Clinical Time Series

Abstract: Laboratory tests in electronic health records are collected irregularly, and the absence of a test order can be as informative as the measurement itself. Such missingness reflects clinicians' decisions and patient physiology, making it important to model it directly rather than treat it as a preprocessing artifact. Here we present a diffusion-based approach for generating clinical time series that jointly models laboratory values and their observation patterns using the public Data Analytics Challenge on Missing Data Imputation (DACMI) benchmark derived from MIMIC-III. To preserve realistic sampling, we align chart times into 4-hour intervals and segment admissions into 7-day windows, producing trajectories that pair each lab value with a corresponding observation indicator. Standard transformations and normalization are applied to stabilize training. Our method extends the TimeDiff framework to learn continuous lab values and discrete missingness patterns through complementary diffusion objectives. Experiments show that the generated data closely match real patient trajectories across individual lab distributions and joint value-missingness embeddings, demonstrating that diffusion models can capture clinically meaningful dependencies between patient physiology and clinicians' testing behavior under MNAR-like (missing-not-at-random) missingness. These preliminary results indicate that our model can serve as an initial component toward developing clinical foundation models. By producing synthetic priors that preserve key physiology-missingness relationships, this work motivates the subsequent training of Prior-Data Fitted Networks capable of leveraging informative missingness, which we will investigate in the extended work.

• The paper adapts diffusion modeling to jointly generate 13 laboratory values and their observation masks, using masked loss functions and recency-based delta conditioning for MNAR-like clinical data.
• Delta conditioning modestly improves fidelity, reducing mean EMD from 0.11 to 0.10 and mean DTW distance from 37.99 to 37.72 compared with the unconditioned model.
• The model reproduces laboratory distributions and trajectory structure well but underestimates missingness prevalence at 0.74 versus 0.79 in real data, limiting its current use for downstream clinical priors.

Overview

This paper addresses a persistent gap in synthetic electronic health record (EHR) generation: most generative models for clinical time series reproduce measurement values well but treat missingness as a nuisance to be imputed or summarized by marginal rates. The authors argue that in laboratory data, absence of an observation is itself informative—reflecting clinician ordering behavior and patient physiology under Missing-Not-At-Random (MNAR) conditions—and therefore must be generated jointly with the values. They adapt the TimeDiff diffusion framework [tian_reliable_nodate] to jointly generate 13 laboratory value channels and their binary observation masks on the public DACMI benchmark derived from MIMIC-III [luo_evaluating_2022, johnson_mimic-iii_2016], positioning the resulting synthetic priors as a preliminary component for training Prior-Data Fitted Networks [muller_transformers_2021].

Method

The pipeline operates on the DACMI training split, extracting 13 common blood labs per admission together with a missingness mask constructed directly from raw tables. Chart times are discretized into 4-hour intervals, chosen to minimize artificially induced missingness, and admissions are segmented into non-overlapping 7-day windows of 42 steps, yielding 16,580 training samples from 8,267 admissions. Each window is represented as 26 channels: 13 standardized lab values (after log or Box-Cox transformations followed by z-scoring) and 13 binary mask channels.

Two modifications distinguish this work from vanilla TimeDiff. First, the Gaussian denoising loss on numerical channels excludes time points where the lab is truly missing, so that missingness is treated as structural absence rather than as noise on the value channel. Second, delta conditioning supplies an auxiliary tensor encoding recency of observation—the number of steps since each lab was last observed, normalized by the window length—as context to the denoising network without adding it to the generated output. The model trains for 200,000 steps with Adam (learning rate $8\times10^{-5}$, batch size 64), weighting the discrete missingness loss at $\lambda=10^{-5}$ relative to the continuous loss; 16,000 synthetic windows are generated for evaluation.

Evaluation and results

Evaluation uses three complementary views: univariate density matching, joint embedding overlap, and per-feature distributional metrics. Qualitatively, synthetic densities track real histograms closely for both a concentrated variable (hemoglobin) and a heavily right-skewed one (white blood cell count), including tail behavior. A UMAP projection using dynamic time warping (DTW) distance over all 26 channels shows substantial overlap between real and synthetic window distributions, with no synthetic mass outside the support of the real data.

The quantitative comparison against a Real–Real reference baseline supports two claims:

Metric (mean over 13 labs)	Real–Real	Synth w/ Δ	Synth w/o Δ
EMD (univariate marginals)	0.05	0.10	0.11
DTW trajectory distance	37.65	37.72	37.99
Missingness prevalence	0.79	0.74	0.76

Delta conditioning consistently improves agreement: mean EMD drops from 0.11 to 0.10 and mean DTW from 37.99 to 37.72, moving closer to the same-domain Real–Real reference. The effect is modest but uniform across labs, suggesting observation recency carries useful signal for the denoiser.

A notable negative result accompanies these figures: synthetic missingness prevalence is systematically below the observed rate (mean 0.74 vs. 0.79), across every lab and both conditioning settings. The authors concede that the model underestimates how often measurements are absent, indicating that the discrete missingness generation process remains the weakest link despite the joint modeling objective. This is a meaningful limitation given that informative missingness is the paper's central motivation—an under-generation of missingness would bias any downstream prior toward denser sampling than clinicians actually exhibit.

Limitations and open questions

The paper is explicit that results constitute preliminary evidence rather than validation of downstream clinical utility. Several open issues follow directly from the reported findings. First, the systematic underestimate of missingness prevalence raises the question of whether the $\lambda=10^{-5}$ loss weighting, adopted from TimeDiff's notation, is appropriate for the highly imbalanced mask objective, or whether alternative formulations (e.g., class-weighted or focal losses) would close the gap. Second, evaluation relies on marginal and embedding-level similarity metrics; whether the model reproduces burst-like ordering patterns and value–mask coupling at the level of individual trajectories—not just aggregate distributions—is asserted via UMAP/DTW overlap but not tested with targeted dependence measures. Third, the claimed utility as a synthetic prior for Prior-Data Fitted Networks trained on imputation and forecasting under distribution shift is deferred to extended work and remains untested here. Finally, all experiments use a single benchmark derived from a single ICU database, so generalization across sites and care settings—where informative missingness patterns are known to differ—is unaddressed.

Conclusion

This work demonstrates a practical adaptation of diffusion-based time-series generation to jointly model laboratory values and observation indicators under MNAR-like missingness, with a preprocessing protocol, a masked Gaussian loss, and delta-based recency conditioning as its concrete technical contributions. Distributional fidelity is strong for lab values and joint embeddings, while missingness frequency remains systematically underestimated—a candidly acknowledged shortcoming. The results establish a reasonable foundation for the stated next step: using such generators to produce priors for Prior-Data Fitted Networks that exploit informative missingness in downstream inference.