Health Gym: Acute Hypotension & Sepsis

• Health Gym Acute Hypotension/Sepsis is a comprehensive synthetic ICU dataset ecosystem that accurately simulates patient vitals, labs, and interventions using GAN-based generation methods.
• It incorporates robust privacy controls and rigorous statistical validations to ensure the synthetic data closely mimics real-world clinical distributions while minimizing disclosure risk.
• The platform supports reinforcement learning with explicitly defined state, action, and reward structures to benchmark algorithmic treatments for life-threatening circulatory and infection challenges.

Health Gym Acute Hypotension/Sepsis refers to a collection of openly available, high-fidelity synthetic datasets and benchmark environments designed to facilitate the development and evaluation of machine learning algorithms—especially reinforcement learning (RL)—for the management of acute hypotension and sepsis in intensive care unit (ICU) settings. Created and curated under the Health Gym project, these resources leverage clinically informed data generation, robust privacy controls, and explicit mappings to RL-ready state, action, reward structures, providing a reproducible platform for algorithmic research targeting life-threatening circulatory instability and infection-driven multiorgan dysfunction (Kuo et al., 2022, Kuo et al., 2021).

1. Dataset Design, Composition, and Generation

The Health Gym Hypotension/Sepsis datasets are based on real MIMIC-III ICU patient episodes, but all records are synthetic, generated by a Wasserstein GAN with gradient penalty (WGAN-GP, λ=10) supplemented by an alignment loss on Pearson correlation matrices to preserve inter-variable association structure (Kuo et al., 2022, Kuo et al., 2021).

Each dataset is formatted as a single CSV: every row encodes a patient ID, timepoint, and feature vector. Acute hypotension data contains 3,910 synthetic patients with 48 1-hour timesteps (187,680 rows) and sepsis contains 2,164 synthetic patients across 20 4-hour windows per patient (43,280 rows) (Kuo et al., 2021). Variables include core clinical measurements (vital signs, laboratory values, administered IV fluids, vasopressor dosing), categorical groupings (e.g., GCS, FiO₂ deciles), and binary “measured” flags for missingness. The data generation process ensures that the empirical feature distributions, bivariate/multivariate correlations, and temporal dynamics closely match those in the source MIMIC-III population (Kuo et al., 2022, Kuo et al., 2021).

2. Privacy Controls and Disclosure Risk

Statistical privacy is enforced at multiple levels. The Euclidean distance between all synthetic and real patient rows is strictly >0 (e.g., 49.06 for hypotension, 328.78 for sepsis) (Kuo et al., 2021). Linkage risk, following El Emam et al. (2020), is quantified as $R = \max(p_{PS}, p_{SP})$, with $p_{PS}$ the population-to-sample match rate and $p_{SP}$ the reverse; in the sepsis cohort, $R$ attains 0.045%, well below the European and Canadian public thresholds (9%) (Kuo et al., 2021). No formal differential privacy was applied, but the synthetic generation approach plus post hoc validation ensures negligible identity disclosure risk (Kuo et al., 2021, Kuo et al., 2022).

3. RL Environment Specification: State, Action, and Reward Schemes

State Space

For acute hypotension: $s_t \in \mathbb{R}^{20}$ numeric features (MAP, SBP, DBP, urine, lactate, creatinine, etc.), 4 one-hot-encoded categorical variables (fluid bolus, vasopressor bins, FiO₂, GCS), 7 missing-value indicators; after embedding, $s_t$ is typically $\sim$54D (Kuo et al., 2022, Kuo et al., 2021).

For sepsis: 35 numeric labs and vitals + 6 categorical + 3 binary = 44-83D post-embedding (Kuo et al., 2022, Kuo et al., 2021).

Action Space

Actions are discretized as joint bins over fluid bolus and vasopressor rates:

| Setting | Fluids Bins | Vasopressor Bins | Total Actions (|A|) | |-------------------|-------------|------------------|----------------| | Acute Hypotension | 4 | 4 | 16 | | Sepsis | 4 | 4 | 16 | | Expanded/research | up to 5 | 5 | 20 or 25 (varying studies)|

Actions are clinically meaningful: e.g., “fluid 0-200 mL/hr, vaso 5-15 mcg·kg⁻¹·hr⁻¹” (Futoma et al., 2020, Ma et al., 2023).

Reward Functions

Acute hypotension modeling adopts a reward $R(s_t, a_t, s_{t+1})$ based on proximity to MAP targets and penalizes large intervention doses:

$$R(s_t,a_t,s_{t+1}) = \alpha (\mathrm{MAP}_{t+1} - 65) - \beta |\mathrm{bolus}_t| - \gamma |\mathrm{vaso}_t|$$

with terminal rewards e.g. $+15$ for survival beyond 48 hours, $p_{PS}$0 for death (Kuo et al., 2022).

Sepsis modeling (Komorowski rl, Raghu reward) includes cluster-based risk rewards or SOFA/lactate change shaping:

$p_{PS}$1

with terminal $p_{PS}$2 for survival (Ma et al., 2023, Kuo et al., 2022).

4. Benchmarking and Models in Health Gym and Related Environments

The Health Gym environment is intentionally isomorphic to prior observational RL studies, including all variables needed for MDP or POMDP formulations (Kuo et al., 2022, Kuo et al., 2021). Multiple RL algorithms have been implemented and evaluated:

• Deep Q-Networks (DQN) and DRQN (LSTM-based): State $p_{PS}$3 Q(s,a), applied to both settings.
• Transformer-based Deep Attention Q-Network (DAQN): Encodes $p_{PS}$4 past observations (states, actions) via multi-head attention, yielding SOTA performance (e.g., sepsis WDR 0.349 ± 0.063 vs. DRQN 0.237, DQN 0.167), and demonstrating interpretability via clinically meaningful attention weights (Ma et al., 2023).
• Safe, Diverse Policy Discovery (SODA-RL): Simultaneously optimizes $p_{PS}$5 distinct, safe, diverse policies (fluid-first, vasopressor-first, mixed), subject to safety operator and regularized by symmetric KL divergence for diversity and cross-entropy to the estimated clinician policy for quality; achieves comparable mean returns to observed clinician behavior (CWPDIS 35.43±1.45 vs. 37.90), zero unsafe actions, and nontrivial policy diversity (average symKL 2.05) (Futoma et al., 2020).

Off-policy evaluation is based on weighted importance sampling (CWPDIS, WDR), with metrics such as effective sample size (ESS), unsafe actions, and reward statistics to assess reliability and clinical plausibility (Futoma et al., 2020, Ma et al., 2023).

5. Extensions: World Models, Phenotyping, and Early Prediction

World Model Environments

The Sepsis World Model (Kiani et al., 2019) employs a VAE (46-dim input → 30-dim latent) coupled with an MDN-RNN (LSTM H=256) to predict next state-embeddings, exposing the environment as an OpenAI Gym API suitable for DQN-based policy learning. Stochasticity is controlled via temperature, and model evaluation includes feature-match metrics to real EHR rollouts. The simulator supports curriculum learning, continuous actions, and alternative reward functions to enhance clinical fidelity.

Early Clinical Phenotyping

Deep temporal interpolation and clustering networks reveal that the initial hours of admission yield informative physiological trajectories, enabling unsupervised discovery of 4 robust clusters (phenotypes):

• Phenotype A: sepsis-prone, high comorbidity/mortality.
• Phenotype D: persistent hypotension, high vasopressor/early surgery rates but moderate short- and long-term mortality.

These learned embeddings and clusters can inform RL environment initialization, reward shaping, and simulation of realistic, phenotype-driven patient trajectories (Ren et al., 2023).

Early Prediction and Multi-Subset Modeling

Incremental, multi-subset approaches to predicting sepsis onset (up to 6 h ahead) leverage iteratively chained probabilistic outputs (e.g., $p_{PS}$6 as features for $p_{PS}$7), combined with temporal trend features (deltas, windowed stats). XGBoost models show substantial improvements in AUROC over direct 6-hour-ahead models (0.7906 vs. 0.6072 for early sepsis, 0.8781 vs. 0.7906 for septic shock), providing data-driven proxies for RL state augmentation or reward shaping (Ewig et al., 2023).

6. Access, Validation, and Use Cases

The Health Gym datasets are distributed via PhysioNet under the Restricted Health Data License 1.5.0, with comprehensive documentation and data dictionaries (Kuo et al., 2021). Validation includes univariate and bivariate distribution matching (KS, t-test, F-test, 3σ coverage), inspection of moment statistics, dynamic statistics via detrended time-series correlation, and privacy risk quantification (Kuo et al., 2022, Kuo et al., 2021).

Primary use cases:

• Offline RL and imitation learning algorithm development and benchmarking.
• Supervised time-series forecasting (lab/vital sign trajectories).
• Causal inference and dynamic treatment regime optimization.
• Educational simulation and demonstration without PHI risk.

7. Clinical and Methodological Implications

The Health Gym platform enables methodologically rigorous comparison of RL policies under a shared, privacy-preserving synthetic data standard, anchored in plausibly realistic state-action-reward mappings. Techniques developed here—safe multi-policy learning (SODA-RL), uncertainty-aware control (distributional RL w/ deep ensembles), interpretable representation learning (deep multi-modal attention, integration of clinical phenotypes)—establish Health Gym as a canonical reference for data-driven acute hypotension and sepsis treatment research (Futoma et al., 2020, Ma et al., 2023, Nanayakkara et al., 2021, Ren et al., 2023).

In summary, Health Gym Acute Hypotension/Sepsis constitutes a comprehensive ecosystem for robust algorithmic investigation of ICU circulatory support, offering realistic data, high statistical fidelity, formal privacy safeguards, and explicit support for RL, simulation, and causal modeling (Kuo et al., 2022, Kuo et al., 2021).