Conditional Recurrent GAN (cR-GAN)

• cR-GAN is a generative framework that uses recurrent structures and explicit conditioning to generate sequential data with control over target attributes.
• It employs GRUs and attention-driven conditioning via a smooth matrix to improve the fidelity and consistency of synthetic EHR sequences, especially for rare diseases.
• The approach demonstrates enhanced performance in synthetic data evaluation metrics and downstream prediction tasks, reducing rare-needle metrics significantly.

A conditional recurrent GAN (cR-GAN) is a generative adversarial framework that incorporates recurrent structures and explicit conditional mechanisms, enabling the generation of sequential data conditioned on specified attributes. In the clinical time-series domain, cR-GANs are realized by Multi-label Time-series GANs (MTGAN), which are designed for generating realistic and controllable electronic health record (EHR) sequences, particularly addressing the challenge of rare disease modeling. These architectures employ gated recurrent units (GRUs) for sequential modeling, smooth conditioning matrices to enforce explicit label control, and critic (discriminator) networks that score entire sequences along with temporal features using Wasserstein distance and gradient penalty for stable training (Lu et al., 2022).

1. Generator Architecture and Recurrence

The generator $G$ accepts as input a noise vector $z\in\mathbb{R}^s$ (where $s$ is the GRU hidden-state dimension) and a target disease index $i\in\{1,\dots,d\}$. The output is a patient-level time-series matrix of disease probabilities: $P = [P_1,\;P_2,\;\dots,\;P_T]\in[0,1]^{d\times T}$ and the corresponding GRU hidden states

$$\widetilde H = [\tilde h_1,\;\tilde h_2,\;\dots,\;\tilde h_T]\in\mathbb{R}^{s\times T}.$$

Generation proceeds by:

• Initializing the hidden state: $\tilde h_0 = 0$.
• First-visit probability: $P_1 = \sigma(W_z\,z + b_z)\in(0,1)^d$.
• For each $t=1,\ldots,T-1$:

$$\tilde h_t = g_{\mathrm{gru}}(P_t,\,\tilde h_{t-1})$$

$z\in\mathbb{R}^s$0

The GRU cell is specified by: $z\in\mathbb{R}^s$1 where $z\in\mathbb{R}^s$2 denotes the element-wise product and all $z\in\mathbb{R}^s$3, $z\in\mathbb{R}^s$4, $z\in\mathbb{R}^s$5 are appropriately dimensioned parameters.

2. Conditional and Attention Mechanisms

Disease-conditioning employs a "smooth conditional matrix" $z\in\mathbb{R}^s$6. Steps are:

• For each time $z\in\mathbb{R}^s$7:

$z\in\mathbb{R}^s$8

$z\in\mathbb{R}^s$9

where $s$0 is a learnable vector.

• Set $s$1 for the target disease $s$2.
• Final output:

$s$3

This smooths the presence of rare diseases across the synthetic trajectory, raising the likelihood of their consistent appearance throughout the sequence and drastically reducing the "rare-needle" (RN) metric from $s$4 down to $s$5 for rare codes (Lu et al., 2022).

3. Critic Network and Wasserstein Loss

The critic $s$6 evaluates the realism of a full time-series sequence in conjunction with its temporal features:

• For real data $s$7, compute temporal features $s$8 via a frozen GRU, $s$9.
• Concatenate at each time: $i\in\{1,\dots,d\}$0.
• Pass $i\in\{1,\dots,d\}$1 through a small MLP and average over $i\in\{1,\dots,d\}$2:

$i\in\{1,\dots,d\}$3

• Use Wasserstein-GP loss:

$i\in\{1,\dots,d\}$4

where $i\in\{1,\dots,d\}$5 are linear interpolations between real and generated samples, and $i\in\{1,\dots,d\}$6 is the penalty coefficient.

The generator loss is

$i\in\{1,\dots,d\}$7

4. Training Procedure and Stabilization

Training alternates updating the critic and the generator. The procedure includes:

• Pre-training the GRU $i\in\{1,\dots,d\}$8 on next-visit binary prediction (binary-cross-entropy), then freezing it to extract real temporal features for the critic.
• Discrete sampling: instead of feeding $i\in\{1,\dots,d\}$9-valued probabilities directly to $P = [P_1,\;P_2,\;\dots,\;P_T]\in[0,1]^{d\times T}$0, each $P = [P_1,\;P_2,\;\dots,\;P_T]\in[0,1]^{d\times T}$1 is Bernoulli-sampled to yield $P = [P_1,\;P_2,\;\dots,\;P_T]\in[0,1]^{d\times T}$2.
• Pseudocode:

$\tilde h_0 = 0$5

Standard settings: GRU hidden size $P = [P_1,\;P_2,\;\dots,\;P_T]\in[0,1]^{d\times T}$3, $P = [P_1,\;P_2,\;\dots,\;P_T]\in[0,1]^{d\times T}$4, batch size $P = [P_1,\;P_2,\;\dots,\;P_T]\in[0,1]^{d\times T}$5, $P = [P_1,\;P_2,\;\dots,\;P_T]\in[0,1]^{d\times T}$6, $P = [P_1,\;P_2,\;\dots,\;P_T]\in[0,1]^{d\times T}$7, learning rates $P = [P_1,\;P_2,\;\dots,\;P_T]\in[0,1]^{d\times T}$8, $P = [P_1,\;P_2,\;\dots,\;P_T]\in[0,1]^{d\times T}$9 with 0.1 decay every $$\widetilde H = [\tilde h_1,\;\tilde h_2,\;\dots,\;\tilde h_T]\in\mathbb{R}^{s\times T}.$$0 steps, Adam with $$\widetilde H = [\tilde h_1,\;\tilde h_2,\;\dots,\;\tilde h_T]\in\mathbb{R}^{s\times T}.$$1, $$\widetilde H = [\tilde h_1,\;\tilde h_2,\;\dots,\;\tilde h_T]\in\mathbb{R}^{s\times T}.$$2, pre-training epochs $$\widetilde H = [\tilde h_1,\;\tilde h_2,\;\dots,\;\tilde h_T]\in\mathbb{R}^{s\times T}.$$3, $$\widetilde H = [\tilde h_1,\;\tilde h_2,\;\dots,\;\tilde h_T]\in\mathbb{R}^{s\times T}.$$4 (Lu et al., 2022).

5. Evaluation Metrics and Downstream Tasks

Assessment comprises statistical and predictive metrics:

• Statistical:
  • GT: Number of distinct disease codes appearing in the synthetic set.
  • JSD$$\widetilde H = [\tilde h_1,\;\tilde h_2,\;\dots,\;\tilde h_T]\in\mathbb{R}^{s\times T}.$$5, JSD$$\widetilde H = [\tilde h_1,\;\tilde h_2,\;\dots,\;\tilde h_T]\in\mathbb{R}^{s\times T}.$$6: Jensen–Shannon divergence, visit and patient-level disease frequency distributions.
  • ND$$\widetilde H = [\tilde h_1,\;\tilde h_2,\;\dots,\;\tilde h_T]\in\mathbb{R}^{s\times T}.$$7, ND$$\widetilde H = [\tilde h_1,\;\tilde h_2,\;\dots,\;\tilde h_T]\in\mathbb{R}^{s\times T}.$$8: Normalized distance (for rare diseases):

  $$\widetilde H = [\tilde h_1,\;\tilde h_2,\;\dots,\;\tilde h_T]\in\mathbb{R}^{s\times T}.$$9 - RN: Number of synthetic samples needed before all $\tilde h_0 = 0$0 real disease codes appear at least once.

• Downstream prediction: Models trained on synthetic + real data are evaluated on:

  • Multi-label diagnosis prediction (weighted $\tilde h_0 = 0$1 for $\tilde h_0 = 0$2 visit).
  • Heart-failure onset at $\tilde h_0 = 0$3 (AUC).
  • Parkinson’s disease onset at $\tilde h_0 = 0$4 (AUC).

RNN-based predictors evaluated include Dipole and GRAM, with and without synthetic pre-training. MTGAN yields lowest JSD/ND, much improved GT/RN, and the greatest improvement on rare-disease prediction (e.g., Parkinson’s onset) in MIMIC-III/IV datasets (Lu et al., 2022).

6. Limitations and Future Directions

MTGAN (cR-GAN) is limited to the generation of discrete diagnosis code sequences. Continuous variables such as labs, vitals, or medication dosages, as well as missing-data patterns, are not modeled in the current framework. Prospective research aims to extend cR-GANs to handle mixed-type EHR (discrete + continuous), incorporate privacy preservation (e.g., differential privacy), and multi-modal data (e.g., notes combined with codes) (Lu et al., 2022).

7. Context within Sequential Conditional GANs

The conditional recurrent GAN paradigm demonstrates utility beyond clinical EHR, such as in perceptual video compression, where recurrent conditional discriminators enforce both spatial and temporal consistency in generated sequences, as exemplified in PLVC (Yang et al., 2021). Both domains leverage conditioning (on target labels, or latent/motion/temporal state) and recurrence (via GRUs or ConvLSTMs) to enable high-fidelity, temporally coherent data generation. This suggests a broader applicability of cR-GANs to diverse sequential generative modeling tasks where explicit control and memory are operational requisites.