Synthetic Control Arms via Generative AI

• Synthetic control arms using generative AI are statistical methods that synthesize counterfactual data to estimate treatment effects when randomization is infeasible.
• They leverage probabilistic models such as Gaussian processes and variational autoencoders to model covariate structures and time-to-event outcomes accurately.
• Applications span policy evaluation and clinical trial simulations, where rigorous calibration and post-generation selection ensure valid inference and privacy protection.

Synthetic control arms using generative AI refer to the construction of counterfactual or synthetic data representing the control group in observational or experimental studies where randomization is infeasible, impractical, or ethically undesirable. These approaches leverage probabilistic generative models—such as Gaussian processes or variational autoencoders—to synthesize control outcomes based on observed untreated cohorts, covariate structures, and, in clinical contexts, time-to-event data. They facilitate rigorous post-treatment counterfactual inference, hypothesis testing, and, under regulatory or privacy constraints, enable data sharing and statistical augmentation.

1. Foundational Frameworks for Generative Synthetic Controls

Generative synthetic control methods estimate the effect of an intervention by constructing a counterfactual trajectory for a treated unit from a population of untreated controls. Let $y_i(t)$ denote measured outcomes for control units $i=1,\ldots,N$ and $y_0(t)$ for the treated unit, indexed over times $t=1,\ldots,T$. The data are partitioned at a known intervention point $T_0$, yielding pre- ($t \leq T_0$) and post-treatment ($t > T_0$) segments. The core statistical goal is to predict the unobserved post-treatment counterfactual $\{y_0(t)\}_{t > T_0}$ under a "no treatment" regime (Modi et al., 2019).

In econometrics, a data-driven generative approach transforms all outcomes via a bijective $f$ (such as Box–Cox, Yeo–Johnson, arcsinh) estimated from controls, enforcing approximate Gaussianity and homoscedasticity: $x_i(t) = f(y_i(t)).$ After standardization, each time series is modeled in Fourier space to exploit stationarity—yielding decorrelated frequency-mode coefficients $\tilde{x}_i(\nu)$, from which the nonparametric power spectrum $S(\nu)$ is estimated. This defines the prior variance structure for a Gaussian process (GP) prior,

$x_0 \sim \mathcal{GP}(0, K),$

with covariance kernel

$$K(t, t') = \frac{1}{T} \sum_{\nu=0}^{T-1} S(\nu) e^{i 2\pi \nu (t - t') / T}.$$

Pre-treatment records for the treated unit set the data likelihood, and standard GP regression provides a closed-form posterior for the post-treatment unobserved outcomes as well as associated error covariances.

2. Extensions to Covariate Adjustment and Time-to-Event Data

Generative approaches accommodate additional covariates $\{z_i^a(t)\}_{a=1}^A$ by forming joint time series and estimating the full cross-spectrum matrix $\mathcal{P}(\nu)_{ab}$ in Fourier space. The multivariate temporal covariance then supports accurate GP multi-output regression for both primary and auxiliary series (Modi et al., 2019).

In clinical trials, especially with survival endpoints under censoring, direct adaptation of standard synthetic control is insufficient. Here, generative models based on variational autoencoders (VAEs) provide a joint latent-variable framework: $\mathcal{D} = \{(x_i, t_i, \delta_i)\}_{i=1}^N$ where $x_i \in \mathbb{R}^d$ are mixed-type features (covariates), $t_i$ is the observed time (minimum of event and censoring times), and $\delta_i \in \{0,1\}$ is the censoring indicator (Chassat et al., 20 Nov 2025). Generative decoders output both covariates and time-to-event/censoring outcomes via neural network parameterizations, explicitly modeling the event-time and censoring-time densities and their survival functions: $$p(t_i,\delta_i \mid y_i, s_i) = [p(t_i \mid \eta_T)\,\overline{P}(t_i \mid \eta_C)]^{\delta_i} [\overline{P}(t_i \mid \eta_T)\,p(t_i \mid \eta_C)]^{1 - \delta_i}$$ with $\eta_T, \eta_C$ neural parameterizations.

3. Training Protocols and Model Architectures

Training of generative synthetic control models follows a maximum-likelihood or variational Bayes objective. In the GP framework, all transforms and prior structures are based on pre-treatment controls, ensuring out-of-sample prediction for the treated unit (Modi et al., 2019). In VAE-based models, optimization targets the evidence lower bound (ELBO): $$\log p(\mathcal{D}) \geq \sum_{i=1}^N \left\{ \mathbb{E}_{q(s_i, z_i)} [\log p(x_i, t_i, \delta_i \mid z_i, s_i)] - \mathbb{E}_{q(s_i)} \mathrm{KL}(q(z_i \mid \cdot) \| p(z_i \mid s_i)) - \mathrm{KL}(q(s_i \mid \cdot) \| p(s_i)) \right\}$$ with categorical and Gaussian variational posteriors for $s_i$ and $z_i$ respectively, leveraging Gumbel–Softmax and reparameterization tricks for differentiable sampling (Chassat et al., 20 Nov 2025).

Hyperparameters—including learning rates, architecture depths, latent dimensions, mixture components, and survival block discretizations—are selected using automatic search (Optuna), targeting minimization of key downstream metrics (e.g., survival-curve distance).

4. Statistical Inference, Hypothesis Testing, and Calibration

Synthetic control arms serve as the foundation for post-treatment statistical inference. For the GP method, the predicted post-treatment counterfactual is compared to observed outcomes using a hypothesis test structured as a Bayes factor. The null hypothesis ($H_0$) posits no treatment effect, so the observed post-treatment outcomes follow the GP-predicted counterfactual. The alternative ($H_1$) adds a flexible deviation $m(t; \theta)$ (polynomial in time since intervention): $$m(t;\theta) = \hat{y}_0(t) + \sum_{k=1}^n \theta_k (t - T_0)^k.$$ Marginal likelihoods under $H_0$ and $H_1$ are compared to compute the Bayes factor: $$\mathrm{BF} = \frac{p(\mathbf{y}_0^{\rm post} | H_1)}{p(\mathbf{y}_0^{\rm post} | H_0)}$$ with BF thresholds interpreted according to standard conventions (e.g., BF $\geq 3$ for substantial evidence) (Modi et al., 2019).

For VAE-based synthetic arms with survival endpoints, classical ML-fidelity (e.g., Jensen–Shannon distances) does not guarantee statistical validity. The Type I error (under true null) and power (under nonzero effect) can be severely miscalibrated. To correct this, a post-generation selection protocol is proposed: for each Monte Carlo run, multiple synthetic datasets are generated and only the one whose survival curve best matches the original control is retained. This procedure empirically restores nominal Type I error and power levels (Chassat et al., 20 Nov 2025).

5. Metrics for Evaluation: Fidelity, Utility, Privacy

Evaluation of synthetic control arms employs a suite of domain-specific metrics:

Metric	Purpose	Typical Value/Notes
Jensen–Shannon Distance	Feature distribution	$\sim 0.006{-}0.01$ for HI-VAE; lower is better
Survival-Curve Distance	Time-to-event similarity	Integrated absolute difference between $S_{\text{orig}}(t)$ and $S_{\text{synth}}(t)$
$K$-map Score	Privacy (minimal group)	4–7 for HI-VAE; EMA recommends ≥11 for open release
NNDR (Nearest-Neighbor)	Privacy similarity	0.2–0.4, lower indicates less risk
Type I Error	Statistical calibration	Post-selection ≈5% (target), naive >10–20%
Power	Statistical sensitivity	Matches theoretical after selection under ind. censoring

Fidelity and privacy scores must be interpreted in context: models with excellent ML-metrics can still yield defective inference. Regulatory-grade privacy is not always attained by high-fidelity models; applying strong differential privacy often impairs utility (Chassat et al., 20 Nov 2025).

6. Empirical Evidence and Use Cases

Generative synthetic control techniques have demonstrated efficacy in multiple settings. In (Modi et al., 2019), the formalism was applied to California’s 1988 cigarette sales tax. After transforming and modeling pre-treatment data, the generative approach predicted post-treatment outcomes and provided a Bayes factor of approximately 5.8:1 in favor of a policy effect—a result substantially stronger than any placebo analysis among 38 control states.

In clinical trial simulation and Phase III datasets (e.g., ACTG 320, NCT00119613, NCT00113763, NCT00339183), VAE-based models generated synthetic control arms whose survival curves were statistically indistinguishable (BH-adjusted log-rank $p > 0.05$) from real data in 80–100% of replications, compared to 30–70% for GAN-baselines. Post-generation selection was shown to restore nominal inference properties (Type I error and power) even under severe data sharing constraints or when only a minority of real control data was used for training (Chassat et al., 20 Nov 2025).

7. Limitations, Open Challenges, and Best Practices

Several persistent issues remain. First, standard machine learning fidelity and privacy metrics are not sufficient for ensuring statistical validity in downstream hypothesis tests. Explicit simulation-based calibration (for both Type I error and power) is necessary. Second, joint modeling of censoring and event times in survival data breaks the standard i.i.d. censoring assumption, and dependent censoring can degrade the benefit of synthetic augmentation. Third, privacy measures—such as $K$-map and NNDR—fall short of regulatory standards for open data release; approaches guaranteeing strong differential privacy sacrifice inference performance (Chassat et al., 20 Nov 2025).

Best practices include: always validating inference via simulation, using post-generation selection to calibrate error rates, and maintaining awareness of regulatory guidelines (FDA, EMA). Future directions proposed in the literature include the development of calibration-aware training objectives, application of diffusion/transformer generative models with survival blocks, and alignment with formal differential privacy standards $(\varepsilon, \delta)$ (Chassat et al., 20 Nov 2025). Bridging the gap between generative modeling accuracy and clinical-trial/statistical validity is an ongoing research frontier.