LMC-Synth Synthetic Data Generator

• LMC-Synth Synthetic Data Generator is a method for creating synthetic patient data via LLMs without requiring access to real records, ensuring inherent privacy.
• It employs in-context learning and structured prompt engineering to generate tabular data that preserves clinical correlations and adheres to detailed schema definitions.
• Empirical evaluations show that LMC-Synth maintains competitive fidelity and utility compared to traditional methods, validating its effectiveness in sensitive biomedical applications.

The LMC-Synth Synthetic Data Generator is a method for generating synthetic, tabular patient data using LLMs without direct access to original patient-level datasets. By leveraging in-context learning and structured prompt engineering, this approach circumvents traditional SDG dependence on real data, instead producing data from natural-language specifications and medical priors. Key attributes include explicit privacy by design, preservation of clinical correlations, and applicability to highly customized cohort descriptions (Tornqvist et al., 2024).

1. Methodological Framework

LMC-Synth operates by instantiating a pipeline that begins with a comprehensive natural-language database specification $S$, covering the database's nature (e.g., registry, RCT, RWD), cohort inclusion/exclusion criteria, column definitions (including names, data types — int, float, categorical — and categorical mappings), and target sample size $N$. This specification is embedded, alongside detailed instructions and 1–3 few-shot examples derived from literature averages, to construct a prompt for the LLM.

Iterative generation proceeds over $m = \lceil N / n \rceil$ batches, where each prompt induces a random permutation of columns to counteract positional bias, and instructs the LLM (e.g., GPT-3.5/4 at temperature $T = 1$) to generate $n$ new rows as a JSON array. The LLM's stochastic mapping $$f : \text{Prompt} \rightarrow T \in \mathbb{R}^{n \times d}$$ is sampled repeatedly:

$$D_{\text{synth}} = \bigcup_{i=1}^m f(\text{Prompt}_i)$$

Post-processing restores column order and ensures completeness. The design requires no access to real-world patient records, substantially lowering data privacy risk and ML expertise barrier (Tornqvist et al., 2024).

2. Prompt Engineering and In-Context Learning

Prompt construction is central to achieving high-fidelity, clinically plausible outputs. Prompts consist of:

• Prior Knowledge block: Database description and stringent cohort criteria, ensuring domain relevancy.
• Instructions block: Explicit schema (variable name, data type, medical definition, categorizations), output format rules (JSON arrays, batch size $n$, no missing values), and the directive to omit commentary.
• Few-Shot/Context block: 1–3 fictitious, literature-consistent patient records exemplifying valid variable correlations and distributions.

This multi-block architecture forces schema adherence, supports realistic joint distributions, and harnesses the LLM's in-context learning for better modeling of variable dependencies. Random column permutation at each batch breaks spurious memorization of column order.

Experimental ablation reveals that prompt composition significantly affects quality metrics: switching from GPT-3.5 to GPT-4 increases ColumnShape score by $+0.07$, and including example rows yields $+0.06$ improvement. Single-row batching ($N$0) markedly degrades both univariate and joint distribution fidelity, demonstrating the value of batch synthesis and in-context few-shot provision (Tornqvist et al., 2024).

3. Algorithmic Pipeline and Formalization

The method is formalized via the following algorithmic pseudocode:

$m = \lceil N / n \rceil$5

The LLM is invoked as a frozen model, with no further fine-tuning, and batch size $N$1 is employed due to token constraints in prompt length. The final dataset $N$2 is produced by concatenation and validation for missing data (Tornqvist et al., 2024).

4. Evaluation Protocols and Metrics

Synthetic data quality is rigorously quantified by three main criteria: fidelity (statistical similarity to target real data), privacy (protection against memorization and linkage), and utility (downstream ML performance). Benchmarks use SDMetrics and custom definitions:

• Fidelity (Statistical Similarity):
  • Univariate: KSComplement, TVComplement, ColumnShapeScore, Jensen-Shannon divergence (JSD), Earth-Mover’s distance (WD), LogisticDetection
  • Multivariate: CorrelationSimilarity ($N$3), ContingencySimilarity (pairwise TVD), ColumnPairTrendScore
• Privacy:
  • NewRowSynthesis: $N$4
  • Distance to Closest Record ($N$5): $N$6 percentile of minimal $N$7-distance to real data
  • Nearest-Neighbor Distance Ratio (NNDR): $N$8 percentile ratio of closest/second-closest pairs
  • CategoricalCAP: adversarial inference success, higher value implies greater safety
• Utility:
  • TSTR (Train-Synthetic, Test-Real): AdaBoost F1 score on real data after training on $N$9
  • TATR (Train-Augmented, Test-Real): AdaBoost F1 after augmenting $m = \lceil N / n \rceil$0 with $m = \lceil N / n \rceil$1

5. Comparative Performance Results

Extensive benchmarks were conducted on the PPMI2024 (N=1,052) and ADNI (N=362) datasets, each split into 5×5 train-test partitions, yielding 25 synthetic datasets per model. The following summarizes PPMI cohort performance for GPT-4 (LMC-Synth), Gaussian Copula, CTGAN, and TVAE:

Model / Metric	ColumnShape ↑	ColPairTrend ↑	CorrSim ↑	LogDetect ↑	TSTR F1 ↑
GPT-4 (LMC-Synth)	0.778±0.008	0.807±0.010	0.947±0.005	0.398±0.022	0.979±0.023
Gaussian Copula	0.913±0.004	0.886±0.007	0.969±0.004	0.866±0.040	0.960±0.014
CTGAN	0.839±0.021	0.851±0.009	0.917±0.007	0.405±0.118	0.838±0.117
TVAE	0.885±0.011	0.879±0.012	0.946±0.009	0.553±0.089	0.953±0.044

While Gaussian Copula and TVAE exhibit higher univariate fidelity, the LMC-Synth method delivers robust correlation preservation (CorrSim: 0.947±0.005) and delivers TSTR F1 nearly matching the strongest baselines. LMC-Synth maintains strong privacy guarantees, with empirical risk metrics favorable due to the absence of direct access to original data. SDG generation time is ~1 hr for $m = \lceil N / n \rceil$2, as compared to seconds–minutes for baselines. Results on ADNI are qualitatively similar (Tornqvist et al., 2024).

6. Advantages, Limitations, and Ablation Insights

The approach's advantages include the elimination of real data dependency, a low barrier to ML expertise, inherent privacy (no memorization of patient-level details), competitive fidelity, adaptation to new disease cohorts by mere schema modification, and maintenance of clinical correlations. Noted limitations are the risk of LLM hallucination (leading to missed outlier distributions and underestimated variance), population characteristic drifts (e.g., gender, symptom shifts), reduced accuracy for complex/rare variables, token-length batching constraints ($m = \lceil N / n \rceil$3), and a lack of formal differential privacy guarantees.

Ablation studies reveal that transitions from GPT-3.5 to GPT-4, as well as inclusion of a single synthetic example, are additive and critical to performance (ColumnShape, ColPairTrend). Minimal batching ($m = \lceil N / n \rceil$4) is detrimental to both quality measures.

7. Future Directions and Open Challenges

Identified priorities for further development include:

• Integration of formal differential privacy (DP) mechanisms such as calibrated output noise directly within the prompt or generation process.
• Expansion to multimodal synthetic generation (tabular, clinical text, images, graphs).
• Advancement of prompt engineering via interactive, chain-of-thought, or expert-guided refinement strategies.
• LLM fine-tuning or retrieval-augmented methods to further reduce any memorization risk and to optimize empirical fidelity and utility.
• Extension of evaluation towards longitudinal and multimodal datasets, addressing current limitation to cross-sectional, tabular settings (Tornqvist et al., 2024).

The development of the LMC-Synth Synthetic Data Generator marks a significant methodological shift in synthetic data generation, particularly in privacy-sensitive biomedical domains, by enabling generation from descriptive schema alone while empirically preserving crucial clinical features.