Computational trade-offs in synthetic data generation and use

Determine optimal strategies for balancing synthetic data quality and quantity against computational cost to maximize downstream statistical utility, including guidance on how many synthetic samples to generate and how to weight them relative to real data.

Background

Training high‑fidelity generative models and producing large volumes of synthetic data can be computationally expensive. Downstream use introduces additional design choices (e.g., how many synthetic samples to generate, how to weight them).

The paper identifies the need for principles that navigate the statistical–computational trade‑off, aiming to allocate computational resources where they yield the most statistical benefit.

References

Understanding how to optimally balance data quality, quantity, and computational cost largely remains an open and practically relevant problem.

— Harnessing Synthetic Data from Generative AI for Statistical Inference  (2603.05396 - Abdel-Azim et al., 5 Mar 2026) in Section 4, Computational Considerations

SDD motivates future work on: (i) numerical experiments for a wider range of synthetic data generators, (ii) whether that compute saving persists at the scale and data mixtures of production TSFM pre-training runs, where synthetic data is only part of the corpus, (iii) applications of SDD beyond TSFM to Tabular Foundation models .

— Distillation of Synthetic Data for Time Series Foundation Models  (2609.09586 - Biswas et al., 9 Sep 2026) in Section Discussion and Future Work

These weights are chosen manually rather than learned, and are not necessarily optimal. Recent work adapts the synthetic generator distribution during pretraining to emphasize harder tasks \citep{peroni2025robust}, which suggests the mixture is a design choice worth tuning. In our own experiments, however, varying these weights did not produce a large change in downstream accuracy (\Cref{app:negative}). We therefore keep the hand-chosen values above and leave a systematic optimization of the mixture to future work.

— Mitra-v2 Technical Report  (2609.04540 - Tao et al., 3 Sep 2026) in Section 2.2, “Outer prior mixture” (label: sec:priormix)

The fidelity study motivates a budget trade-off between fewer accurate labels, more approximate labels, and mixtures. It holds sample count fixed, not generation compute; a compute-optimal allocation remains open.

— Scaling Cheap Synthetic Pretraining Improves Indoor Radio-Map Prediction  (2610.19015 - Petrosyan et al., 13 Oct 2026) in Discussion and limitations