GenTS: A Comprehensive Benchmark Library for Generative Time Series Models

Abstract: Generative models have demonstrated remarkable potential in time series analysis tasks, like synthesis, forecasting, imputation, etc. However, offering limited coverage for generative models, existing time series libraries are mainly engineered for discriminative models, with standardized workflows for specific tasks, such as optimizing Mean Squared Errors for time series forecasting. This rigid structure is fundamentally incompatible with the distinct and often complex paradigms of generative models (e.g., adversarial training, diffusion processes), which learn the underlying data distribution rather than a direct input-output mapping. To this end, we proposed GenTS, a comprehensive and extensible benchmark library designed for systematic assessment on generative time series models. GenTS features a unified data preprocessing pipeline, a collection of versatile models, and panoramic evaluation metrics. Its modular design also enables the researchers to flexibly customize beyond our built-in datasets and models. Based on GenTS, we conducted benchmarking experiments under diverse tasks, accordingly offering suggestions for model selection and identifying potential directions for future research. Our codes are open-source at https://github.com/WillWang1113/GenTS. The official tutorials and document are available at https://willwang1113.github.io/GenTS/.

• The paper introduces GenTS, a comprehensive benchmark library designed to evaluate generative time series models through modular, task-oriented pipelines.
• Key findings highlight the cross-task superiority of diffusion-based models, especially in tasks requiring high distributional fidelity and model robustness.
• Implications suggest utilizing COSCIGAN for synthesis, TimeVQVAE for label-guided tasks, and diffusion models like CSDI for imputation, guiding future research.

GenTS: Benchmarking Generative Time Series Models

Motivation and Context

The increasing diversity and sophistication in generative modeling for time series has exposed fundamental limitations in the existing benchmarking ecosystem. While discriminative time series libraries are mature, their integration capabilities and standardized workflows are tightly coupled to input-output mappings and simple loss functions, introducing incompatibilities for generative paradigms such as adversarial learning, diffusion-based generation, and stochastic differential equation (SDE) modeling. GenTS addresses a critical gap by providing an extensible, modular, and comprehensive benchmark library focused on generative time series models, spanning unconditional and conditional synthesis, forecasting, and imputation (2605.17804).

Architectural Design and Modular Pipelines

GenTS is engineered with a three-stage modular pipeline:

1. Unified Data Preprocessing: Supports over 15 built-in datasets across six domains (traffic, financial, medical, weather, physics, energy) and scriptable simulation datasets (e.g., Spiral2D, SineND), handled via a customizable BaseDataModule with automated splits, missing value simulation, and irregular sampling. Researchers can extend data modules for domain-specific preprocessing requirements.
2. Flexible Model Training: Includes implementations and adaptations of 25+ state-of-the-art generative models spanning GANs (e.g., TimeGAN, COSCIGAN), VAEs (TimeVAE, KoVAE), diffusion models (DiffusionTS, ImagenTime, TMDM, FourierDiffusion), normalizing flows (MAF, FourierFlow), and neural differential equation models (LatentODE, LS4, SDEGAN). The BaseModel abstraction enables rapid prototyping and adaptation using PyTorch Lightning, with seamless integration of conditions (labels, observed windows, masks).
3. Panoramic Evaluation Platform: Provides both model-free metrics (Wasserstein distance, CRPS, MSE) and model-based metrics (Predictive Score, Discriminative Score, Context-FID) to evaluate generated data for distributional fidelity, discriminability, downstream predictive utility, and contextual embedding alignment. Visualization modules (2D t-SNE, prediction curves, imputation overlays) offer qualitative assessment.

Empirical Benchmarking and Comparative Analysis

Time Series Synthesis

Diffusion-based models demonstrated robust performance across statistical and neural metrics, consistently outperforming other classes, especially on model-based criteria (Predictive Score and Discriminative Score). VAEs produced competitive distributional matches (low Wasserstein and Context-FID). Within model categories, ImagenTime, COSCIGAN, and TimeVAE showed distinct advantages. Notably, COSCIGAN excelled in multivariate settings due to tailored discriminators for interchannel correlation. For class label-guided synthesis, TimeVQVAE achieved superior alignment, with GANs producing clear clusters but inferior representativeness. In univariate synthesis, FourierFlow emerged as an efficient baseline.

Probabilistic Forecasting

CSDI and TMDM emerged as top-performing models for both deterministic and probabilistic metrics (MSE, CRPS), with TMDM exhibiting distinct superiority on highly non-stationary datasets such as Stocks. Naive baselines (VanillaMAF, VanillaVAE) occasionally matched SOTA performance in certain domains, highlighting regimes where architectural simplicity is competitive.

Imputation

Diffusion-based models (CSDI, ImagenTime, DiffusionTS) dominated the imputation task, achieving consistently lower imputation error and uncertainty calibration (CRPS, MSE) across domains. SDE-based and naive models lagged, indicating their architectural limitations under standard missingness settings.

Computational Overheads

GANs and VAEs are computationally efficient on inference with burdens shifted to the training phase, while diffusion models exhibit slower sampling due to iterative denoising, making them preferable for offline synthetic augmentation. SDE-based models incur significant overhead from solving initial value problems (IVPs), limiting their scalability in current form.

Implications and Recommendations

The systematic benchmarking revealed clear cross-task versatility of diffusion-based models, establishing their suitability as baseline architectures for future generative time series research. Task-specific recommendations include COSCIGAN for synthesis, TimeVQVAE for label-guided tasks, CSDI/TMDM for forecasting, and diffusion models (especially CSDI) for imputation. The results underscore the need for evaluating architectural complexity vis-à-vis application domain, with simple baselines sometimes achieving satisfactory performance.

Differential equation-based models, though currently underperforming, possess inductive biases that make them promising for continuous-time generation, warranting further exploration and algorithmic refinement.

Future Research Directions

GenTS highlights several open directions:

• Foundation Model Construction: Investigate universal generative time series models capable of fitting global distributions and enabling controlled multi-task inference, moving toward foundation models for time series.
• Robust Model-Based Evaluation: Develop evaluation metrics less sensitive to auxiliary neural architectures, e.g., more robust discriminative and predictive scores, as current metrics can exhibit instability and model dependence.
• Online Modeling and Adaptation: Architect models and libraries for adaptive, low-latency online scenarios, accommodating dynamic non-stationarities and real-time distribution shifts. Diffusion models’ inference bottlenecks should be addressed for practical deployment.
• Extensibility: Continue to expand GenTS with new datasets, emerging generative paradigms, and systematic evaluation tools, including for rare but practically significant tasks like super-resolution, extreme value simulation, and irregular sampling.

Conclusion

GenTS delivers a consolidated benchmarking library for generative time series models, spanning dataset diversity, model class coverage, and task generalization. The empirical analyses provided baseline recommendations and isolated cross-task architectural strengths, particularly diffusion models. GenTS’ extensible framework facilitates rigorous evaluation and rapid development, and charts critical paths for future research in foundation modeling, robust evaluation, and online adaptation within generative time series analysis (2605.17804).