Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

TransFusion: Generating Long, High Fidelity Time Series using Diffusion Models with Transformers (2307.12667v2)

Published 24 Jul 2023 in cs.LG

Abstract: The generation of high-quality, long-sequenced time-series data is essential due to its wide range of applications. In the past, standalone Recurrent and Convolutional Neural Network-based Generative Adversarial Networks (GAN) were used to synthesize time-series data. However, they are inadequate for generating long sequences of time-series data due to limitations in the architecture. Furthermore, GANs are well known for their training instability and mode collapse problem. To address this, we propose TransFusion, a diffusion, and transformers-based generative model to generate high-quality long-sequence time-series data. We have stretched the sequence length to 384, and generated high-quality synthetic data. Also, we introduce two evaluation metrics to evaluate the quality of the synthetic data as well as its predictive characteristics. We evaluate TransFusion with a wide variety of visual and empirical metrics, and TransFusion outperforms the previous state-of-the-art by a significant margin.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. Synthea: An approach, method, and software mechanism for generating synthetic patients and the synthetic electronic health care record. Journal of the American Medical Informatics Association, 25(3):230–238, 2018.
  2. CorGAN: Correlation-Capturing Convolutional Generative Adversarial Networks for Generating Synthetic Healthcare Records. In The Thirty-Third International Flairs Conference, 2020.
  3. Time-Series Generative Adversarial Networks. Advances in neural information processing systems, 32, 2019.
  4. LSTM-Based VAE-GAN for Time-Series Anomaly Detection. Sensors, 20(13):3738, 2020.
  5. Cot-Gan: Generating Sequential Data via Causal Optimal Transport. Advances in Neural Information Processing Systems, 33:8798–8809, 2020.
  6. Gt-gan: General purpose time series synthesis with generative adversarial networks. Advances in Neural Information Processing Systems, 35:36999–37010, 2022.
  7. Generative Adversarial Networks. Communications of the ACM, 63(11):139–144, 2020.
  8. Deep Learning for Time Series Classification: A Review. Data mining and knowledge discovery, 33(4):917–963, 2019.
  9. The Performance of LSTM and BiLSTM in Forecasting Time Series. In 2019 IEEE International Conference on Big Data (Big Data), pages 3285–3292. IEEE, 2019.
  10. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In International Conference on Machine Learning, pages 8857–8868. PMLR, 2021.
  11. Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  12. Improved Denoising Diffusion Probabilistic Models. In International Conference on Machine Learning (ICML), pages 8162–8171. PMLR, 2021.
  13. Attention is All You Need. Advances in neural information processing systems, 30, 2017.
  14. Memory-based transformer with shorter window and longer horizon for multivariate time series forecasting. Pattern Recognition Letters, 160:26–33, 2022.
  15. Tts-gan: A transformer-based time-series generative adversarial network. In International Conference on Artificial Intelligence in Medicine, pages 133–143. Springer, 2022.
  16. Adversarial audio synthesis. In International Conference on Learning Representations (ICLR), 2019.
  17. CVAE-GAN: Fine-Grained Image Generation Through Asymmetric Training. In Proceedings of the IEEE international conference on computer vision, pages 2745–2754, 2017.
  18. Adversarial Feature Matching for Text Generation. In International Conference on Machine Learning (ICML), pages 4006–4015. PMLR, 2017.
  19. Quant gans: deep generation of financial time series. Quantitative Finance, 20(9):1419–1440, 2020.
  20. Temporal convolutional networks for action segmentation and detection. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 156–165, 2017.
  21. Sig-wasserstein gans for time series generation. In Proceedings of the Second ACM International Conference on AI in Finance, pages 1–8, 2021.
  22. Video Diffusion Models. In ICLR Workshop on Deep Generative Models for Highly Structured Data, 2022.
  23. DiffWave: A Versatile Diffusion Model for Audio Synthesis. In International Conference on Learning Representations, 2020.
  24. Data Driven Prediction Models of Energy Use of Appliances in a Low-Energy House. Energy and buildings, 140:81–97, 2017.
  25. On Field Calibration of an Electronic Nose for Benzene Estimation in an Urban Pollution Monitoring Scenario. Sensors and Actuators B: Chemical, 129(2):750–757, 2008.
  26. Principal-Components Analysis and Exploratory and Confirmatory Factor Analysis. 1995.
  27. Laurens Van der Maaten and Geoffrey Hinton. Visualizing Data using T-SNE. Journal of machine learning research, 9(11), 2008.
  28. The Jensen-Shannon Divergence. Journal of the Franklin Institute, 334(2):307–318, 1997.
  29. How faithful is your synthetic data? sample-level metrics for evaluating and auditing generative models. In International Conference on Machine Learning, pages 290–306. PMLR, 2022.
  30. Pytorch: An Imperative Style, High-Performance Deep Learning Library. In Proceedings of the Advances in neural information processing systems, 32, 2019.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Md Fahim Sikder (6 papers)
  2. Resmi Ramachandranpillai (8 papers)
  3. Fredrik Heintz (18 papers)
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now