FFAD: A Novel Metric for Assessing Generated Time Series Data Utilizing Fourier Transform and Auto-encoder (2403.06576v1)
Abstract: The success of deep learning-based generative models in producing realistic images, videos, and audios has led to a crucial consideration: how to effectively assess the quality of synthetic samples. While the Fr\'{e}chet Inception Distance (FID) serves as the standard metric for evaluating generative models in image synthesis, a comparable metric for time series data is notably absent. This gap in assessment capabilities stems from the absence of a widely accepted feature vector extractor pre-trained on benchmark time series datasets. In addressing these challenges related to assessing the quality of time series, particularly in the context of Fr\'echet Distance, this work proposes a novel solution leveraging the Fourier transform and Auto-encoder, termed the Fr\'{e}chet Fourier-transform Auto-encoder Distance (FFAD). Through our experimental results, we showcase the potential of FFAD for effectively distinguishing samples from different classes. This novel metric emerges as a fundamental tool for the evaluation of generative time series data, contributing to the ongoing efforts of enhancing assessment methodologies in the realm of deep learning-based generative models.
- Goodfellow et al., “Generative adversarial nets,” in Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, ser. NIPS’14. Cambridge, MA, USA: MIT Press, 2014, p. 2672–2680. [Online]. Available: https://dl.acm.org/doi/10.5555/2969033.2969125
- M. Mirza and S. Osindero, “Conditional generative adversarial nets,” 2014. [Online]. Available: http://arxiv.org/abs/1411.1784
- D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,” in 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014.
- A. Radford et al., “Unsupervised representation learning with deep convolutional generative adversarial networks,” CoRR, vol. abs/1511.06434, 2016.
- J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2242–2251. [Online]. Available: https://doi.org/10.1109/ICCV.2017.244
- Y. Chen, D. J. Kempton, A. Ahmadzadeh, J. Wen, A. Ji, and R. A. Angryk, “Cgan-based synthetic multivariate time-series generation: a solution to data scarcity in solar flare forecasting,” Neural Comput. Appl., vol. 34, no. 16, p. 13339–13353, aug 2022. [Online]. Available: https://doi.org/10.1007/s00521-022-07361-8
- T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniques for training gans,” in Proceedings of the 30th International Conference on Neural Information Processing Systems, ser. NIPS’16. Red Hook, NY, USA: Curran Associates Inc., 2016, p. 2234–2242.
- M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems, vol. 30, 2017.
- C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2818–2826.
- Y. Chen, D. J. Kempton, and R. A. Angryk, “Examining effects of class imbalance on conditional gan training,” in Artificial Intelligence and Soft Computing. Cham: Springer Nature Switzerland, 2023, pp. 475–486.
- J. W. Cooley, P. A. W. Lewis, and P. D. Welch, “The fast fourier transform and its applications,” IEEE Transactions on Education, vol. 12, no. 1, pp. 27–34, 1969.
- Q. Kemao, “Windowed fourier transform for fringe pattern analysis,” Appl. Opt., vol. 43, no. 13, pp. 2695–2702, May 2004.
- X.-J. Mao, C. Shen, and Y.-B. Yang, “Image restoration using convolutional auto-encoders with symmetric skip connections,” arXiv preprint arXiv:1606.08921, 2016.
- K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning phrase representations using RNN encoder–decoder for statistical machine translation,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar: Association for Computational Linguistics, Oct. 2014, pp. 1724–1734.
- S. Har-Peled and B. Raichel, “The fréchet distance revisited and extended,” ACM Trans. Algorithms, vol. 10, no. 1, jan 2014. [Online]. Available: https://doi.org/10.1145/2532646
- D. Dowson and B. Landau, “The fréchet distance between multivariate normal distributions,” Journal of Multivariate Analysis, vol. 12, no. 3, pp. 450–455, 1982.
- K. Preuer, P. Renz, T. Unterthiner, S. Hochreiter, and G. Klambauer, “Fréchet chemnet distance: a metric for generative models for molecules in drug discovery,” Journal of chemical information and modeling, vol. 58, no. 9, pp. 1736–1741, 2018.
- E. Goli, S. Vyas, S. Koric, N. Sobh, and P. H. Geubelle, “Chemnet: A deep neural network for advanced composites manufacturing,” The Journal of Physical Chemistry B, vol. 124, no. 42, pp. 9428–9437, 2020.
- J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling,” arXiv preprint arXiv:1412.3555, 2014.
- R. A. Angryk et al., “Multivariate time series dataset for space weather data analytics,” Scientific Data, vol. 7, no. 1, Jul. 2020. [Online]. Available: https://doi.org/10.1038/s41597-020-0548-x
- J. T. Hoeksema et al., “The helioseismic and magnetic imager (HMI) vector magnetic field pipeline: Overview and performance,” Sol. Phys., vol. 289, no. 9, pp. 3483–3530, Mar. 2014. [Online]. Available: https://doi.org/10.1007/s11207-014-0516-8
- M. G. Bobra et al., “The helioseismic and magnetic imager (hmi) vector magnetic field pipeline: Sharps–space-weather hmi active region patches,” Solar Physics, vol. 289, no. 9, pp. 3549–3578, 2014. [Online]. Available: https://doi.org/10.1007/s11207-014-0529-3
- R. Angryk et al., “SWAN-SF,” 2020. [Online]. Available: https://doi.org/10.7910/DVN/EBCFKM