From Chaos to Clarity: Time Series Anomaly Detection in Astronomical Observations (2403.10220v1)
Abstract: With the development of astronomical facilities, large-scale time series data observed by these facilities is being collected. Analyzing anomalies in these astronomical observations is crucial for uncovering potential celestial events and physical phenomena, thus advancing the scientific research process. However, existing time series anomaly detection methods fall short in tackling the unique characteristics of astronomical observations where each star is inherently independent but interfered by random concurrent noise, resulting in a high rate of false alarms. To overcome the challenges, we propose AERO, a novel two-stage framework tailored for unsupervised anomaly detection in astronomical observations. In the first stage, we employ a Transformer-based encoder-decoder architecture to learn the normal temporal patterns on each variate (i.e., star) in alignment with the characteristic of variate independence. In the second stage, we enhance the graph neural network with a window-wise graph structure learning to tackle the occurrence of concurrent noise characterized by spatial and temporal randomness. In this way, AERO is not only capable of distinguishing normal temporal patterns from potential anomalies but also effectively differentiating concurrent noise, thus decreasing the number of false alarms. We conducted extensive experiments on three synthetic datasets and three real-world datasets. The results demonstrate that AERO outperforms the compared baselines. Notably, compared to the state-of-the-art model, AERO improves the F1-score by up to 8.76% and 2.63% on synthetic and real-world datasets respectively.
- C. Yang, Z. Du, X. Meng, X. Zhang, X. Hao, and D. A. Bader, “Anomaly detection in catalog streams,” IEEE Transactions on Big Data, vol. 9, no. 1, pp. 294–311, 2023.
- S. Han and S. S. Woo, “Learning sparse latent graph representations for anomaly detection in multivariate time series,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, ser. KDD ’22, 2022, p. 2977–2986.
- Z. Li, Y. Zhao, J. Han, Y. Su, R. Jiao, X. Wen, and D. Pei, “Multivariate time series anomaly detection and interpretation using hierarchical inter-metric and temporal embedding,” in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, ser. KDD ’21, 2021, p. 3220–3230.
- P. Boniol, J. Paparizzos, and T. Palpanas, “New trends in time series anomaly detection,” in International Conference on Extending Database Technology, 2023.
- Z. Chen, D. Chen, X. Zhang, Z. Yuan, and X. Cheng, “Learning graph structures with transformer for multivariate time-series anomaly detection in iot,” IEEE Internet of Things Journal, vol. 9, no. 12, pp. 9179–9189, 2022.
- C. Zhang, D. Song, Y. Chen, X. Feng, C. Lumezanu, W. Cheng, J. Ni, B. Zong, H. Chen, and N. V. Chawla, “A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data,” in Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, ser. AAAI’19, 2019.
- A. Deng and B. Hooi, “Graph neural network-based anomaly detection in multivariate time series,” in Proceedings of the AAAI conference on artificial intelligence, vol. 35, no. 5, 2021, pp. 4027–4035.
- Y. Cui, K. Zheng, D. Cui, J. Xie, L. Deng, F. Huang, and X. Zhou, “Metro: A generic graph neural network framework for multivariate time series forecasting,” Proc. VLDB Endow., vol. 15, no. 2, p. 224–236, oct 2021.
- Y. Wu, M. Gu, L. Wang, Y. Lin, F. Wang, and H. Yang, “Event2graph: Event-driven bipartite graph for multivariate time-series anomaly detection,” ArXiv preprint, vol. abs/2108.06783, 2021. [Online]. Available: https://arxiv.org/abs/2108.06783
- A. Siffer, P.-A. Fouque, A. Termier, and C. Largouet, “Anomaly detection in streams with extreme value theory,” in Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ser. KDD ’17, 2017, p. 1067–1075.
- J. Li, S. Di, Y. Shen, and L. Chen, “Fluxev: A fast and effective unsupervised framework for time-series anomaly detection,” in Proceedings of the 14th ACM International Conference on Web Search and Data Mining, ser. WSDM ’21, 2021, p. 824–832.
- H. Ren, B. Xu, Y. Wang, C. Yi, C. Huang, X. Kou, T. Xing, M. Yang, J. Tong, and Q. Zhang, “Time-series anomaly detection service at microsoft,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ser. KDD ’19, 2019, p. 3009–3017.
- P. Malhotra, L. Vig, G. Shroff, P. Agarwal et al., “Long short term memory networks for anomaly detection in time series.” in Esann, vol. 2015, 2015, p. 89.
- S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.
- H. Xu, W. Chen, N. Zhao, Z. Li, J. Bu, Z. Li, Y. Liu, Y. Zhao, D. Pei, Y. Feng, J. Chen, Z. Wang, and H. Qiao, “Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications,” in Proceedings of the 2018 World Wide Web Conference, ser. WWW ’18, 2018, p. 187–196.
- J. An and S. Cho, “Variational autoencoder based anomaly detection using reconstruction probability,” Special lecture on IE, vol. 2, no. 1, pp. 1–18, 2015.
- S. Lin, R. Clark, R. Birke, S. Schönborn, N. Trigoni, and S. Roberts, “Anomaly detection for time series using vae-lstm hybrid model,” in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 4322–4326.
- K. Hundman, V. Constantinou, C. Laporte, I. Colwell, and T. Soderstrom, “Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ser. KDD ’18, 2018, p. 387–395.
- Y. Su, Y. Zhao, C. Niu, R. Liu, W. Sun, and D. Pei, “Robust anomaly detection for multivariate time series through stochastic recurrent neural network,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ser. KDD ’19, 2019, p. 2828–2837.
- T. Kieu, B. Yang, C. Guo, R.-G. Cirstea, Y. Zhao, Y. Song, and C. S. Jensen, “Anomaly detection in time series with robust variational quasi-recurrent autoencoders,” in 2022 IEEE 38th International Conference on Data Engineering (ICDE), 2022, pp. 1342–1354.
- D. Li, D. Chen, B. Jin, L. Shi, J. Goh, and S.-K. Ng, “Mad-gan: Multivariate anomaly detection for time series data with generative adversarial networks,” in Artificial Neural Networks and Machine Learning – ICANN 2019: Text and Time Series: 28th International Conference on Artificial Neural Networks, Munich, Germany, September 17–19, 2019, Proceedings, Part IV, 2019, p. 703–716.
- X. Chen, L. Deng, F. Huang, C. Zhang, Z. Zhang, Y. Zhao, and K. Zheng, “Daemon: Unsupervised anomaly detection and interpretation for multivariate time series,” in 2021 IEEE 37th International Conference on Data Engineering (ICDE), 2021, pp. 2225–2230.
- H. Wu, T. Hu, Y. Liu, H. Zhou, J. Wang, and M. Long, “Timesnet: Temporal 2d-variation modeling for general time series analysis,” arXiv preprint arXiv:2210.02186, 2022.
- F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” IEEE Transactions on Neural Networks, vol. 20, no. 1, pp. 61–80, 2009.
- Y. Wu, H.-N. Dai, and H. Tang, “Graph neural networks for anomaly detection in industrial internet of things,” IEEE Internet of Things Journal, vol. 9, no. 12, pp. 9214–9231, 2022.
- H. Zhao, Y. Wang, J. Duan, C. Huang, D. Cao, Y. Tong, B. Xu, J. Bai, J. Tong, and Q. Zhang, “Multivariate time-series anomaly detection via graph attention network,” in 2020 IEEE International Conference on Data Mining (ICDM), 2020, pp. 841–850.
- J. Zhan, S. Wang, X. Ma, C. Wu, C. Yang, D. Zeng, and S. Wang, “Stgat-mad : Spatial-temporal graph attention network for multivariate time series anomaly detection,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 3568–3572.
- Z. Wu, S. Pan, G. Long, J. Jiang, X. Chang, and C. Zhang, “Connecting the dots: Multivariate time series forecasting with graph neural networks,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ser. KDD ’20, 2020, p. 753–763.
- H. Yu, T. Li, W. Yu, J. Li, Y. Huang, L. Wang, and A. Liu, “Regularized graph structure learning with semantic knowledge for multi-variates time-series forecasting,” arXiv preprint arXiv:2210.06126, 2022.
- X. Chen, Q. Qiu, C. Li, and K. Xie, “Graphad: A graph neural network for entity-wise multivariate time-series anomaly detection,” in Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, ser. SIGIR ’22, 2022, p. 2297–2302.
- J. Chen, Y. Yang, T. Yu, Y. Fan, X. Mo, and C. Yang, “Brainnet: Epileptic wave detection from seeg with hierarchical graph diffusion learning,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, ser. KDD ’22, 2022, p. 2741–2751.
- J. Ye, Z. Liu, B. Du, L. Sun, W. Li, Y. Fu, and H. Xiong, “Learning the evolutionary and multi-scale graph structure for multivariate time series forecasting,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, ser. KDD ’22, 2022, p. 2296–2306.
- W. T. Ng, K. Siu, A. C. Cheung, and M. K. Ng, “Expressing multivariate time series as graphs with time series attention transformer,” arXiv preprint arXiv:2208.09300, 2022.
- Y. Fang, K. Ren, C. Shan, Y. Shen, Y. Li, W. Zhang, Y. Yu, and D. Li, “Learning decomposed spatial relations for multi-variate time-series modeling,” in Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence, ser. AAAI’23, 2023.
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, ser. NIPS’17, 2017, p. 6000–6010.
- Q. Wen, T. Zhou, C. Zhang, W. Chen, Z. Ma, J. Yan, and L. Sun, “Transformers in time series: A survey,” arXiv preprint arXiv:2202.07125, 2022.
- S. Tuli, G. Casale, and N. R. Jennings, “Tranad: Deep transformer networks for anomaly detection in multivariate time series data,” Proc. VLDB Endow., vol. 15, no. 6, p. 1201–1214, feb 2022.
- J. Xu, H. Wu, J. Wang, and M. Long, “Anomaly transformer: Time series anomaly detection with association discrepancy,” arXiv preprint arXiv:2110.02642, 2021.
- J. D. M.-W. C. Kenton and L. K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of naacL-HLT, vol. 1, 2019, p. 2.
- L. Dong, S. Xu, and B. Xu, “Speech-transformer: A no-recurrence sequence-to-sequence model for speech recognition,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp. 5884–5888.
- X. Hu, L.-X. Zhang, L. Gao, W. Dai, X. Han, Y.-K. Lai, and Y. Chen, “Glim-net: Chronic glaucoma forecast transformer for irregularly sampled sequential fundus images,” IEEE Transactions on Medical Imaging, vol. 42, no. 6, pp. 1875–1884, 2023.
- S. N. Shukla and B. M. Marlin, “Multi-time attention networks for irregularly sampled time series,” arXiv preprint arXiv:2101.10318, 2021.
- J. R. Davenport, S. L. Hawley, L. Hebb, J. P. Wisniewski, A. F. Kowalski, E. C. Johnson, M. Malatesta, J. Peraza, M. Keil, S. M. Silverberg et al., “Kepler flares. ii. the temporal morphology of white-light flares on gj 1243,” The Astrophysical Journal, vol. 797, no. 2, p. 122, 2014.
- G.-W. Li, C. Wu, G.-P. Zhou, C. Yang, H.-L. Li, J. Chen, L.-P. Xin, J. Wang, H. Haerken, C.-H. Ma et al., “Magnetic activity and parameters of 43 flare stars in the gwac archive,” Research in Astronomy and Astrophysics, vol. 23, no. 1, p. 015016, 2023.
- Z. Duan, C. Yang, X. Meng, Y. Du, J. Qiu, X. Ma, Z. Du, X. Zhang, B. Niu, and C. Wu, “Scidetector: Scientific event discovery by tracking variable source data streaming,” in 2019 IEEE 35th International Conference on Data Engineering (ICDE), 2019, pp. 2040–2043.
- X. Hou and L. Zhang, “Saliency detection: A spectral residual approach,” in 2007 IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1–8.