Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Interpreting Time Series Transformer Models and Sensitivity Analysis of Population Age Groups to COVID-19 Infections (2401.15119v1)

Published 26 Jan 2024 in cs.LG, cs.AI, and q-bio.PE

Abstract: Interpreting deep learning time series models is crucial in understanding the model's behavior and learning patterns from raw data for real-time decision-making. However, the complexity inherent in transformer-based time series models poses challenges in explaining the impact of individual features on predictions. In this study, we leverage recent local interpretation methods to interpret state-of-the-art time series models. To use real-world datasets, we collected three years of daily case data for 3,142 US counties. Firstly, we compare six transformer-based models and choose the best prediction model for COVID-19 infection. Using 13 input features from the last two weeks, we can predict the cases for the next two weeks. Secondly, we present an innovative way to evaluate the prediction sensitivity to 8 population age groups over highly dynamic multivariate infection data. Thirdly, we compare our proposed perturbation-based interpretation method with related work, including a total of eight local interpretation methods. Finally, we apply our framework to traffic and electricity datasets, demonstrating that our approach is generic and can be applied to other time-series domains.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (31)
  1. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Medical Informatics and Decision Making, 20.
  2. Self-Adaptive Forecasting for Improved Deep Learning on Non-Stationary Time-Series. arXiv preprint arXiv:2202.02403.
  3. Centers for Disease Control and Prevention. 2023a. COVID-19 Weekly Cases and Deaths by Age, Race/Ethnicity, and Sex.
  4. Centers for Disease Control and Prevention. 2023b. COVID-19 Weekly Cases and Deaths by Age, Race/Ethnicity, and Sex.
  5. A survey on mathematical, machine learning and deep learning models for COVID-19 transmission and diagnosis. IEEE reviews in biomedical engineering, 15: 325–340.
  6. ERASER: A benchmark to evaluate rationalized NLP models. arXiv preprint arXiv:1911.03429.
  7. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.
  8. Enguehard, J. 2023. Time Interpret: a Unified Model Interpretability Library for Time Series. arXiv preprint arXiv:2306.02968.
  9. Learning Explainable Models Using Attribution Priors. ArXiv, abs/1906.10670.
  10. Benchmarking deep learning interpretability in time series predictions. Advances in neural information processing systems, 33: 6441–6452.
  11. Toward SALib 2.0: Advancing the accessibility and interpretability of global sensitivity analyses. Socio-Environmental Systems Modelling, 4: 18155.
  12. Covid-eenet: Predicting fine-grained impact of COVID-19 on local economies. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, 11971–11981.
  13. Captum: A unified and generic model interpretability library for PyTorch. arXiv:2009.07896.
  14. Molnar, C. 2020. Interpretable machine learning. Lulu. com.
  15. Morris, M. 1991. Factorial sampling plans for preliminary computational experiments. Technometrics, 33(2): 161–174.
  16. A time series is worth 64 words: Long-term forecasting with transformers. International Conference on Learning Representations.
  17. Evaluation of interpretability methods for multivariate time series forecasting. Applied Intelligence, 1–17.
  18. Deepcovidnet: An interpretable deep learning model for predictive surveillance of covid-19 using heterogeneous features and their interactions. Ieee Access, 8: 159915–159930.
  19. Deepcovid: An operational deep learning-driven framework for explainable real-time covid-19 forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 15393–15400.
  20. Explainable artificial intelligence (xai) on timeseries data: A survey. arXiv preprint arXiv:2104.00950.
  21. Learning important features through propagating activation differences. In International conference on machine learning, 3145–3153. PMLR.
  22. Axiomatic Attribution for Deep Networks. In International Conference on Machine Learning.
  23. Clinical Intervention Prediction and Understanding using Deep Networks. ArXiv, abs/1705.08498.
  24. Evaluation of post-hoc interpretability methods in time-series classification. Nature Machine Intelligence, 5(3): 250–260.
  25. US Census Bureau. 2020. County Population by Characteristics: 2010-2020.
  26. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. In International Conference on Learning Representations.
  27. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting. In Advances in Neural Information Processing Systems.
  28. Visualizing and Understanding Convolutional Networks. In European Conference on Computer Vision.
  29. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The Eleventh International Conference on Learning Representations.
  30. Interpretation of Time-Series Deep Models: A Survey. arXiv preprint arXiv:2305.14582.
  31. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In Proc. 39th International Conference on Machine Learning (ICML 2022).

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets