Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

TimeX++: Learning Time-Series Explanations with Information Bottleneck (2405.09308v1)

Published 15 May 2024 in cs.LG and cs.AI

Abstract: Explaining deep learning models operating on time series data is crucial in various applications of interest which require interpretable and transparent insights from time series signals. In this work, we investigate this problem from an information theoretic perspective and show that most existing measures of explainability may suffer from trivial solutions and distributional shift issues. To address these issues, we introduce a simple yet practical objective function for time series explainable learning. The design of the objective function builds upon the principle of information bottleneck (IB), and modifies the IB objective function to avoid trivial solutions and distributional shift issues. We further present TimeX++, a novel explanation framework that leverages a parametric network to produce explanation-embedded instances that are both in-distributed and label-preserving. We evaluate TimeX++ on both synthetic and real-world datasets comparing its performance against leading baselines, and validate its practical efficacy through case studies in a real-world environmental application. Quantitative and qualitative evaluations show that TimeX++ outperforms baselines across all datasets, demonstrating a substantial improvement in explanation quality for time series data. The source code is available at \url{https://github.com/zichuan-liu/TimeXplusplus}.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. Can CO2 emissions and energy consumption determine the economic performance of south korea? A time series analysis. Environmental Science and Pollution Research, pp.  38969–38984, 2021.
  2. Neural additive models: Interpretable machine learning with neural nets. In NeurIPS, pp.  4699–4711, 2021.
  3. Deep variational information bottleneck. In ICLR, 2017.
  4. Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state. Physical Review E, pp.  061907, 2001.
  5. Timeshap: Explaining recurrent models through sequence perturbations. In SIGKDD, pp.  2565–2573, 2021.
  6. Analysis of explainers of black box deep neural networks for computer vision: A survey. Machine Learning and Knowledge Extraction, 3(4):966–989, 2021.
  7. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. In NeurIPS, pp.  3504–3512, 2016.
  8. CoRTX: Contrastive framework for real-time explanation. In ICLR, pp.  1–23, 2023.
  9. Explaining time series predictions with dynamic masks. In ICML, pp.  2166–2177, 2021.
  10. A survey of the state of explainable ai for natural language processing. In IJCNLP-AACL, pp.  447–459, 2020.
  11. The ucr time series archive. IEEE/CAA Journal of Automatica Sinica, 6(6):1293–1305, 2019.
  12. Enguehard, J. Learning perturbations to explain time series predictions. In ICML, pp.  9329–9342, 2023.
  13. When comparing to ground truth is wrong: On evaluating gnn explanation methods. In SIGKDD, pp.  332–341, 2021.
  14. Interpretable explanations of black boxes by meaningful perturbation. In CVPR, pp.  3429–3437, 2017.
  15. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, 2020.
  16. The information bottleneck problem and its applications in machine learning. IEEE Journal on Selected Areas in Information Theory, 1(1):19–38, 2020.
  17. Guidotti, R. Counterfactual explanations and how to find them: literature review and benchmarking. Data Mining and Knowledge Discovery, pp.  1–55, 2022.
  18. Factorized explainer for graph neural networks. In AAAI, 2024.
  19. Improving deep learning interpretability by saliency guided training. In NeurIPS, pp.  26726–26739, 2021.
  20. Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? In ACL, pp.  4198–4205, 2020.
  21. Categorical reparameterization with gumbel-softmax. In ICLR, pp.  1–12, 2017.
  22. AI in healthcare: time-series forecasting using statistical, neural, and ensemble architectures. Frontiers in Big Data, 3:4, 2020.
  23. Temporal dependencies in feature importance for time series prediction. In ICLR, pp.  1–18, 2023.
  24. Preserving dynamic attention for long-term spatial-temporal prediction. In SIGKDD, pp.  36–46, 2020.
  25. Explainable ai: A review of machine learning interpretability methods. Entropy, 23(1):18, 2020.
  26. NA2Q: Neural attention additive model for interpretable multi-agent Q-learning. In ICML, pp.  22539–22558, 2023.
  27. Protecting your llms with information bottleneck. arXiv preprint arXiv:2404.13968, 2024a.
  28. Explaining time series via contrastive and locally sparse perturbations. In ICLR, pp.  1–21, 2024b.
  29. A unified approach to interpreting model predictions. In NeurIPS, pp.  4765–4774, 2017.
  30. Parameterized explainer for graph neural network. In NeurIPS, pp.  19620–19631, 2020.
  31. Towards inductive and efficient explanations for graph neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
  32. Significance-aware information bottleneck for domain adaptive semantic segmentation. In ICCV, pp.  6778–6787, 2019.
  33. Variational information bottleneck for effective low-resource fine-tuning. In ICLR, pp.  1–13, 2021.
  34. Formal limitations on the measurement of mutual information. In AISTATS, pp.  875–884, 2020.
  35. Interpretable and generalizable graph learning via stochastic attention mechanism. In ICML, pp.  15524–15543, 2022.
  36. The impact of the MIT-BIH arrhythmia database. IEEE Engineering in Medicine and Biology Magazine, 20:45–50, 2001.
  37. Encoding time-series explanations through self-supervised model behavior consistency. In NeurIPS, 2023.
  38. Razavi, S. Deep learning, explained: Fundamentals, explainability, and bridgeability to process-based modelling. Environmental Modelling & Software, 144:105159, 2021.
  39. Deep learning and process understanding for data-driven earth system science. Nature, 566(7743):195–204, 2019.
  40. Introducing a new benchmarked dataset for activity monitoring. In ISWC, pp.  108–109, 2012.
  41. “Why should I trust you?” Explaining the predictions of any classifier. In SIGKDD, pp.  1135–1144, 2016.
  42. The power of explainability in forecast-informed deep learning models for flood mitigation. arXiv preprint arXiv:2310.19166, 2023a.
  43. Deep learning models for water stage predictions in south florida. arXiv preprint arXiv:2306.15907, 2023b.
  44. Learning important features through propagating activation differences. In ICML, pp.  3145–3153, 2017.
  45. Optimization methods for interpretable differentiable decision trees applied to reinforcement learning. In AISTATS, pp.  1855–1865, 2020.
  46. Axiomatic attribution for deep networks. In ICML, pp.  3319–3328, 2017.
  47. Clinical intervention prediction and understanding with deep neural networks. In MLHC, pp.  322–337, 2017.
  48. Explainable ai for time series classification: a review, taxonomy and research directions. IEEE Access, 10:100700–100724, 2022.
  49. Deep learning and the information bottleneck principle. In IEEE Information Theory Workshop, pp.  1–5, 2015.
  50. The information bottleneck method. In Allerton Conference on Communicationn, Control, and Computing, pp.  368–377, 1999.
  51. What went wrong and when? Instance-wise feature importance for time-series black-box models. In NeurIPS, pp.  799–809, 2020.
  52. Attention is all you need. In NeurIPS, pp.  5998–6008, 2017.
  53. Bottlesum: Unsupervised and self-supervised sentence summarization using the information bottleneck principle. In EMNLP-IJCNLP, pp.  3752–3761, 2019.
  54. Graph information bottleneck. In NeurIPS, pp.  20437–20448, 2020.
  55. A survey on neural network interpretability. IEEE Transactions on Emerging Topics in Computational Intelligence, 5(5):726–742, 2021.
  56. OOD-CV: A benchmark for robustness to out-of-distribution shifts of individual nuisances in natural images. In ECCV, pp.  163–180, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (10)
  1. Zichuan Liu (27 papers)
  2. Tianchun Wang (19 papers)
  3. Jimeng Shi (14 papers)
  4. Xu Zheng (88 papers)
  5. Zhuomin Chen (10 papers)
  6. Lei Song (60 papers)
  7. Wenqian Dong (7 papers)
  8. Jayantha Obeysekera (4 papers)
  9. Farhad Shirani (45 papers)
  10. Dongsheng Luo (46 papers)
Citations (3)

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com