Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Causal Explanation for Reinforcement Learning: Quantifying State and Temporal Importance (2210.13507v2)

Published 24 Oct 2022 in cs.AI and cs.LG

Abstract: Explainability plays an increasingly important role in machine learning. Furthermore, humans view the world through a causal lens and thus prefer causal explanations over associational ones. Therefore, in this paper, we develop a causal explanation mechanism that quantifies the causal importance of states on actions and such importance over time. We also demonstrate the advantages of our mechanism over state-of-the-art associational methods in terms of RL policy explanation through a series of simulation studies, including crop irrigation, Blackjack, collision avoidance, and lunar lander.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. Openai gym. arXiv preprint arXiv:1606.01540.
  2. Bryson, A. E. 1975. Applied optimal control: optimization, estimation and control. Boca Raton: CRC Press.
  3. Byrne, R. M. 2019. Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning. In IJCAI, 6276–6282.
  4. Neural network attributions: A causal perspective. In International Conference on Machine Learning, 981–990. PMLR.
  5. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. In 2016 IEEE symposium on security and privacy (SP), 598–617. IEEE.
  6. A survey of uncertainty in deep neural networks. arXiv preprint arXiv:2107.03342.
  7. Causal inference in statistics: A primer. Hoboken: John Wiley & Sons.
  8. Visualizing and understanding atari agents. In International Conference on Machine Learning, 1792–1801. PMLR.
  9. Explainability in deep reinforcement learning. Knowledge-Based Systems, 214: 106685.
  10. Hilton, D. 2007. Causal explanation: From social perception to knowledge-based causal attribution.
  11. Nonlinear causal discovery with additive noise models. Advances in neural information processing systems, 21: 689–696.
  12. Transparency and explanation in deep reinforcement learning neural networks. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, 144–150.
  13. Bidirectional conditional generative adversarial networks. In Asian Conference on Computer Vision, 216–232. Springer.
  14. Explainable reinforcement learning via reward decomposition. In IJCAI/ECAI Workshop on Explainable Artificial Intelligence.
  15. Causal discovery toolbox: Uncover causal relationships in python. arXiv preprint arXiv:1903.02278.
  16. Discovering causal signals in images. In Proceedings of the IEEE conference on computer vision and pattern recognition, 6979–6987.
  17. A unified approach to interpreting model predictions. arXiv preprint arXiv:1705.07874.
  18. Explainable reinforcement learning through a causal lens. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, 2493–2500.
  19. Miller, T. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, 267: 1–38.
  20. Towards interpretable reinforcement learning using attention augmented agents. arXiv preprint arXiv:1906.02500.
  21. Counterfactual state explanations for reinforcement learning agents via generative deep learning. Artificial Intelligence, 295: 103455.
  22. Pearl, J. 2009. Causality. Causality: Models, Reasoning, and Inference. Cambridge: Cambridge University Press. ISBN 9780521895606.
  23. Causal discovery with continuous additive noise models.
  24. Explainable reinforcement learning: A survey. In International cross-domain conference for machine learning and knowledge extraction, 77–95. Springer.
  25. Explain your move: Understanding agent actions using specific and relevant feature attribution. arXiv preprint arXiv:1912.12191.
  26. ” Why should i trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 1135–1144.
  27. Cxplain: Causal explanations for model interpretation under uncertainty. arXiv preprint arXiv:1910.12336.
  28. A linear non-Gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10).
  29. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034.
  30. Deep inside convolutional networks: Visualising image classification models and saliency maps.
  31. Causation, prediction, and search. Cambridge: MIT press.
  32. Axiomatic attribution for deep networks. In International Conference on Machine Learning, 3319–3328. PMLR.
  33. Reinforcement learning: An introduction. Cambridge: MIT press.
  34. Deep reinforcement learning with double q-learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30.
  35. Programmatically interpretable reinforcement learning. In International Conference on Machine Learning, 5045–5054. PMLR.
  36. Explainable ai and reinforcement learning—a systematic review of current approaches and trends. Frontiers in artificial intelligence, 4: 550030.
  37. The EPIC crop growth model. Transactions of the ASAE, 32(2): 497–0511.
  38. Causalvae: Disentangled representation learning via neural structural causal models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9593–9602.
  39. gcastle: A python toolbox for causal discovery. arXiv preprint arXiv:2111.15155.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now