Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Diverse Explanations From Data-Driven and Domain-Driven Perspectives in the Physical Sciences (2402.00347v2)

Published 1 Feb 2024 in cs.LG

Abstract: Machine learning methods have been remarkably successful in material science, providing novel scientific insights, guiding future laboratory experiments, and accelerating materials discovery. Despite the promising performance of these models, understanding the decisions they make is also essential to ensure the scientific value of their outcomes. However, there is a recent and ongoing debate about the diversity of explanations, which potentially leads to scientific inconsistency. This Perspective explores the sources and implications of these diverse explanations in ML applications for physical sciences. Through three case studies in materials science and molecular property prediction, we examine how different models, explanation methods, levels of feature attribution, and stakeholder needs can result in varying interpretations of ML outputs. Our analysis underscores the importance of considering multiple perspectives when interpreting ML models in scientific contexts and highlights the critical need for scientists to maintain control over the interpretation process, balancing data-driven insights with domain expertise to meet specific scientific needs. By fostering a comprehensive understanding of these inconsistencies, we aim to contribute to the responsible integration of eXplainable Artificial Intelligence (XAI) into physical sciences and improve the trustworthiness of ML applications in scientific discovery.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access, 6:52138–52160, 2018. doi: 10.1109/ACCESS.2018.2870052.
  2. Gold Nanoparticle Data Set. v1. CSIRO. Data Collection., 2019. doi: 10.25919/5d395ef9a4291.
  3. Palladium Nanoparticle Data Set. v2. CSIRO. Data Collection., 2023. doi: 10.25919/epxd-8p61.
  4. Platinum Nanoparticle Data Set. v2. CSIRO. Data Collection., 2018. doi: 10.25919/5d3958d9bf5f7.
  5. Barnard, A. S. Explainable prediction of N-V-related defects in nanodiamond using neural networks and Shapley values. Cell Reports Physical Science, 3(1):100696, 2022. ISSN 2666-3864. doi: 10.1016/j.xcrp.2021.100696.
  6. Importance of Structural Features and the Influence of Individual Structures of Graphene Oxide Using Shapley Value Analysis. Chemistry of Materials, 35(21):8840–8856, 2023. doi: 10.1021/acs.chemmater.3c00715.
  7. Beck, M. W. NeuralNetTools: Visualization and analysis tools for neural networks. Journal of statistical software, 85(11):1, 2018. doi: 10.18637/jss.v085.i11.
  8. Breiman, L. Random forests. Machine learning, 45:5–32, 2001. doi: 10.1023/A:1010933404324.
  9. Artificial intelligence driven in-silico discovery of novel organic lithium-ion battery cathodes. Energy storage materials, 44:313–325, 2022. doi: 10.1016/j.ensm.2021.10.029.
  10. An interpretable model with globally consistent explanations for credit risk. arXiv preprint arXiv:1811.12615, 2018.
  11. The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation. PeerJ Computer Science, 7:e623, 2021. doi: 10.7717/peerj-cs.623.
  12. Concept activation regions: A generalized framework for concept-based explanations. Advances in Neural Information Processing Systems, 35:2590–2607, 2022.
  13. Explaining latent representations with a corpus of examples. Advances in Neural Information Processing Systems, 34:12154–12166, 2021.
  14. Exploring the cloud of variable importance for the set of all good models. Nature Machine Intelligence, 2(12):810–824, 2020.
  15. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608, 2017.
  16. Expanding explainability: Towards Social Transparency in AI Systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp.  1–19, 2021.
  17. All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously. J. Mach. Learn. Res., 20(177):1–81, 2019.
  18. Freitas, A. A. Comprehensible classification models: a position paper. ACM SIGKDD explorations newsletter, 15(1):1–10, 2014. doi: 10.1145/2594473.2594475.
  19. Towards automatic concept-based explanations. Advances in neural information processing systems, 32, 2019.
  20. Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA), pp.  80–89. IEEE, 2018.
  21. Explainable AI: current status and future directions. arXiv preprint arXiv:2107.07045, 2021.
  22. Advanced microstructure classification by data mining methods. Computational Materials Science, 148:324–335, 2018. doi: 10.1016/j.commatsci.2018.03.004.
  23. A survey of methods for explaining black box models. ACM computing surveys (CSUR), 51(5):1–42, 2018.
  24. Gunning, D. Explainable artificial intelligence (xai). Defense advanced research projects agency (DARPA), nd Web, 2(2):1, 2017.
  25. Rashomon Capacity: A Metric for Predictive Multiplicity in Classification. Advances in Neural Information Processing Systems, 35:28988–29000, 2022.
  26. Explainable discovery of disease biomarkers: The case of ovarian cancer to illustrate the best practice in machine learning and Shapley analysis. Journal of Biomedical Informatics, 141:104365, 2023.
  27. Multiple stakeholders drive diverse interpretability requirements for machine learning in healthcare. Nature Machine Intelligence, 5(8):824–829, 2023.
  28. Explaining explanations: Axiomatic feature interactions for deep networks. The Journal of Machine Learning Research, 22(1):4687–4740, 2021.
  29. Drug discovery with explainable artificial intelligence. Nature Machine Intelligence, 2(10):573–584, 2020.
  30. Reliable and explainable machine-learning methods for accelerated material discovery. npj Computational Materials, 5(1):108, 2019. doi: 10.1038/s41524-019-0248-2.
  31. Theory-guided data science: A new paradigm for scientific discovery from data. IEEE Transactions on knowledge and data engineering, 29(10):2318–2331, 2017.
  32. Examples are not enough, learn to criticize! criticism for interpretability. Advances in neural information processing systems, 29, 2016.
  33. Exploration of the Rashomon Set Assists Trustworthy Explanations for Medical Data. arXiv e-prints, pp.  arXiv–2308, 2023.
  34. Understanding machine-learned density functionals. International Journal of Quantum Chemistry, 116(11):819–833, 2016.
  35. Variance Tolerance Factors For Interpreting All Neural Networks. In 2023 International Joint Conference on Neural Networks (IJCNN), pp.  1–9, 2023a. doi: 10.1109/IJCNN54540.2023.10191646.
  36. Multi-target neural network predictions of MXenes as high-capacity energy storage materials in a Rashomon set. Cell Reports Physical Science, 4(11), 2023b.
  37. The impact of domain-driven and data-driven feature selection on the inverse design of nanoparticle catalysts. Journal of Computational Science, 65:101896, 2022.
  38. Exploring the cloud of feature interaction scores in a Rashomon set. arXiv preprint arXiv:2305.10181, 2023.
  39. Explainable ai: A review of machine learning interpretability methods. Entropy, 23(1):18, 2020.
  40. Lipton, Z. C. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3):31–57, 2018.
  41. London, A. J. Artificial intelligence and black-box medical decisions: accuracy versus explainability. Hastings Center Report, 49(1):15–21, 2019.
  42. Reliable graph neural network explanations through adversarial training. arXiv preprint arXiv:2106.13427, 2021. doi: 10.48550/arXiv.2106.13427.
  43. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.
  44. Miller, T. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, 267:1–38, 2019. doi: 10.1016/j.artint.2018.07.007.
  45. Miller, T. Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven Decision Support using Evaluative AI. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp.  333–342, 2023.
  46. Molnar, C. Interpretable Machine Learning. 2 edition, 2022. URL https://christophm.github.io/interpretable-ml-book.
  47. From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI. ACM Computing Surveys, 55(13s):1–42, 2023. doi: 10.1145/3583558.
  48. An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data. Ecological modelling, 178(3-4):389–397, 2004. doi: 10.1016/j.ecolmodel.2004.03.013.
  49. Machine learning and statistical analysis for materials science: stability and transferability of fingerprint descriptors and chemical insights. Chemistry of Materials, 29(10):4190–4201, 2017. doi: 10.1021/acs.chemmater.6b04229.
  50. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
  51. Deep learning and process understanding for data-driven Earth system science. Nature, 566(7743):195–204, 2019.
  52. Model-agnostic interpretability of machine learning. arXiv preprint arXiv:1606.05386, 2016a.
  53. ” why should i trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp.  1135–1144, 2016b.
  54. Explainable Machine Learning for Scientific Insights and Discoveries. IEEE Access, 8:42200–42216, 2020a. doi: 10.1109/ACCESS.2020.2976199.
  55. Explainable machine learning for scientific insights and discoveries. Ieee Access, 8:42200–42216, 2020b.
  56. Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence, 1(5):206–215, 2019. doi: 10.1038/s42256-019-0048-x.
  57. Axiomatic Attribution for Deep Networks. In Precup, D. and Teh, Y. W. (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp.  3319–3328. PMLR, 06–11 Aug 2017. URL https://proceedings.mlr.press/v70/sundararajan17a.html.
  58. Scientific machine learning benchmarks. Nature Reviews Physics, 4(6):413–420, 2022.
  59. Sphractal: Estimating the Fractal Dimension of Surfaces Computed from Precise Atomic Coordinates via Box-Counting Algorithm, 2024.
  60. On the safety of machine learning: Cyber-physical systems, decision sciences, and data products. Big data, 5(3):246–255, 2017.
  61. Informed machine learning–a taxonomy and survey of integrating prior knowledge into learning systems. IEEE Transactions on Knowledge and Data Engineering, 35(1):614–633, 2021.
  62. Machine learning for healthcare: on the verge of a major shift in healthcare epidemiology. Clinical infectious diseases, 66(1):149–153, 2018.
  63. Breast Cancer Wisconsin (Diagnostic). UCI Machine Learning Repository, 1995. DOI: https://doi.org/10.24432/C5DW2B.
  64. Exploring the whole rashomon set of sparse decision trees. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, volume 35, pp.  14071–14084. Curran Associates, Inc., 2022. doi: 10.48550/arXiv.2209.08040.
  65. Explainable machine learning in materials science. npj Computational Materials, 8(1):204, 2022. doi: 10.1038/s41524-022-00884-7.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Sichao Li (7 papers)
  2. Amanda Barnard (4 papers)
  3. Xin Wang (1307 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets