Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Modeling Uncertainties of Self-explaining Neural Networks via Conformal Prediction (2401.01549v1)

Published 3 Jan 2024 in cs.LG

Abstract: Despite the recent progress in deep neural networks (DNNs), it remains challenging to explain the predictions made by DNNs. Existing explanation methods for DNNs mainly focus on post-hoc explanations where another explanatory model is employed to provide explanations. The fact that post-hoc methods can fail to reveal the actual original reasoning process of DNNs raises the need to build DNNs with built-in interpretability. Motivated by this, many self-explaining neural networks have been proposed to generate not only accurate predictions but also clear and intuitive insights into why a particular decision was made. However, existing self-explaining networks are limited in providing distribution-free uncertainty quantification for the two simultaneously generated prediction outcomes (i.e., a sample's final prediction and its corresponding explanations for interpreting that prediction). Importantly, they also fail to establish a connection between the confidence values assigned to the generated explanations in the interpretation layer and those allocated to the final predictions in the ultimate prediction layer. To tackle the aforementioned challenges, in this paper, we design a novel uncertainty modeling framework for self-explaining networks, which not only demonstrates strong distribution-free uncertainty modeling performance for the generated explanations in the interpretation layer but also excels in producing efficient and effective prediction sets for the final predictions based on the informative high-level basis explanations. We perform the theoretical analysis for the proposed framework. Extensive experimental evaluation demonstrates the effectiveness of the proposed uncertainty framework.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Openxai: Towards a transparent evaluation of model explanations. Advances in Neural Information Processing Systems, 35: 15784–15799.
  2. Towards robust interpretability with self-explaining neural networks. Advances in neural information processing systems, 31.
  3. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511.
  4. Explaining a black-box by using a deep variational information bottleneck approach. In Proceedings of the AAAI Conference on Artificial Intelligence, 11396–11404.
  5. Implications of Model Indeterminacy for Explanations of Automated Decisions. Advances in Neural Information Processing Systems, 35: 7810–7823.
  6. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), 39–57. Ieee.
  7. Interactive concept bottleneck models. In Proceedings of the AAAI Conference on Artificial Intelligence, 5948–5955.
  8. Exact and robust conformal inference methods for predictive machine learning with dependent data. In Conference On learning theory, 732–749. PMLR.
  9. Exact optimization of conformal predictors via incremental and decremental learning. In International Conference on Machine Learning, 1836–1845. PMLR.
  10. Deng, L. 2012. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6): 141–142.
  11. Self explaining neural networks: A review with extensions. Fairness, Accountability, Confidentiality and Transparency in AI.
  12. DL2: training and querying neural networks with logic. In International Conference on Machine Learning, 1931–1941. PMLR.
  13. Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems, 34: 1660–1672.
  14. Which explanation should i choose? a function approximation perspective to characterizing post hoc explanations. Advances in Neural Information Processing Systems, 35: 5256–5268.
  15. Addressing leakage in concept bottleneck models. Advances in Neural Information Processing Systems, 35: 23386–23397.
  16. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778.
  17. Human-centered concept explanations for neural networks. Neuro-Symbolic Artificial Intelligence: The State of the Art, 342(337): 2.
  18. Concept-Centric Transformers: Concept Transformers with Object-Centric Concept Learning for Interpretability. arXiv preprint arXiv:2305.15775.
  19. Towards automating model explanations with certified robustness guarantees. In Proceedings of the AAAI Conference on Artificial Intelligence, 6935–6943.
  20. Towards interpretation of pairwise learning. In Proceedings of the AAAI Conference on Artificial Intelligence, 4166–4173.
  21. Metapoison: Practical general-purpose clean-label data poisoning. Advances in Neural Information Processing Systems, 33: 12080–12091.
  22. Shapley-value based inductive conformal prediction. In Conformal and Probabilistic Prediction and Applications, 52–71. PMLR.
  23. Probabilistic Concept Bottleneck Models. arXiv preprint arXiv:2306.01574.
  24. Concept bottleneck models. In International conference on machine learning, 5338–5348. PMLR.
  25. Learning multiple layers of features from tiny images. University of Toronto.
  26. Instance-wise or class-wise? a tale of neighbor shapley for concept-based explanation. In Proceedings of the 29th ACM International Conference on Multimedia, 3664–3672.
  27. Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions. In Proceedings of the AAAI Conference on Artificial Intelligence.
  28. Mri reconstruction with interpretable pixel-wise operations using reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence, 792–799.
  29. Conformalized Fairness via Quantile Regression. Advances in Neural Information Processing Systems, 35: 11561–11572.
  30. Fair conformal predictors for applications in medical imaging. In Proceedings of the AAAI Conference on Artificial Intelligence, 12008–12016.
  31. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.
  32. Approximating Full Conformal Prediction at Scale via Influence Functions. In Proceedings of the AAAI Conference on Artificial Intelligence, 6631–6639.
  33. Relative attributing propagation: Interpreting the comparative contributions of individual units in deep neural networks. In Proceedings of the AAAI conference on artificial intelligence, 2501–2508.
  34. Ndiaye, E. 2022. Stable conformal prediction sets. In International Conference on Machine Learning, 16462–16479. PMLR.
  35. Selfexplain: A self-explaining architecture for neural text classifiers. arXiv preprint arXiv:2103.12279.
  36. ” Why should i trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 1135–1144.
  37. Attention-based interpretability with concept transformers. In International Conference on Learning Representations.
  38. Rudin, C. 2018. Please stop explaining black box models for high stakes decisions. Stat, 1050: 26.
  39. Conformal frequency estimation with sketched data. Advances in Neural Information Processing Systems, 35: 6589–6602.
  40. Understanding and enhancing robustness of concept-based models. In Proceedings of the AAAI Conference on Artificial Intelligence, 15127–15135.
  41. Reliable post hoc explanations: Modeling uncertainty in explainability. Advances in neural information processing systems, 34: 9391–9404.
  42. Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 180–186.
  43. T-sci: A two-stage conformal inference algorithm with guaranteed coverage for cox-mlp. In International Conference on Machine Learning, 10203–10213. PMLR.
  44. Predictive inference with feature conformal prediction. arXiv preprint arXiv:2210.00173.
  45. Conformal prediction under covariate shift. Advances in neural information processing systems, 32.
  46. Sequential predictive conformal inference for time series. In International Conference on Machine Learning, 38707–38727. PMLR.
  47. Concept-Level Model Interpretation From the Causal Aspect. IEEE Transactions on Knowledge and Data Engineering.
  48. On completeness-aware concept-based explanations in deep neural networks. Advances in neural information processing systems, 33: 20554–20565.
  49. Post-hoc Concept Bottleneck Models. In The Eleventh International Conference on Learning Representations.
  50. TabCBM: Concept-based Interpretable Neural Networks for Tabular Data. Transactions on Machine Learning Research.
  51. Learning to Select Prototypical Parts for Interpretable Sequential Data Modeling. In Proceedings of the AAAI Conference on Artificial Intelligence, 6612–6620.
  52. Automated Natural Language Explanation of Deep Visual Neurons with Large Models. arXiv preprint arXiv:2310.10708.
  53. What do deep nets learn? class-wise patterns revealed in the input space. arXiv preprint arXiv:2101.06898.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Wei Qian (51 papers)
  2. Chenxu Zhao (29 papers)
  3. Yangyi Li (2 papers)
  4. Fenglong Ma (66 papers)
  5. Chao Zhang (907 papers)
  6. Mengdi Huai (12 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now