Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Knowledge-Aware Neuron Interpretation for Scene Classification (2401.15820v1)

Published 29 Jan 2024 in cs.CV and cs.AI

Abstract: Although neural models have achieved remarkable performance, they still encounter doubts due to the intransparency. To this end, model prediction explanation is attracting more and more attentions. However, current methods rarely incorporate external knowledge and still suffer from three limitations: (1) Neglecting concept completeness. Merely selecting concepts may not sufficient for prediction. (2) Lacking concept fusion. Failure to merge semantically-equivalent concepts. (3) Difficult in manipulating model behavior. Lack of verification for explanation on original model. To address these issues, we propose a novel knowledge-aware neuron interpretation framework to explain model predictions for image scene classification. Specifically, for concept completeness, we present core concepts of a scene based on knowledge graph, ConceptNet, to gauge the completeness of concepts. Our method, incorporating complete concepts, effectively provides better prediction explanations compared to baselines. Furthermore, for concept fusion, we introduce a knowledge graph-based method known as Concept Filtering, which produces over 23% point gain on neuron behaviors for neuron interpretation. At last, we propose Model Manipulation, which aims to study whether the core concepts based on ConceptNet could be employed to manipulate model behavior. The results show that core concepts can effectively improve the performance of original model by over 26%.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (31)
  1. Rethinking Interpretation: Input-Agnostic Saliency Mapping of Deep Visual Classifiers. In Proceedings of the AAAI Conference on Artificial Intelligence, 178–186. AAAI Press.
  2. Network Dissection: Quantifying Interpretability of Deep Visual Representations. In IEEE Conference on Computer Vision and Pattern Recognition,CVPR, 3319–3327. IEEE Computer Society.
  3. Intrinsic images in the wild. ACM Trans. Graph., 33(4): 159:1–159:12.
  4. Translating Embeddings for Modeling Multi-relational Data. Curran Associates Inc., 2787–2795.
  5. A Survey on Explainable AI for 6G O-RAN: Architecture, Use Cases, Challenges and Research Directions. arXiv:2307.00319.
  6. Hierarchical classification: combining Bayes with SVM. In Cohen, W. W.; and Moore, A. W., eds., Machine Learning, Proceedings of the Twenty-Third International Conference, ICML, volume 148, 177–184. ACM.
  7. Knowledge-based Transfer Learning Explanation. In Proc. of the International Conference on Principles of Knowledge Representation and Reasoning (KR2018), 349–358.
  8. Knowledge-Driven Stock Trend Prediction and Explanation via Temporal Convolutional Network. In Proc. of the World Wide Web Conference (WWW 2019), 678–685.
  9. A Peek Into the Reasoning of Neural Networks: Interpreting With Structural Visual Concepts. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, 2195–2204. Computer Vision Foundation / IEEE.
  10. Towards Automatic Concept-based Explanations. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems, NeurIPS, 9273–9282.
  11. Trigger-Argument based Explanation for Event Detection. In Findings of the Association for Computational Linguistics: ACL 2023, 5046–5058. Toronto, Canada: Association for Computational Linguistics.
  12. Knowledge Graph Embedding via Dynamic Mapping Matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL, 687–696. The Association for Computer Linguistics.
  13. Alignment Rationale for Natural Language Inference. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP, 5372–5387. Association for Computational Linguistics.
  14. Compositional Convolutional Neural Networks: A Robust and Interpretable Model for Object Recognition Under Occlusion. Int. J. Comput. Vis., 129(3): 736–760.
  15. Towards a definition of the concept of scene: Communicating on the basis of things that matter. Sociologija, 55(3): 353–374.
  16. Explaining nonlinear classification decisions with deep Taylor decomposition. Pattern Recognit., 65: 211–222.
  17. Compositional Explanations of Neurons. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems, NeurIPS, 11.
  18. CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks. arXiv:2204.10965.
  19. OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774.
  20. Reasoning Web: Logical Foundation of Knowledge Graph Construction and Querying Answering. Springer.
  21. Exploiting Linked Data and Knowledge Graphs for Large Organisations. Springer.
  22. Shahroudnejad, A. 2021. A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks. arXiv:2102.01792.
  23. ProjE: Embedding Projection for Knowledge Graph Completion. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI, 1236–1242. AAAI Press.
  24. Interpreting Face Inference Models Using Hierarchical Network Dissection. Int. J. Comput. Vis., 130(5): 1277–1292.
  25. LaMDA: Language Models for Dialog Applications. arXiv:2201.08239.
  26. MANE: Model-Agnostic Non-linear Explanations for Deep Learning Model. In 2020 IEEE World Congress on Services, SERVICES 2020, Beijing, China, October 18-23, 2020, 33–36. IEEE.
  27. Towards Effective Online Knowledge Graph Fusion. In Proc. of 14th International Semantic Web Conference (ISWC 2015).
  28. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. arXiv:1412.6575.
  29. On Completeness-aware Concept-Based Explanations in Deep Neural Networks. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems, NeurIPS, 12. Red Hook, NY, USA: Curran Associates Inc.
  30. Scene Parsing through ADE20K Dataset. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, 5122–5130. IEEE Computer Society.
  31. TBox Learning from Incomplete Data by Inference in BelNet+. Knowledge Based Systems, 30–40.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now