Papers
Topics
Authors
Recent
Detailed Answer
Quick Answer
Concise responses based on abstracts only
Detailed Answer
Well-researched responses based on abstracts and relevant paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses
Gemini 2.5 Flash
Gemini 2.5 Flash 84 tok/s
Gemini 2.5 Pro 45 tok/s Pro
GPT-5 Medium 28 tok/s Pro
GPT-5 High 21 tok/s Pro
GPT-4o 92 tok/s Pro
GPT OSS 120B 425 tok/s Pro
Kimi K2 157 tok/s Pro
2000 character limit reached

FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methods (2308.06248v1)

Published 11 Aug 2023 in cs.CV and cs.LG

Abstract: The field of explainable artificial intelligence (XAI) aims to uncover the inner workings of complex deep neural models. While being crucial for safety-critical domains, XAI inherently lacks ground-truth explanations, making its automatic evaluation an unsolved problem. We address this challenge by proposing a novel synthetic vision dataset, named FunnyBirds, and accompanying automatic evaluation protocols. Our dataset allows performing semantically meaningful image interventions, e.g., removing individual object parts, which has three important implications. First, it enables analyzing explanations on a part level, which is closer to human comprehension than existing methods that evaluate on a pixel level. Second, by comparing the model output for inputs with removed parts, we can estimate ground-truth part importances that should be reflected in the explanations. Third, by mapping individual explanations into a common space of part importances, we can analyze a variety of different explanation types in a single common framework. Using our tools, we report results for 24 different combinations of neural models and XAI methods, demonstrating the strengths and weaknesses of the assessed methods in a fully automatic and systematic manner.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (69)
  1. Quantifying attention flow in transformers. In ACL, pages 4190–4197, 2020.
  2. Debugging tests for model explanations. In NeurIPS*2020, pages 700–712.
  3. Towards robust interpretability with self-explaining neural networks. In NeurIPS*2018, pages 7786–7795.
  4. Towards better understanding of gradient-based attribution methods for deep neural networks. In ICLR, 2018.
  5. Hassan Aziz. Comparison between field research and controlled laboratory research. Archives of Clinical and Biomedical Research, 1:101–104, 04 2017.
  6. Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet. In ICLR, 2019.
  7. A naturalistic open source movie for optical flow evaluation. In ECCV, volume 4, pages 611–625. 2012.
  8. Convolutional dynamic alignment networks for interpretable classifications. In CVPR, pages 10029–10038, 2021.
  9. B-cos networks: Alignment is all we need for interpretability. In CVPR, pages 10319–10328, 2022.
  10. Explaining image classifiers by counterfactual generation. In ICLR, 2019.
  11. Transformer interpretability beyond attention visualization. In CVPR, pages 782–791, 2021.
  12. This looks like that: Deep learning for interpretable image recognition. In NeurIPS*2019, pages 8928–8939.
  13. Learning to explain: An information-theoretic perspective on model interpretation. In ICML, pages 882–891, 2018.
  14. Explaining neural networks semantically and quantitatively. In ICCV, pages 9186–9195, 2019.
  15. What I cannot predict, I do not understand: A human-centered evaluation framework for explainability methods. arXiv:2112.04417 [cs.CV], 2021.
  16. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In NeurIPS*2018, pages 590–601.
  17. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
  18. Flownet: Learning optical flow with convolutional networks. In ICCV, pages 2758–2766, 2017.
  19. Towards explanation of DNN-based prediction with guided feature inversion. In SIGKDD, pages 1358–1367, 2018.
  20. Improving performance of deep learning models with axiomatic attribution priors and expected gradients. Nature Machine Intelligence, 3(7):620–631, 2021.
  21. Interpretable explanations of black boxes by meaningful perturbation. In ICCV, pages 3449–3457, 2017.
  22. Counterfactual visual explanations. In ICML, pages 2376–2384, 2019.
  23. The out-of-distribution problem in explainability and search methods for feature importance explanations. In NeurIPS*2021, pages 3650–3666.
  24. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016.
  25. Quantus: An explainable AI toolkit for responsible evaluation of neural network explanations. arXiv:2202.06861 [cs.LG], 2022.
  26. Generating visual explanations. In ECCV, volume 9908, pages 3–19, 2016.
  27. Fast axiomatic attribution for neural networks. In NeurIPS*2021, pages 19513–19524.
  28. A benchmark for interpretability methods in deep neural networks. In NeurIPS*2019, pages 9734–9745.
  29. CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning. In CVPR, pages 1988–1997, 2017.
  30. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In ICML, pages 2673–2682, 2018.
  31. HIVE: Evaluating the human interpretability of visual explanations. In ECCV, volume 13672, pages 280–298, 2022.
  32. Donald E. Knuth. Two notes on notation. The American Mathematical Monthly, 99(5):403–422, 1992.
  33. Understanding neural networks through representation erasure. arXiv:1612.08220 [cs.CL], 2016.
  34. Visual explanation by interpretation: Improving visual feedback capabilities of deep neural networks. In ICLR, 2019.
  35. Concepts. In The Stanford Encyclopedia of Philosophy. 2022.
  36. The effects of demand characteristics on research participant behaviours in non-laboratory settings: A systematic review. PLOS ONE, 7(6):1–6, 06 2012.
  37. Chapter 3 - Categorization. Cognitive Science, pages 99–143, 1999.
  38. From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable AI. arXiv:2201.08164 [cs.AI], 2022.
  39. Neural prototype trees for interpretable fine-grained image recognition. In CVPR, pages 14933–14943, 2021.
  40. The effectiveness of feature attribution methods and its correlation with automatic evaluation scores. In NeurIPS*2021, pages 26422–26436.
  41. Generative causal explanations of black-box classifiers. In NeurIPS*2020, pages 5453–5467.
  42. Robust change captioning. In ICCV, pages 4623–4632, 2019.
  43. Elements of causal inference: Foundations and learning algorithms. The MIT Press, 2017.
  44. RISE: randomized input sampling for explanation of black-box models. In BMVC, page 151, 2018.
  45. Explainability methods for graph convolutional neural networks. In CVPR, pages 10772–10781, 2019.
  46. Towards better understanding attribution methods. In CVPR, pages 10213–10222, 2022.
  47. ”Why should I trust you?”: Explaining the predictions of any classifier. In ACM SIGKDD, pages 1135–1144, 2016.
  48. Playing for data: Ground truth from computer games. In ECCV, volume 1, pages 102–118, 2016.
  49. Concepts and Categories: Memory, Meaning, and Metaphysics. In The Oxford Handbook of Thinking and Reasoning. Oxford University Press, 2012.
  50. Right for the right reasons: Training differentiable models by constraining their explanations. In IJCAI, pages 2662–2670, 2017.
  51. ImageNet large scale visual recognition challenge. Int. J. Comput. Vision, 115(13):211–252, 2015.
  52. Evaluating the visualization of what a deep neural network has learned. IEEE Trans. Neural Networks Learn. Syst., 28(11):2660–2673, 2017.
  53. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In ICCV, pages 618–626, 2017.
  54. How useful are the machine-generated interpretations to general users? A human evaluation on guessing the incorrectly predicted labels. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 8(1):168–172, Oct. 2020.
  55. Learning important features through propagating activation differences. In ICML, volume 70, pages 3145–3153, 2017.
  56. Not just a black box: Learning important features through propagating activation differences. arXiv:1605.01713 [cs.LG], 2016.
  57. Deep inside convolutional networks: Visualising image classification models and saliency maps. In ICLR, 2014.
  58. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
  59. Axiomatic attribution for deep networks. In ICML, pages 3319–3328, 2017.
  60. Interpretable and fine-grained visual explanations for convolutional neural networks. In CVPR, pages 9097–9107, 2019.
  61. The Caltech-UCSD Birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011.
  62. Re-calibrating feature attributions for model interpretation. In ICLR, 2023.
  63. Visualizing and understanding convolutional networks. In ECCV, volume 1, pages 818–833, 2014.
  64. Categorization, concept learning, and behavior analysis: An introduction. Journal of the Experimental Analysis of Behavior, 78:237–48, 12 2002.
  65. Top-down neural attention by excitation backprop. In ECCV, volume 1, pages 543–559, 2016.
  66. Interpretable convolutional neural networks. In CVPR, pages 8827–8836, 2018.
  67. Interpreting CNNs via decision trees. In CVPR, pages 6261–6270, 2019.
  68. Predicting effects of noncoding variants with deep learning–based sequence model. Nature Methods, 12(10):931–934, 2015.
  69. Visualizing deep neural network decisions: Prediction difference analysis. In ICLR, 2017.
Citations (19)
View on Semantic Scholar
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Ai Generate Text Spark Streamline Icon: https://streamlinehq.com

Paper Prompts

Sign up for free to create and run prompts on this paper using GPT-5.

List Streamline Icon: https://streamlinehq.com Explore 10 Community Prompts
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now