Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

AttributionScanner: A Visual Analytics System for Model Validation with Metadata-Free Slice Finding (2401.06462v3)

Published 12 Jan 2024 in cs.CV and cs.HC

Abstract: Data slice finding is an emerging technique for validating ML models by identifying and analyzing subgroups in a dataset that exhibit poor performance, often characterized by distinct feature sets or descriptive metadata. However, in the context of validating vision models involving unstructured image data, this approach faces significant challenges, including the laborious and costly requirement for additional metadata and the complex task of interpreting the root causes of underperformance. To address these challenges, we introduce AttributionScanner, an innovative human-in-the-loop Visual Analytics (VA) system, designed for metadata-free data slice finding. Our system identifies interpretable data slices that involve common model behaviors and visualizes these patterns through an Attribution Mosaic design. Our interactive interface provides straightforward guidance for users to detect, interpret, and annotate predominant model issues, such as spurious correlations (model biases) and mislabeled data, with minimal effort. Additionally, it employs a cutting-edge model regularization technique to mitigate the detected issues and enhance the model's performance. The efficacy of AttributionScanner is demonstrated through use cases involving two benchmark datasets, with qualitative and quantitative evaluations showcasing its substantial effectiveness in vision model validation, ultimately leading to more reliable and accurate models.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018.
  2. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information fusion, 58:82–115, 2020.
  3. Bridging the gap between ML solutions and their business requirements using feature interactions. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 1048–1058, 2019.
  4. The quickhull algorithm for convex hulls. ACM Transactions on Mathematical Software (TOMS), 22(4):469–483, 1996.
  5. FairVis: Visual analytics for discovering intersectional bias in machine learning. In Proceedings of 2019 IEEE Conference on Visual Analytics Science and Technology (VAST), pp. 46–56. IEEE, 2019.
  6. Activation atlas. Distill, 2019. https://distill.pub/2019/activation-atlas. doi: 10 . 23915/distill . 00015
  7. Slice finder: Automated data slicing for model validation. In 2019 IEEE 35th International Conference on Data Engineering (ICDE), pp. 1550–1553. IEEE, 2019.
  8. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255. Ieee, 2009.
  9. The spotlight: A general method for discovering systematic errors in deep learning models. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 1962–1981, 2022.
  10. Visualizing higher-layer features of a deep network. University of Montreal, 1341(3):1, 2009.
  11. Domino: Discovering systematic errors with cross-modal embeddings. In International Conference on Learning Representations, 2022.
  12. Interpretation of neural networks is fragile. In Proceedings of the AAAI conference on artificial intelligence, vol. 33, pp. 3681–3688, 2019.
  13. D. Gunning. Explainable artificial intelligence (xai). Defense advanced research projects agency (DARPA), nd Web, 2(2):1, 2017.
  14. J. Y. Halpern and J. Pearl. Causes and explanations: A structural-model approach. part i: Causes. The British journal for the philosophy of science, 2020.
  15. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
  16. Summit: Scaling deep learning interpretability by visualizing activation and attribution summarizations. IEEE transactions on visualization and computer graphics, 26(1):1096–1106, 2019.
  17. Visual exploration of machine learning results using data cube analysis. In Proceedings of the Workshop on Human-In-the-Loop Data Analytics, pp. 1–6, 2016.
  18. Udis: Unsupervised discovery of bias in deep visual recognition models. In British Machine Vision Conference (BMVC), vol. 1, p. 3, 2021.
  19. Towards mitigating spurious correlations in image classifiers with simple yes-no feedback.
  20. Viscuit: Visual auditor for bias in cnn image classifier. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21475–21483, 2022.
  21. Explainable ai: A review of machine learning interpretability methods. Entropy, 23(1):18, 2020.
  22. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
  23. Co-designing checklists to understand organizational challenges and opportunities around fairness in ai. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–14, 2020.
  24. A. Mahendran and A. Vedaldi. Understanding deep image representations by inverting them. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5188–5196, 2015.
  25. Causal transportability for visual recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7521–7531, 2022.
  26. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426, 2018.
  27. Feature visualization. Distill, 2017. https://distill.pub/2017/feature-visualization. doi: 10 . 23915/distill . 00007
  28. Automated detection of covid-19 cases using deep neural networks with x-ray images. Computers in biology and medicine, 121:103792, 2020.
  29. Looking for trouble: Analyzing classifier behavior via pattern divergence. In Proceedings of the 2021 International Conference on Management of Data, pp. 1400–1412, 2021.
  30. How divergent is your data? Proceedings of the VLDB Endowment, 14(12):2835–2838, 2021.
  31. J. Pearl. Causality. Cambridge university press, 2009.
  32. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825–2830, 2011.
  33. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421, 2018.
  34. Interpretations are useful: penalizing explanations to align neural networks with prior knowledge. In International conference on machine learning, pp. 8116–8126. PMLR, 2020.
  35. S. Sagadeeva and M. Boehm. Sliceline: Fast, linear-algebra-based slice finding for ml model debugging. In Proceedings of the 2021 International Conference on Management of Data, pp. 2290–2299, 2021.
  36. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731, 2019.
  37. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pp. 618–626, 2017.
  38. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  39. Core risk minimization using salient imagenet. arXiv, 2022.
  40. No subclass left behind: Fine-grained robustness in coarse-grained classification problems. Advances in Neural Information Processing Systems, 33:19339–19352, 2020.
  41. E. Tjoa and C. Guan. A survey on explainable artificial intelligence (xai): Toward medical xai. IEEE transactions on neural networks and learning systems, 32(11):4793–4813, 2020.
  42. The caltech-ucsd birds-200-2011 dataset. 2011.
  43. Discover and cure: Concept-aware mitigation of spurious correlation. arXiv preprint arXiv:2305.00650, 2023.
  44. Errudite: Scalable, reproducible, and testable error analysis. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
  45. Mitigating spurious correlations in multi-modal models during fine-tuning. arXiv preprint arXiv:2304.03916, 2023.
  46. Correct-n-contrast: A contrastive approach for improving robustness to spurious correlations. arXiv preprint arXiv:2203.01517, 2022.
  47. Sliceteller: A data slice-driven approach for machine learning model validation. IEEE Transactions on Visualization and Computer Graphics, 29(1):842–852, 2022.
  48. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2921–2929, 2016.
  49. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452–1464, 2017.
  50. Learning with local and global consistency. Advances in neural information processing systems, 16, 2003.
  51. Care: Class attention to regions of lesion for classification on imbalanced data. In International Conference on Medical Imaging with Deep Learning, pp. 588–597. PMLR, 2019.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Xiwei Xuan (9 papers)
  2. Jorge Piazentin Ono (8 papers)
  3. Liang Gou (18 papers)
  4. Kwan-Liu Ma (80 papers)
  5. Liu Ren (57 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets