Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Interactive Visual Feature Search (2211.15060v2)

Published 28 Nov 2022 in cs.CV

Abstract: Many visualization techniques have been created to explain the behavior of computer vision models, but they largely consist of static diagrams that convey limited information. Interactive visualizations allow users to more easily interpret a model's behavior, but most are not easily reusable for new models. We introduce Visual Feature Search, a novel interactive visualization that is adaptable to any CNN and can easily be incorporated into a researcher's workflow. Our tool allows a user to highlight an image region and search for images from a given dataset with the most similar model features. We demonstrate how our tool elucidates different aspects of model behavior by performing experiments on a range of applications, such as in medical imaging and wildlife classification.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. Tensorflow: A system for large-scale machine learning. In USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2016.
  2. Pass: An imagenet replacement for self-supervised pretraining without humans. NeurIPS Track on Datasets and Benchmarks, 2021.
  3. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7):e0130140, 2015.
  4. Network dissection: Quantifying interpretability of deep visual representations. In CVPR, 2017.
  5. Semantic photo manipulation with a generative image prior. SIGGRAPH, 2019.
  6. The iwildcam 2020 competition dataset. arXiv preprint arXiv:2004.10340, 2020.
  7. G. Bradski. The OpenCV Library. Dr. Dobb’s Journal of Software Tools, 2000.
  8. Activation atlas. Distill, 2019.
  9. Deep learning for instance retrieval: A survey, 2021. URL https://arxiv.org/abs/2101.11282.
  10. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020.
  11. TorchXRayVision: A library of chest X-ray datasets and models. In Medical Imaging with Deep Learning, 2022. URL https://github.com/mlmed/torchxrayvision.
  12. S. R. Dubey. A decade survey of content based image retrieval using deep learning. IEEE Transactions on Circuits and Systems for Video Technology, 2021.
  13. R. Fong and A. Vedaldi. Interpretable explanations of black boxes by meaningful perturbation. In ICCV, 2017.
  14. Interactive similarity overlays. In VISxAI, 2021. URL https://www.ruthfong.com/projects/interactive_overlay/.
  15. A. W. Harley. An interactive node-link visualization of convolutional neural networks. In International Symposium on Visual Computing (ISVC), 2015.
  16. Deep residual learning for image recognition. In CVPR, 2016.
  17. Natural adversarial examples. CVPR, 2021.
  18. Densely connected convolutional networks, 2018.
  19. Wilds: A benchmark of in-the-wild distribution shifts, 2021.
  20. Viscuit: Visual auditor for bias in cnn image classifier. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21475–21483, June 2022.
  21. A. Mahendran and A. Vedaldi. Visualizing deep convolutional neural networks using natural pre-images. IJCV, 2016.
  22. zarr-developers/zarr-python: v2.4.0, Jan. 2020. URL https://doi.org/10.5281/zenodo.3773450.
  23. A. P. Norton and Y. Qi. Adversarial-playground: A visualization suite showing how adversarial examples fool deep learning. In VizSec, 2017.
  24. Feature visualization. Distill, 2(11):e7, 2017.
  25. The building blocks of interpretability. Distill, 3(3):e10, 2018.
  26. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS. 2019.
  27. Imagenet large scale visual recognition challenge. IJCV, 115(3):211–252, 2015.
  28. Editing a classifier by rewriting its prediction rules. In NeurIPS, 2021.
  29. OpenAI Microscope, 2020. URL https://openai.com/blog/microscope/.
  30. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In ICCV, 2017.
  31. Deep inside convolutional networks: Visualising image classification models and saliency maps. In ICLR workshop, 2014.
  32. Smoothgrad: removing noise by adding noise. arXiv, 2017.
  33. Learning robust global representations by penalizing local predictive power. In NeurIPS, 2019.
  34. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
  35. B. Webster. Now anyone can explore machine learning, no coding required. Online, 2017. URL https://www.blog.google/topics/machine-learning/now-anyone-can-explore-machine-learning-no-coding-required.
  36. The what-if tool: Interactive probing of machine learning models. TCVG, 2019.
  37. M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In ECCV, 2014.
  38. Sift meets cnn: A decade survey of instance retrieval. IEEE transactions on pattern analysis and machine intelligence, 40(5):1224–1244, 2017.
  39. Learning deep features for discriminative localization. In CVPR, 2016.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now