Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Negative Label Guided OOD Detection with Pretrained Vision-Language Models (2403.20078v1)

Published 29 Mar 2024 in cs.CV and cs.LG

Abstract: Out-of-distribution (OOD) detection aims at identifying samples from unknown classes, playing a crucial role in trustworthy models against errors on unexpected inputs. Extensive research has been dedicated to exploring OOD detection in the vision modality. Vision-LLMs (VLMs) can leverage both textual and visual information for various multi-modal applications, whereas few OOD detection methods take into account information from the text modality. In this paper, we propose a novel post hoc OOD detection method, called NegLabel, which takes a vast number of negative labels from extensive corpus databases. We design a novel scheme for the OOD score collaborated with negative labels. Theoretical analysis helps to understand the mechanism of negative labels. Extensive experiments demonstrate that our method NegLabel achieves state-of-the-art performance on various OOD detection benchmarks and generalizes well on multiple VLM architectures. Furthermore, our method NegLabel exhibits remarkable robustness against diverse domain shifts. The codes are available at https://github.com/tmlr-group/NegLabel.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Certifiably adversarially robust detection of out-of-distribution data. In NeurIPS, 2020.
  2. Food-101–mining discriminative components with random forests. In ECCV, 2014.
  3. Altclip: Altering the language encoder in clip for extended language capabilities. arXiv preprint, 2022.
  4. Describing textures in the wild. In CVPR, 2014.
  5. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
  6. Learning confidence for out-of-distribution detection in neural networks. arXiv preprint arXiv:1802.04865, 2018.
  7. Extremely simple activation shaping for out-of-distribution detection. 2023.
  8. Unknown-aware object detection: Learning what you don’t know from videos in the wild. In CVPR, 2022.
  9. Zero-shot out-of-distribution detection based on the pre-trained model clip. In AAAI, 2022.
  10. Christiane Fellbaum. WordNet: An Electronic Lexical Database. Bradford Books, 1998. URL https://mitpress.mit.edu/9780262561167/.
  11. Exploring the limits of out-of-distribution detection. In NeurIPS, 2021.
  12. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR, 2017.
  13. The many faces of robustness: A critical analysis of out-of-distribution generalization. ICCV, 2021a.
  14. Natural adversarial examples. CVPR, 2021b.
  15. The inaturalist species classification and detection dataset. In CVPR, 2018.
  16. Generalized ODIN: detecting out-of-distribution image without learning from out-of-distribution data. In CVPR, 2020a.
  17. Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data. In CVPR, 2020b.
  18. On the importance of gradients for detecting distributional shifts in the wild. In NeurIPS, 2021.
  19. Scaling up visual and vision-language representation learning with noisy text supervision. In ICML, 2021.
  20. Detecting out-of-distribution data through in-distribution class prior. In ICML, 2023.
  21. 3d object representations for fine-grained categorization. ICCV, 2013.
  22. Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009.
  23. A tutorial on energy-based learning. Predicting structured data, 1(0), 2006.
  24. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurIPS, 2018.
  25. Enhancing the reliability of out-of-distribution image detection in neural networks. In ICLR, 2018.
  26. Microsoft coco: Common objects in context. In ECCV, 2014.
  27. Visual instruction tuning. In NeurIPS, 2023.
  28. Energy-based out-of-distribution detection. In NeurIPS, 2020.
  29. Large-scale long-tailed recognition in an open world. In CVPR, 2019.
  30. Delving into out-of-distribution detection with vision-language representations. In NeurIPS, 2022a.
  31. On the impact of spurious correlation for out-of-distribution detection. In AAAI, 2022b.
  32. Cider: Exploiting hyperspherical embeddings for out-of-distribution detection. ICLR, 2023.
  33. Ravi Parameswaran. Statistics for experimenters: an introduction to design, data analysis, and model building. JMR, Journal of Marketing Research, 16(000002):291, 1979.
  34. Cats and dogs. In CVPR, 2012.
  35. The case against accuracy estimation for comparing induction algorithms. In ICML, 1998.
  36. Learning transferable visual models from natural language supervision. In ICML, 2021.
  37. Do imagenet classifiers generalize to imagenet? In ICML, 2019.
  38. Dice: Leveraging sparsification for out-of-distribution detection. In ECCV, 2022.
  39. React: Out-of-distribution detection with rectified activations. In NeurIPS, 2021.
  40. Out-of-distribution detection with deep nearest neighbors. ICML, 2022.
  41. Csi: Novelty detection via contrastive learning on distributionally shifted instances. NeurIPS, 2020.
  42. Non-parametric outlier synthesis. ICLR, 2023.
  43. Open-set recognition: A good closed-set classifier is all you need. In ICLR, 2022.
  44. The caltech-ucsd birds-200-2011 dataset. 2011.
  45. Learning robust global representations by penalizing local predictive power. In NeurIPS, 2019.
  46. Vim: Out-of-distribution with virtual-logit matching. In CVPR, 2022.
  47. Can multi-label classification networks know what they don’t know? In NeurIPS, 2021a.
  48. Clipn for zero-shot ood detection: Teaching clip to say no. ICCV, 2023.
  49. Energy-based open-world uncertainty modeling for confidence calibration. In ICCV, 2021b.
  50. Mitigating neural network overconfidence with logit normalization. In ICML, 2022.
  51. SUN database: Large-scale scene recognition from abbey to zoo. In CVPR, 2010.
  52. Groupvit: Semantic segmentation emerges from text supervision. In CVPR, 2022.
  53. Semantically coherent out-of-distribution detection. In ICCV, 2021.
  54. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452–1464, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Xue Jiang (82 papers)
  2. Feng Liu (1212 papers)
  3. Zhen Fang (58 papers)
  4. Hong Chen (230 papers)
  5. Tongliang Liu (251 papers)
  6. Feng Zheng (117 papers)
  7. Bo Han (282 papers)
Citations (18)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets