Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Hyperbolic Metric Learning for Visual Outlier Detection (2403.15260v2)

Published 22 Mar 2024 in cs.CV

Abstract: Out-Of-Distribution (OOD) detection is critical to deploy deep learning models in safety-critical applications. However, the inherent hierarchical concept structure of visual data, which is instrumental to OOD detection, is often poorly captured by conventional methods based on Euclidean geometry. This work proposes a metric framework that leverages the strengths of Hyperbolic geometry for OOD detection. Inspired by previous works that refine the decision boundary for OOD data with synthetic outliers, we extend this method to Hyperbolic space. Interestingly, we find that synthetic outliers do not benefit OOD detection in Hyperbolic space as they do in Euclidean space. Furthermore we explore the relationship between OOD detection performance and Hyperbolic embedding dimension, addressing practical concerns in resource-constrained environments. Extensive experiments show that our framework improves the FPR95 for OOD detection from 22\% to 15\% and from 49% to 28% on CIFAR-10 and CIFAR-100 respectively compared to Euclidean methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (78)
  1. GPr-Net: Geometric Prototypical Network for Point Cloud Few-Shot Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4178–4187, 2023.
  2. Hyperbolic image segmentation. In CVPR, pages 4443–4452, 2022.
  3. Constant Curvature Graph Convolutional Networks. In ICML, pages 486–496, 2020.
  4. Towards open world recognition. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015, pages 1893–1902. IEEE Computer Society, 2015.
  5. Discriminative out-of-distribution detection for semantic segmentation, 2018.
  6. In or out? fixing imagenet out-of-distribution detection evaluation. In ICML, 2023.
  7. Metric Spaces of Non-Positive Curvature. Springer, 1999.
  8. Hyperbolic Graph Convolutional Neural Networks. In NeurIPS, 2019.
  9. A rotated hyperbolic wrapped normal distribution for hierarchical representation learning. In NeurIPS, 2022.
  10. Describing textures in the wild. In CVPR, pages 3606–3613, 2014.
  11. A Hyperbolic-to-Hyperbolic Graph Convolutional Network. In CVPR, pages 154–163, 2021.
  12. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248–255, 2009.
  13. Li Deng. The MNIST database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Processing Magazine, 29(6):141–142, 2012.
  14. Hyperbolic image-text representations, 2023.
  15. SIREN: Shaping representations for detecting out-of-distribution objects. In Advances in Neural Information Processing Systems, 2022a.
  16. VOS: learning what you don’t know by virtual outlier synthesis. In ICLR. OpenReview.net, 2022b.
  17. Are we certain it’s anomalous? CVPRW, 2022.
  18. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In ICML, pages 1050–1059, 2016.
  19. Hyperbolic entailment cones for learning hierarchical embeddings. In ICML, pages 1632–1641, 2018.
  20. Selectivenet: A deep neural network with an integrated reject option. In ICML, pages 2151–2159, 2019.
  21. Robust T-Loss for Medical Image Segmentation. In MICCAI, pages 714–724, 2023a.
  22. SANO: Score-Based Diffusion Model for Anomaly Localization in Dermatology. In CVPRW, pages 2988–2994, 2023b.
  23. Clipped hyperbolic classifiers are super-hyperbolic classifiers. In CVPR, pages 1–10, 2022.
  24. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016.
  25. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In CVPR, pages 41–50, 2019.
  26. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR, 2017.
  27. Deep anomaly detection with outlier exposure. In ICLR, 2019.
  28. Scaling out-of-distribution detection for real-world settings. In ICML, pages 8759–8773, 2022.
  29. The inaturalist species classification and detection dataset. In CVPR, pages 8769–8778, 2018.
  30. Generalized ODIN: detecting out-of-distribution image without learning from out-of-distribution data. In CVPR, pages 10948–10957, 2020.
  31. On the importance of gradients for detecting distributional shifts in the wild. In NeurIPS, pages 677–689, 2021.
  32. OOD-MAML: meta-learning for few-shot out-of-distribution detection and classification. In NeurIPS, 2020.
  33. Training OOD detectors in their natural habitats. In ICML, pages 10848–10865, 2022.
  34. Supervised Contrastive Learning. In NeurIPS, pages 18661–18673, 2020.
  35. Hyperbolic image embeddings. In CVPR, pages 6417–6427, 2020.
  36. Being bayesian, even just a bit, fixes overconfidence in relu networks. In ICML, pages 5436–5446, 2020.
  37. Alex Krizhevsky. Learning multiple layers of features from tiny images, 2009.
  38. Matryoshka representation learning. In NeurIPS, pages 30233–30249, 2022.
  39. Simple and scalable predictive uncertainty estimation using deep ensembles. In NeurIPS, pages 6402–6413, 2017.
  40. Lorentzian Distance Learning for Hyperbolic Representations. In ICML, pages 3672–3681, 2019.
  41. John M. Lee. Riemannian Manifolds. Springer, New York, NY, 1997.
  42. Training confidence-calibrated classifiers for detecting out-of-distribution samples. In ICLR, 2018a.
  43. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurIPS, pages 7167–7177, 2018b.
  44. Fully Hyperbolic Convolutional Neural Networks for Computer Vision. In ICLR, 2023.
  45. Enhancing the reliability of out-of-distribution image detection in neural networks. In ICLR, 2018.
  46. Energy-based out-of-distribution detection. In NeurIPS, 2020.
  47. Sgdr: Stochastic gradient descent with warm restarts. ICLR, 2016.
  48. Decoupled weight decay regularization. In ICLR, 2019.
  49. A simple baseline for bayesian uncertainty in deep learning. In NeurIPS, pages 13132–13143, 2019.
  50. Andrey Malinin and Mark J. F. Gales. Predictive uncertainty estimation via prior networks. In NeurIPS, pages 7047–7058, 2018.
  51. Towards neural networks that provably know when they don’t know. In ICLR, 2020.
  52. Hyperbolic deep learning in computer vision: A survey, 2023.
  53. POEM: out-of-distribution detection with posterior sampling. In ICML, pages 15650–15665, 2022.
  54. How to exploit hyperspherical embeddings for out-of-distribution detection? In ICLR, 2023.
  55. The Numerical Stability of Hyperbolic Representation Learning. In ICML, pages 24925–24949, 2023.
  56. A Wrapped Normal Distribution on Hyperbolic Space for Gradient-Based Learning. In ICML, pages 4693–4702, 2019.
  57. Reading digits in natural images with unsupervised feature learning. In NeurIPS, 2011.
  58. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In CVPR, pages 427–436, 2015.
  59. Poincar\’e embeddings for learning hierarchical representations. In NeurIPS, 2017.
  60. Unsupervised hyperbolic representation learning via message passing auto-encoders. In CVPR, pages 5516–5526, 2021.
  61. Hyperbolic deep neural networks: A survey, 2021.
  62. A simple fix to mahalanobis distance for improving near-OOD detection. ArXiv preprint, abs/2106.09022, 2021.
  63. Representation tradeoffs for hyperbolic embeddings. In ICML, pages 4457–4466, 2018.
  64. SSD: A unified framework for self-supervised outlier detection. In ICLR, 2021.
  65. Hyperbolic neural networks++. In ICLR, 2021.
  66. Out-of-distribution detection with deep nearest neighbors. In ICML, pages 20827–20840, 2022.
  67. Non-parametric outlier synthesis. In ICLR, 2022.
  68. Poincaré GloVe: Hyperbolic Word Embeddings, 2018.
  69. Poincaré resnet. ICCV, 2023.
  70. Open-set recognition: A good closed-set classifier is all you need. In ICLR, 2022.
  71. ViM: Out-of-distribution with virtual-logit matching. In CVPR, 2022.
  72. Batchensemble: an alternative approach to efficient ensemble and lifelong learning. In ICLR, 2020.
  73. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv: 1504.06755, 2015.
  74. Semantically coherent out-of-distribution detection. In ICCV, pages 8281–8289, 2021.
  75. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop, 2015.
  76. OpenOOD v1.5: Enhanced benchmark for out-of-distribution detection, 2023.
  77. Places: A 10 million image database for scene recognition. IEEE TPAMI, 40(6):1452–1464, 2017.
  78. HyperText: Endowing FastText with Hyperbolic Geometry. In Findings of the Association for Computational Linguistics : EMNLP 2020, pages 1166–1171, Online, 2020. Association for Computational Linguistics.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now