Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Know Yourself Better: Diverse Discriminative Feature Learning Improves Open Set Recognition (2404.10370v1)

Published 16 Apr 2024 in cs.CV and cs.LG

Abstract: Open set recognition (OSR) is a critical aspect of machine learning, addressing the challenge of detecting novel classes during inference. Within the realm of deep learning, neural classifiers trained on a closed set of data typically struggle to identify novel classes, leading to erroneous predictions. To address this issue, various heuristic methods have been proposed, allowing models to express uncertainty by stating "I don't know." However, a gap in the literature remains, as there has been limited exploration of the underlying mechanisms of these methods. In this paper, we conduct an analysis of open set recognition methods, focusing on the aspect of feature diversity. Our research reveals a significant correlation between learning diverse discriminative features and enhancing OSR performance. Building on this insight, we propose a novel OSR approach that leverages the advantages of feature diversity. The efficacy of our method is substantiated through rigorous evaluation on a standard OSR testbench, demonstrating a substantial improvement over state-of-the-art methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Cifar-10 (canadian institute for advanced research). a.
  2. Cifar-100 (canadian institute for advanced research). b.
  3. Towards open set deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16), pages 1563–1572, 2016.
  4. A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses. In European conference on computer vision, pages 548–564. Springer, 2020.
  5. E. Cantu-Paz. Feature subset selection, class separability, and genetic algorithms. In Genetic and Evolutionary Computation–GECCO 2004: Genetic and Evolutionary Computation Conference, Seattle, WA, USA, June 26-30, 2004. Proceedings, Part I, pages 959–970. Springer, 2004.
  6. Open-set recognition with gaussian mixture variational autoencoders. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 6877–6884, 2021.
  7. Regularizing neural network training via identity-wise discriminative feature suppression. In 2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pages 1–7. IEEE, 2022.
  8. Learning open set network with discriminative reciprocal points. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, pages 507–522. Springer, 2020a.
  9. Adversarial reciprocal points learning for open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  10. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PMLR, 2020b.
  11. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’09), pages 248–255. IEEE, 2009.
  12. L. Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141–142, 2012.
  13. Reducing network agnostophobia. Advances in Neural Information Processing Systems, 31, 2018.
  14. Generative openmax for multi-class open set classification. In British Machine Vision Conference 2017 (BMVC’17), London, UK, 2017.
  15. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, 2020.
  16. Learning a neural-network-based representation for open set recognition. In Proceedings of the 2020 SIAM International Conference on Data Mining, pages 154–162. SIAM, 2020.
  17. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  18. Robust pre-training by adversarial contrastive learning. Advances in neural information processing systems, 33:16199–16210, 2020.
  19. Why is the mahalanobis distance effective for anomaly detection? arXiv preprint arXiv:2003.00402, 2020.
  20. Large-scale video classification with convolutional neural networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 1725–1732, 2014.
  21. Deep metric learning: A survey. Symmetry, 11(9):1066, 2019.
  22. Supervised contrastive learning for detecting anomalous driving behaviours from multimodal videos. In 2022 19th Conference on Robots and Vision (CRV), pages 16–23. IEEE, 2022.
  23. Supervised contrastive learning. Advances in neural information processing systems, 33:18661–18673, 2020.
  24. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  25. Similarity of neural network representations revisited. In International conference on machine learning, pages 3519–3529. PMLR, 2019.
  26. Temperature schedules for self-supervised contrastive methods on long-tail data. In The Eleventh International Conference on Learning Representations, 2022.
  27. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018.
  28. Applications of machine learning to machine fault diagnosis: A review and roadmap. Mechanical Systems and Signal Processing, 138:106587, 2020.
  29. Few-shot open-set recognition using meta-learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8798–8807, 2020.
  30. Experimental study of information measure and inter-intra class distance ratios on feature selection and orderings. IEEE Transactions on Systems, Man, and Cybernetics, (2):172–181, 1973.
  31. Class anchor clustering: A loss for distance-based open set recognition. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV’21), pages 3570–3578, 2021.
  32. Representation learning via invariant causal mechanisms. arXiv preprint arXiv:2010.07922, 2020.
  33. When does label smoothing help? Advances in neural information processing systems, 32, 2019.
  34. S. Narkhede. Understanding auc-roc curve. Towards Data Science, 26(1):220–227, 2018.
  35. Open set learning with counterfactual images. In Proceedings of the European Conference on Computer Vision (ECCV’18), pages 613–628, 2018.
  36. Street view hause number dataset. http://ufldl.stanford.edu/housenumbers, 2011.
  37. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.
  38. C2ae: Class conditioned auto-encoder for open-set recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2307–2316, 2019.
  39. Generative-discriminative feature representations for open-set recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11814–11823, 2020.
  40. Demystifying contrastive self-supervised learning: Invariances, augmentations and dataset biases. Advances in Neural Information Processing Systems, 33:3407–3418, 2020.
  41. Contrastive learning with hard negative samples. arXiv preprint arXiv:2010.04592, 2020.
  42. Can contrastive learning avoid shortcut solutions? Advances in neural information processing systems, 34:4974–4986, 2021.
  43. Toward open set recognition. IEEE transactions on pattern analysis and machine intelligence, 35(7):1757–1772, 2012.
  44. Toward causal representation learning. Proceedings of the IEEE, 109(5):612–634, 2021.
  45. Data augmentation as feature manipulation. In International conference on machine learning, pages 19773–19808. PMLR, 2022.
  46. Open-set recognition: a good closed-set classifier is all you need? In International Conference on Learning Representations (ICLR’22), 2022.
  47. Understanding the behaviour of contrastive loss. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2495–2504, 2021.
  48. Openauc: Towards auc-oriented open-set recognition. Advances in Neural Information Processing Systems, 35:25033–25045, 2022.
  49. Convolutional prototype network for open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(5):2358–2370, 2020.
  50. Classification-reconstruction learning for open-set recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4016–4025, 2019.
  51. Open category classification by adversarial sample generation. In International Joint Conference on Artificial Intelligence (IJCAI’17), pages 3357–3363, Melbourne, Australia, 2017.
  52. Modeling discriminative representations for out-of-domain detection with supervised contrastive learning. arXiv preprint arXiv:2105.14289, 2021.
  53. Hybrid models for open set recognition. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, pages 102–117. Springer, 2020.
  54. Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4):e1253, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets