Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detection (2310.13380v1)

Published 20 Oct 2023 in cs.CL

Abstract: Detecting out-of-domain (OOD) intents from user queries is essential for a task-oriented dialogue system. Previous OOD detection studies generally work on the assumption that plenty of labeled IND intents exist. In this paper, we focus on a more practical few-shot OOD setting where there are only a few labeled IND data and massive unlabeled mixed data that may belong to IND or OOD. The new scenario carries two key challenges: learning discriminative representations using limited IND data and leveraging unlabeled mixed data. Therefore, we propose an adaptive prototypical pseudo-labeling (APP) method for few-shot OOD detection, including a prototypical OOD detection framework (ProtoOOD) to facilitate low-resource OOD detection using limited IND data, and an adaptive pseudo-labeling method to produce high-quality pseudo OOD&IND labels. Extensive experiments and analysis demonstrate the effectiveness of our method for few-shot OOD detection.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. Satoshi Akasaki and Nobuhiro Kaji. 2017. Chat detection in an intelligent assistant: Combining task-oriented and non-task-oriented spoken dialogue systems. ArXiv, abs/1705.00746.
  2. Abhijit Bendale and Terrance E. Boult. 2016. Towards open set deep networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1563–1572.
  3. Efficient intent detection with dual sentence encoders. arXiv preprint arXiv:2003.04807.
  4. Curriculum labeling: Revisiting pseudo-labeling for semi-supervised learning. In AAAI Conference on Artificial Intelligence.
  5. Prototypical verbalizer for prompt-based few-shot tuning. In Annual Meeting of the Association for Computational Linguistics.
  6. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT.
  7. Geli Fei and Bing Liu. 2016. Breaking the closed world assumption in text classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 506–514.
  8. Dan Hendrycks and Kevin Gimpel. 2016. A baseline for detecting misclassified and out-of-distribution examples in neural networks. ArXiv, abs/1610.02136.
  9. Dan Hendrycks and Kevin Gimpel. 2017. A baseline for detecting misclassified and out-of-distribution examples in neural networks. ArXiv, abs/1610.02136.
  10. Joo-Kyung Kim and Young-Bum Kim. 2018. Joint learning of domain classification and out-of-domain detection with dynamic class weighting for satisficing false acceptance rates. ArXiv, abs/1807.00072.
  11. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
  12. An evaluation dataset for intent classification and out-of-scope prediction. In EMNLP/IJCNLP.
  13. Dong-Hyun Lee. 2013. Pseudo-label : The simple and efficient semi-supervised learning method for deep neural networks.
  14. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. ArXiv, abs/1807.03888.
  15. Simclr-based data-driven tight frame for seismic denoising and interpolation. GEOPHYSICS.
  16. Prototypical contrastive learning of unsupervised representations. ArXiv, abs/2005.04966.
  17. Principled detection of out-of-distribution examples in neural networks. ArXiv, abs/1706.02690.
  18. Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv: Learning.
  19. Ting-En Lin and Hua Xu. 2019. Deep unknown intent detection with margin loss. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5491–5496.
  20. Prasanta Chandra Mahalanobis. 1936. On the generalized distance in statistics.
  21. Uninl: Aligning representation learning with scoring function for ood detection via unified neighborhood learning. ArXiv, abs/2210.10722.
  22. Likelihood ratios for out-of-distribution detection. ArXiv, abs/1906.02845.
  23. In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning. ArXiv, abs/2101.06329.
  24. Doc: Deep open classification of text documents. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2911–2916.
  25. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. ArXiv, abs/2001.07685.
  26. Amrita S Tulshan and Sudhir Namdeorao Dhage. 2018. Survey on virtual assistant: Google assistant, siri, cortana, alexa. In International symposium on signal processing and intelligent recognition systems, pages 190–201.
  27. Interpolation consistency training for semi-supervised learning. Neural networks : the official journal of the International Neural Network Society, 145:90–106.
  28. Revisit overconfidence for ood detection: Reassigned contrastive learning with adaptive class-dependent threshold. In NAACL.
  29. Distribution calibration for out-of-domain detection with bayesian approximation. In International Conference on Computational Linguistics.
  30. Disentangling confidence score distribution for out-of-domain intent detection with energy-based learning. ArXiv, abs/2210.08830.
  31. 3d semi-supervised learning with uncertainty-aware multi-view co-training. 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 3635–3644.
  32. A deep generative distance-based classifier for out-of-domain detection with mahalanobis space. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1452–1460, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  33. Short text clustering via convolutional neural networks. In Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, pages 62–69.
  34. Uncertainty-aware self-ensembling model for semi-supervised 3d left atrium segmentation. ArXiv, abs/1907.07034.
  35. Modeling discriminative representations for out-of-domain detection with supervised contrastive learning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 870–878, Online. Association for Computational Linguistics.
  36. Adversarial self-supervised learning for out-of-domain detection. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5631–5639, Online. Association for Computational Linguistics.
  37. Adversarial self-supervised learning for out-of-domain detection. In North American Chapter of the Association for Computational Linguistics.
  38. A closer look at few-shot out-of-distribution intent detection. In Proceedings of the 29th International Conference on Computational Linguistics, pages 451–460.
  39. Out-of-domain detection for natural language understanding in dialog systems. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28:1198–1209.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (9)
  1. Pei Wang (240 papers)
  2. Keqing He (47 papers)
  3. Yutao Mou (16 papers)
  4. Xiaoshuai Song (16 papers)
  5. Yanan Wu (40 papers)
  6. Jingang Wang (71 papers)
  7. Yunsen Xian (17 papers)
  8. Xunliang Cai (63 papers)
  9. Weiran Xu (58 papers)
Citations (1)
View on Semantic Scholar