Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 62 tok/s
Gemini 2.5 Pro 51 tok/s Pro
GPT-5 Medium 36 tok/s Pro
GPT-5 High 30 tok/s Pro
GPT-4o 67 tok/s Pro
Kimi K2 192 tok/s Pro
GPT OSS 120B 430 tok/s Pro
Claude Sonnet 4.5 34 tok/s Pro
2000 character limit reached

TK-KNN: A Balanced Distance-Based Pseudo Labeling Approach for Semi-Supervised Intent Classification (2310.11607v1)

Published 17 Oct 2023 in cs.LG

Abstract: The ability to detect intent in dialogue systems has become increasingly important in modern technology. These systems often generate a large amount of unlabeled data, and manually labeling this data requires substantial human effort. Semi-supervised methods attempt to remedy this cost by using a model trained on a few labeled examples and then by assigning pseudo-labels to further a subset of unlabeled examples that has a model prediction confidence higher than a certain threshold. However, one particularly perilous consequence of these methods is the risk of picking an imbalanced set of examples across classes, which could lead to poor labels. In the present work, we describe Top-K K-Nearest Neighbor (TK-KNN), which uses a more robust pseudo-labeling approach based on distance in the embedding space while maintaining a balanced set of pseudo-labeled examples across classes through a ranking-based approach. Experiments on several datasets show that TK-KNN outperforms existing models, particularly when labeled data is scarce on popular datasets such as CLINC150 and Banking77. Code is available at https://github.com/ServiceNow/tk-knn

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. Pseudo-labeling and confirmation bias in deep semi-supervised learning. In IJCNN, 2020.
  2. M. Arjovsky and L. Bottou. Towards principled methods for training generative adversarial networks. arXiv preprint arXiv:1701.04862, 2017.
  3. Semi-supervised few-shot intent classification and slot filling. arXiv preprint arXiv:2109.08754, 2021.
  4. Efficient intent detection with dual sentence encoders. In Workshop on Natural Language Processing for Conversational AI, 2020.
  5. Curriculum labeling: Revisiting pseudo-labeling for semi-supervised learning. In AAAI, 2021.
  6. Mixtext: Linguistically-informed interpolation of hidden space for semi-supervised text classification. In ACL, 2020.
  7. An empirical survey of data augmentation for limited data learning in nlp. arXiv preprint arXiv:2106.07499, 2021.
  8. Contrastnet: A contrastive learning framework for few-shot text classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10492–10500, 2022.
  9. Learning to classify open intent via soft labeling and manifold mixup. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:635–645, 2022.
  10. Gan-bert: Generative adversarial learning for robust text classification with a bunch of labeled examples. In ACL, 2020.
  11. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  12. Feature affinity-based pseudo labeling for semi-supervised person re-identification. IEEE Transactions on Multimedia, 21(11):2891–2902, 2019.
  13. The hitchhiker’s guide to testing statistical significance in natural language processing. In ACL, 2018.
  14. Training vision transformers for image retrieval. arXiv preprint arXiv:2102.05644, 2021.
  15. Large margin deep networks for classification. Advances in neural information processing systems, 31, 2018.
  16. Zero-shot text classification with self-training. arXiv preprint arXiv:2210.17541, 2022.
  17. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
  18. Supervised contrastive learning. Advances in neural information processing systems, 33:18661–18673, 2020.
  19. Sample estimate of the entropy of a random vector. Problemy Peredachi Informatsii, 23(2):9–16, 1987.
  20. Ssr: Semi-supervised soft rasterizer for single-view 2d to 3d reconstruction. arXiv preprint arXiv:2108.09593, 2021.
  21. An evaluation dataset for intent classification and out-of-scope prediction. In EMNLP-IJCNLP, 2019.
  22. D.-H. Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, 2013.
  23. Benchmarking natural language understanding services for building conversational agents. arXiv preprint arXiv:1903.05566, 2019.
  24. A multi-level approach for hierarchical ticket classification. In W-NUT, 2022.
  25. K. Nigam and R. Ghani. Analyzing the effectiveness and applicability of co-training. In Proceedings of the ninth international conference on Information and knowledge management, pages 86–93, 2000.
  26. An overview of deep semi-supervised learning. arXiv preprint arXiv:2006.05278, 2020a.
  27. Semi-supervised semantic segmentation with cross-consistency training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020b.
  28. L. Prechelt. Early stopping-but when? In Neural Networks: Tricks of the trade, pages 55–69. Springer, 1998.
  29. N. Reimers and I. Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, 2019.
  30. In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning. arXiv preprint arXiv:2101.06329, 2021.
  31. Spreading vectors for similarity search. In ICLR 2019-7th International Conference on Learning Representations, pages 1–13, 2019.
  32. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
  33. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In NeurIPS, 2020.
  34. Self-supervised wasserstein pseudo-labeling for semi-supervised image classification. In CVPR, 2021.
  35. Contrastive learning-enhanced nearest neighbor mechanism for multi-label text classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 672–679, 2022.
  36. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019.
  37. Billion-scale semi-supervised learning for image classification. arXiv preprint arXiv:1905.00546, 2019.
  38. S4l: Self-supervised semi-supervised learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019.
  39. Out-of-scope intent detection with self-supervision and discriminative training. arXiv preprint arXiv:2106.08616, 2021.
  40. Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. In NeurIPS, 2021a.
  41. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412, 2017.
  42. Deep open intent classification with adaptive decision boundary. In AAAI, 2021b.
  43. Knn-contrastive learning for out-of-domain intent classification. In ACL, 2022.
  44. X. Zhu and A. B. Goldberg. Introduction to semi-supervised learning. Synthesis lectures on artificial intelligence and machine learning, 3(1):1–130, 2009.
  45. Detecting corrupted labels without training a model to predict. In ICML, 2022.
  46. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In Proceedings of the European conference on computer vision (ECCV), pages 289–305, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up