Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Decoupling Representation and Knowledge for Few-Shot Intent Classification and Slot Filling (2312.13495v1)

Published 21 Dec 2023 in cs.CL and cs.AI

Abstract: Few-shot intent classification and slot filling are important but challenging tasks due to the scarcity of finely labeled data. Therefore, current works first train a model on source domains with sufficiently labeled data, and then transfer the model to target domains where only rarely labeled data is available. However, experience transferring as a whole usually suffers from gaps that exist among source domains and target domains. For instance, transferring domain-specific-knowledge-related experience is difficult. To tackle this problem, we propose a new method that explicitly decouples the transferring of general-semantic-representation-related experience and the domain-specific-knowledge-related experience. Specifically, for domain-specific-knowledge-related experience, we design two modules to capture intent-slot relation and slot-slot relation respectively. Extensive experiments on Snips and FewJoint datasets show that our method achieves state-of-the-art performance. The method improves the joint accuracy metric from 27.72% to 42.20% in the 1-shot setting, and from 46.54% to 60.79% in the 5-shot setting.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Semi-Supervised Few-Shot Intent Classification and Slot Filling. arXiv preprint arXiv:2109.08754.
  2. Bert for joint intent classification and slot filling. arXiv preprint arXiv:1902.10909.
  3. Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv preprint arXiv:1805.10190.
  4. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171–4186.
  5. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, 1126–1135. PMLR.
  6. Zero-Shot Learning for Joint Intent and Slot Labeling. arXiv preprint arXiv:2212.07922.
  7. Making Pre-trained Language Models Better Few-shot Learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 3816–3830.
  8. Slot-Gated Modeling for Joint Slot Filling and Intent Prediction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), 753–757.
  9. Can ChatGPT Detect Intent? Evaluating Large Language Models for Spoken Language Understanding. arXiv preprint arXiv:2305.13512.
  10. Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 1381–1393.
  11. Learning to Bridge Metric Spaces: Few-shot Joint Learning of Intent Detection and Slot Filling. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 3190–3200.
  12. Fewjoint: A few-shot learning benchmark for joint language understanding. arXiv preprint arXiv:2009.08138.
  13. Universal Language Model Fine-tuning for Text Classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 328–339.
  14. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
  15. Learning to Classify Intents and Slot Labels Given a Handful of Examples. In Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, 96–108.
  16. Zero-Shot Slot and Intent Detection in Low-Resource Languages. arXiv preprint arXiv:2304.13292.
  17. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proceedings of the Eighteenth International Conference on Machine Learning, 282–289.
  18. Prefix-Tuning: Optimizing Continuous Prompts for Generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 4582–4597.
  19. An Explicit-Joint and Supervised-Contrastive Learning Framework for Few-Shot Intent Classification and Slot Filling. In Findings of the Association for Computational Linguistics: EMNLP 2021, 1945–1955.
  20. Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 19–25.
  21. Frustratingly Simple Few-Shot Slot Tagging. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 1028–1033.
  22. UniTranSeR: A Unified Transformer Semantic Representation Framework for Multimodal Task-Oriented Dialog System. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 103–114.
  23. Intention Reasoning Network for Multi-Domain End-to-end Task-Oriented Dialogue. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2273–2285.
  24. ZEROTOP: Zero-Shot Task-Oriented Semantic Parsing using Large Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 5792–5799.
  25. Is ChatGPT a General-Purpose Natural Language Processing Task Solver? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 1339–1384.
  26. A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2078–2087.
  27. A Co-Interactive Transformer for Joint Slot Filling and Intent Detection. In ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 8193–8197.
  28. A Survey on Spoken Language Understanding: Recent Advances and New Frontiers. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, 4577–4584.
  29. Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 338–348.
  30. Low-Resource Compositional Semantic Parsing with Concept Pretraining. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 1410–1419.
  31. LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging. In Proceedings of the 29th International Conference on Computational Linguistics, 218–241.
  32. It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2339–2352.
  33. Partial is better than all: revisiting fine-tuning strategy for few-shot learning. In Proceedings of the AAAI Conference on Artificial Intelligence, 9594–9602.
  34. Prototypical Networks for Few-shot Learning. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4077–4087.
  35. Meta-transfer learning for few-shot learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 403–412.
  36. A Bi-Model Based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), 309–314.
  37. A survey of joint intent detection and slot filling models in natural language understanding. ACM Computing Surveys, 55(8): 1–38.
  38. DCEN: A Decoupled Context Enhanced Network For Few-shot Slot Tagging. In 2021 International Joint Conference on Neural Networks (IJCNN), 1–7.
  39. Hit-scir at mmnlu-22: Consistency regularization for multilingual spoken language understanding. arXiv preprint arXiv:2301.02010.
  40. Vector projection network for few-shot slot tagging in natural language understanding. arXiv preprint arXiv:2009.09568.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now