Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

MISCA: A Joint Model for Multiple Intent Detection and Slot Filling with Intent-Slot Co-Attention (2312.05741v1)

Published 10 Dec 2023 in cs.CL

Abstract: The research study of detecting multiple intents and filling slots is becoming more popular because of its relevance to complicated real-world situations. Recent advanced approaches, which are joint models based on graphs, might still face two potential issues: (i) the uncertainty introduced by constructing graphs based on preliminary intents and slots, which may transfer intent-slot correlation information to incorrect label node destinations, and (ii) direct incorporation of multiple intent labels for each token w.r.t. token-level intent voting might potentially lead to incorrect slot predictions, thereby hurting the overall performance. To address these two issues, we propose a joint model named MISCA. Our MISCA introduces an intent-slot co-attention mechanism and an underlying layer of label attention mechanism. These mechanisms enable MISCA to effectively capture correlations between intents and slot labels, eliminating the need for graph construction. They also facilitate the transfer of correlation information in both directions: from intents to slots and from slots to intents, through multiple levels of label-specific representations, without relying on token-level intent information. Experimental results show that MISCA outperforms previous models, achieving new state-of-the-art overall accuracy performances on two benchmark datasets MixATIS and MixSNIPS. This highlights the effectiveness of our attention mechanisms.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. SLIM: Explicit Slot-Intent Mapping with BERT for Joint Multi-Intent Detection and Slot Filling. In Proceedings of ICASSP, pages 7607–7611.
  2. Joint multiple intent detection and slot filling via self-distillation. In Proceedings of ICASSP, pages 7612–7616.
  3. A Transformer-based Threshold-Free Framework for Multi-Intent NLU. In Proceedings of COLING, pages 7187–7192.
  4. BERT for Joint Intent Classification and Slot Filling. arXiv preprint, arXiv:1902.10909.
  5. A Scope Sensitive and Result Attentive Model for Multi-Intent Spoken Language Understanding. arXiv preprint, arXiv:2211.12220.
  6. Alice Coucke et al. 2018. Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces. arXiv preprint, arXiv:1805.10190.
  7. Intent Detection and Slot Filling for Vietnamese. In Proceedings of INTERSPEECH, pages 4698–4702.
  8. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of NAACL, pages 4171–4186.
  9. A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling. In Proceedings of ACL, pages 5467–5471.
  10. Rashmi Gangadharaiah and Balakrishnan Narayanaswamy. 2019. Joint Multiple Intent Detection and Slot Labeling for Goal-Oriented Dialog. In Proceedings of NAACL, pages 564–569.
  11. Slot-Gated Modeling for Joint Slot Filling and Intent Prediction. In Proceedings of NAACL, pages 753–757.
  12. The ATIS Spoken Language Systems Pilot Corpus. In Proceedings of the Third DARPA Speech and Natural Language Workshop, pages 96–101.
  13. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation, 9(8):1735–1780.
  14. Two-stage multi-intent detection for spoken language understanding. Multimedia Tools and Applications, 76:11377–11390.
  15. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proceedings of ICML, page 282–289.
  16. Neural Architectures for Named Entity Recognition. In Proceedings of NAACL, pages 260–270.
  17. A Self-Attentive Model with Gate Mechanism for Spoken Language Understanding. In Proceedings of EMNLP, pages 3824–3833.
  18. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint, arXiv:1907.11692.
  19. Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In Proceedings of ICLR.
  20. Samuel Louvan and Bernardo Magnini. 2020. Recent Neural Methods on Slot Filling and Intent Classification for Task-Oriented Dialogue Systems: A Survey. In Proceedings of COLING, pages 480–496.
  21. Hierarchical Question-Image Co-Attention for Visual Question Answering. In Proceedings of NIPS.
  22. A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding. In Proceedings of EMNLP-IJCNLP, pages 2078–2087.
  23. GL-GIN: Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling. In Proceedings of ACL-IJCNLP, pages 178–188.
  24. AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling. In Findings of the ACL: EMNLP, pages 1807–1816.
  25. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. The Journal of Machine Learning Research, 21(1):5485–5551.
  26. Enhancing Joint Multiple Intent Detection and Slot Filling with Global Intent-Slot Co-occurrence. In Proceedings of EMNLP, pages 7967–7977.
  27. Gokhan Tur and Renato De Mori. 2011. Spoken Language Understanding: Systems for Extracting Semantic Information from Speech. John Wiley & Sons, Inc.
  28. Attention is All you Need. In Proceedings of NIPS.
  29. Graph Attention Networks. In Proceedings of ICLR.
  30. A Label Attention Model for ICD Coding from Clinical Text. In Proceedings of IJCAI-20, pages 3335–3341.
  31. A Survey of Joint Intent Detection and Slot Filling Models in Natural Language Understanding. ACM Computing Surveys, 55(8):1–38.
  32. Incorporating Instructional Prompts into a Unified Generative Framework for Joint Multiple Intent Detection and Slot Filling. In Proceedings of COLING, pages 7203–7208.
  33. Label-Specific Document Representation for Multi-Label Text Classification. In Proceedings of EMNLP-IJCNLP, pages 466–475.
  34. Bowen Xing and Ivor Tsang. 2022a. Co-guiding Net: Achieving Mutual Guidances between Multiple Intent Detection and Slot Filling via Heterogeneous Semantics-Label Graphs. In Proceedings of EMNLP, pages 159–169.
  35. Bowen Xing and Ivor Tsang. 2022b. Group is better than individual: Exploiting Label Topologies and Label Relations for Joint Multiple Intent Detection and Slot Filling. In Proceedings of EMNLP, pages 3964–3975.
  36. Joint Slot Filling and Intent Detection via Capsule Neural Networks. In Proceedings of ACL, pages 5259–5267.
  37. A Joint Learning Framework With BERT for Spoken Language Understanding. Ieee Access, 7:168849–168858.
  38. A Dynamic Graph Interactive Framework with Label-Semantic Injection for Spoken Language Understanding. In Proceedings of ICASSP, pages 1–5.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Thinh Pham (5 papers)
  2. Chi Tran (6 papers)
  3. Dat Quoc Nguyen (55 papers)
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now