Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

TransferTOD: A Generalizable Chinese Multi-Domain Task-Oriented Dialogue System with Transfer Capabilities (2407.21693v3)

Published 31 Jul 2024 in cs.AI

Abstract: Task-oriented dialogue (TOD) systems aim to efficiently handle task-oriented conversations, including information collection. How to utilize TOD accurately, efficiently and effectively for information collection has always been a critical and challenging task. Recent studies have demonstrated that LLMs excel in dialogue, instruction generation, and reasoning, and can significantly enhance the performance of TOD through fine-tuning. However, current datasets primarily cater to user-led systems and are limited to predefined specific scenarios and slots, thereby necessitating improvements in the proactiveness, diversity, and capabilities of TOD. In this study, we present a detailed multi-domain task-oriented data construction process for conversations, and a Chinese dialogue dataset generated based on this process, TransferTOD, which authentically simulates human-computer dialogues in 30 popular life service scenarios. Leveraging this dataset, we trained a model called TransferTOD-7B using full-parameter fine-tuning, showcasing notable abilities in slot filling and questioning. Our work has demonstrated its strong generalization capabilities in various downstream scenarios, significantly enhancing both data utilization efficiency and system performance. The data is released in https://github.com/KongLongGeFDU/TransferTOD.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. Qwen technical report. arXiv preprint arXiv:2309.16609.
  2. Baichuan. 2023. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305.
  3. Language Models are Few-Shot Learners. ArXiv:2005.14165 [cs].
  4. MultiWOZ – A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling. ArXiv:1810.00278 [cs].
  5. A survey on evaluation of large language models. ACM Trans. Intell. Syst. Technol., 15(3).
  6. Evaluating Large Language Models Trained on Code. ArXiv:2107.03374 [cs].
  7. PaLM: Scaling Language Modeling with Pathways. ArXiv:2204.02311 [cs].
  8. Pre-training with whole word masking for chinese bert.
  9. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  10. All NLP tasks are generation tasks: A general pretraining framework. CoRR, abs/2103.10360.
  11. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 320–335.
  12. Improved and Efficient Conversational Slot Labeling through Question Answering. ArXiv:2204.02123 [cs].
  13. Multi-Domain Joint Semantic Frame Parsing using Bi-directional RNN-LSTM.
  14. The ATIS Spoken Language Systems Pilot Corpus. In Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27,1990.
  15. ConveRT: Efficient and Accurate Conversational Representations from Transformers. ArXiv:1911.03688 [cs].
  16. The Second Dialog State Tracking Challenge. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 263–272, Philadelphia, PA, U.S.A. Association for Computational Linguistics.
  17. Lora: Low-rank adaptation of large language models.
  18. Exploring the impact of instruction data scaling on large language models: An empirical study on real-world use cases. arXiv preprint arXiv:2303.14742.
  19. Unified named entity recognition as word-word relation classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10965–10973.
  20. End-to-End Task-Completion Neural Dialogue Systems. ArXiv:1703.01008 [cs].
  21. MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems. ArXiv:2009.12005 [cs].
  22. Lexicon enhanced Chinese sequence labeling using BERT adapter. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 5847–5858, Online. Association for Computational Linguistics.
  23. RoBERTa: A Robustly Optimized BERT Pretraining Approach. ArXiv:1907.11692 [cs].
  24. GPT-4 Technical Report. ArXiv:2303.08774 [cs].
  25. Soloist: Building Task Bots at Scale with Transfer Learning and Machine Teaching. Transactions of the Association for Computational Linguistics, 9:807–824.
  26. Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2231–2240, Copenhagen, Denmark. Association for Computational Linguistics.
  27. Spoken Question Answering. In Spoken Language Understanding, pages 147–170. John Wiley & Sons, Ltd.
  28. Simplify the usage of lexicon in chinese ner. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5951–5960.
  29. A comparative study between full-parameter and lora-based fine-tuning on chinese instruction data for instruction following large language model. ArXiv, abs/2304.08109.
  30. BlueLM Team. 2023. Bluelm: An open multilingual 7b language model. https://github.com/vivo-ai-lab/BlueLM.
  31. Q-tod: A query-driven task-oriented dialogue system.
  32. Llama 2: Open foundation and fine-tuned chat models.
  33. KddRES: A Multi-level Knowledge-driven Dialogue Dataset for Restaurant Towards Customized Dialogue System. ArXiv:2011.08772 [cs].
  34. A Survey of the Evolution of Language Model-Based Dialogue Systems. ArXiv:2311.16789 [cs].
  35. A Network-based End-to-End Trainable Task-oriented Dialogue System. ArXiv:1604.04562 [cs, stat].
  36. TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue. ArXiv:2004.06871 [cs].
  37. Agenttuning: Enabling generalized agent abilities for llms. ArXiv, abs/2310.12823.
  38. Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414.
  39. SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Prompting. ArXiv:2305.09067 [cs].
  40. Llmeval: A preliminary study on how to evaluate large language models. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pages 19615–19622. AAAI Press.
  41. Llmeval: A preliminary study on how to evaluate large language models. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI.
  42. A Dataset for Document Grounded Conversations. ArXiv:1809.07358 [cs].
  43. CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Dataset. ArXiv:2002.11893 [cs].
  44. A Chinese Multi-type Complex Questions Answering Dataset over Wikidata. ArXiv:2111.06086 [cs].
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (13)
  1. Ming Zhang (313 papers)
  2. Caishuang Huang (13 papers)
  3. Yilong Wu (11 papers)
  4. Shichun Liu (8 papers)
  5. Huiyuan Zheng (10 papers)
  6. Yurui Dong (4 papers)
  7. Yujiong Shen (6 papers)
  8. Shihan Dou (46 papers)
  9. Jun Zhao (469 papers)
  10. Junjie Ye (66 papers)
  11. Qi Zhang (784 papers)
  12. Tao Gui (127 papers)
  13. Xuanjing Huang (287 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets