2000 character limit reached
Contextual Data Augmentation for Task-Oriented Dialog Systems (2310.10380v1)
Published 16 Oct 2023 in cs.CL
Abstract: Collection of annotated dialogs for training task-oriented dialog systems have been one of the key bottlenecks in improving current models. While dialog response generation has been widely studied on the agent side, it is not evident if similar generative models can be used to generate a large variety of, and often unexpected, user inputs that real dialog systems encounter in practice. Existing data augmentation techniques such as paraphrase generation do not take the dialog context into consideration. In this paper, we develop a novel dialog augmentation model that generates a user turn, conditioning on full dialog context. Additionally, with a new prompt design for LLM, and output re-ranking, the dialogs generated from our model can be directly used to train downstream dialog systems. On common benchmark datasets MultiWoZ and SGD, we show that our dialog augmentation model generates high quality dialogs and improves dialog success rate by as much as $8\%$ over baseline.
- D. Hakkani-Tür, G. Tür, A. Celikyilmaz et al., “Multi-domain joint semantic frame parsing using bi-directional RNN-LSTM,” in Interspeech, 2016.
- C.-W. Goo, G. Gao, Y.-K. Hsu et al., “Slot-gated modeling for joint slot filling and intent prediction,” in NAACL-HLT, 2018.
- N. Mrksic, D. Ó. Séaghdha, T. Wen et al., “Neural belief tracker: Data-driven dialogue state tracking,” in ACL 2017, 2017.
- L. Chen, B. Lv, C. Wang et al., “Schema-guided multi-domain dialogue state tracking with graph attention neural networks,” in AAAI, 2020.
- T. Wen, Y. Miao, P. Blunsom et al., “Latent intention dialogue models,” in ICML, 2017.
- B. Peng, X. Li, J. Gao, J. Liu, and K. Wong, “Deep dyna-q: Integrating planning for task-completion dialogue policy learning,” in ACL, 2018.
- T. Wen, M. Gasic, N. Mrksic, P. Su, D. Vandyke, and S. J. Young, “Semantically conditioned lstm-based natural language generation for spoken dialogue systems,” in EMNLP, 2015.
- Y. Zhang, S. Sun, M. Galley et al., “DIALOGPT : Large-scale generative pre-training for conversational response generation,” in ACL, 2020.
- E. Hosseini-Asl, B. McCann, C. Wu et al., “A simple language model for task-oriented dialogue,” in NeurIPS, 2020.
- B. Peng, C. Li, J. Li et al., “SOLOIST: building task bots at scale with transfer learning and machine teaching,” TACL, vol. 9, 2021.
- Y. Yang, Y. Li, and X. Quan, “UBAR: towards fully end-to-end task-oriented dialog system with GPT-2,” in AAAI, 2021.
- P. Shah, D. Hakkani-Tür, G. Tür et al., “Building a conversational agent overnight with dialogue self-play,” arXiv arXiv:1801.04871, 2018.
- C.-W. Lin, V. Auvray, D. Elkind et al., “Dialog simulation with realistic variations for training goal-oriented conversational systems,” arXiv preprint arXiv:2011.08243, 2020.
- A. Radford, J. Wu, R. Child et al., “Language models are unsupervised multitask learners,” OpenAI blog, 2019.
- C. Raffel, N. Shazeer, A. Roberts et al., “Exploring the limits of transfer learning with a unified text-to-text transformer,” JMLR, vol. 21, 2020.
- J. Gu, Q. Wu, C. Wu, W. Shi, and Z. Yu, “PRAL: A tailored pre-training model for task-oriented dialog generation,” in ACL/IJCNLP, 2021.
- M. Kale and A. Rastogi, “Template guided text generation for task-oriented dialogue,” in EMNLP, 2020.
- ——, “Text-to-text pre-training for data-to-text tasks,” in INLG, 2020.
- X. Xu, G. Wang, Y. Kim, and S. Lee, “Augnlg: Few-shot natural language generation using self-trained data augmentation,” in ACL/IJCNLP, 2021.
- Y. Hou, Y. Liu, W. Che, and T. Liu, “Sequence-to-sequence data augmentation for dialogue language understanding,” in Proc. of COLING, 2018.
- Z. Zhao, S. Zhu, and K. Yu, “Data augmentation with atomic templates for spoken language understanding,” in Proc. of EMNLP-IJCNLP, 2019.
- H. Lin, L. Xiang, Y. Zhou, J. Zhang, and C. Zong, “Augmenting slot values and contexts for spoken language understanding with pretrained models,” in Interspeech, 2021.
- M. Lewis, Y. Liu, N. Goyal et al., “BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” in ACL, 2020.
- M. Eric, R. Goel et al., “Multiwoz 2.1: A consolidated multi-domain dialogue dataset with state corrections and state tracking baselines,” in LREC, 2020.
- A. Rastogi, X. Zang, S. Sunkara et al., “Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset,” in AAAI, 2020.
- T. Sellam, D. Das, and A. P. Parikh, “Bleurt: Learning robust metrics for text generation,” arXiv preprint arXiv:2004.04696, 2020.
- P. Budzianowski, T. Wen et al., “Multiwoz - A large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling,” in EMNLP, 2018.
- K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in ACL, 2022.
- T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “Bertscore: Evaluating text generation with BERT,” in ICLR, 2020.
- N. Lubis, C. Geishauser, M. Heck, H. Lin, M. Moresi, C. van Niekerk, and M. Gasic, “LAVA: latent action spaces via variational auto-encoding for dialogue policy optimization,” in COLING, 2020, pp. 465–479.
- S. Gao, Y. Zhang, Z. Ou, and Z. Yu, “Paraphrase augmented task-oriented dialog generation,” in ACL, 2020.
- Baolin Peng and Chunyuan Li and Jinchao Li and Shahin Shayandeh and Lars Liden and Jianfeng Gao, “Soloist,” https://github.com/pengbaolin/soloisthttps://github.com/pengbaolin/soloist, 2021.
- Dustin Axman (1 paper)
- Avik Ray (11 papers)
- Shubham Garg (8 papers)
- Jing Huang (140 papers)