Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

MORPHEUS: Modeling Role from Personalized Dialogue History by Exploring and Utilizing Latent Space (2407.02345v1)

Published 2 Jul 2024 in cs.CL

Abstract: Personalized Dialogue Generation (PDG) aims to create coherent responses according to roles or personas. Traditional PDG relies on external role data, which can be scarce and raise privacy concerns. Approaches address these issues by extracting role information from dialogue history, which often fail to generically model roles in continuous space. To overcome these limitations, we introduce a novel framework \textbf{MO}dels \textbf{R}oles from \textbf{P}ersonalized Dialogue \textbf{H}istory by \textbf{E}xploring and \textbf{U}tilizing Latent \textbf{S}pace (MORPHEUS) through a three-stage training process. Specifically, we create a persona codebook to represent roles in latent space compactly, and this codebook is used to construct a posterior distribution of role information. This method enables the model to generalize across roles, allowing the generation of personalized dialogues even for unseen roles. Experiments on both Chinese and English datasets demonstrate that MORPHEUS enhances the extraction of role information, and improves response generation without external role data. Additionally, MORPHEUS can be considered an efficient fine-tuning for LLMs.

PDF Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. Modeling personalization in continuous space for response generation via augmented wasserstein autoencoders. In Conference on Empirical Methods in Natural Language Processing.
  2. From persona to personalization: A survey on role-playing language agents.
  3. Recent trends in personalized dialogue generation: A review of datasets, methodologies, and evaluations. In International Conference on Language Resources and Evaluation.
  4. Maximum likelihood from incomplete data via the em - algorithm plus discussions on the paper.
  5. The second conversational intelligence challenge (convai2). arXiv preprint arXiv:1902.00098.
  6. Simcse: Simple contrastive learning of sentence embeddings. In Conference on Empirical Methods in Natural Language Processing.
  7. Personapkt: Building personalized dialogue agents via parameter-efficient knowledge transfer. In SUSTAINLP.
  8. The curious case of neural text degeneration. In International Conference on Learning Representations.
  9. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. International Conference on Learning Representations.
  10. J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics, pages 159–174.
  11. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110–119, San Diego, California. Association for Computational Linguistics.
  12. A persona-based neural conversation model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 994–1003, Berlin, Germany. Association for Computational Linguistics.
  13. Chin-Yew Lin and Franz Josef Och. 2004. Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. In Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, ACL ’04, page 605–es, USA. Association for Computational Linguistics.
  14. Composable text controls in latent space with odes. In Conference on Empirical Methods in Natural Language Processing.
  15. You impress me: Dialogue generation via mutual persona perception. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1417–1427, Online. Association for Computational Linguistics.
  16. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations.
  17. Miracle: Towards personalized dialogue generation with latent-space multiple personal attribute control. In Conference on Empirical Methods in Natural Language Processing.
  18. Effective approaches to attention-based neural machine translation. ArXiv, abs/1508.04025.
  19. One chatbot per person: Creating personalized chatbots based on implicit user profiles. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21, page 555–564, New York, NY, USA. Association for Computing Machinery.
  20. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
  21. Assigning personality/profile to a chatting machine for coherent conversation generation. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pages 4279–4285. International Joint Conferences on Artificial Intelligence Organization.
  22. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  23. BoB: BERT over BERT for training persona-based dialogue models from limited personalized data. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 167–177, Online. Association for Computational Linguistics.
  24. Exploiting persona information for diverse generation of conversational responses. In IJCAI.
  25. Exploiting persona information for diverse generation of conversational responses. In International Joint Conference on Artificial Intelligence.
  26. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc.
  27. Enhancing personalized dialogue generation with contrastive latent variables: Combining sparse and dense persona. In Annual Meeting of the Association for Computational Linguistics.
  28. Llama: Open and efficient foundation language models. ArXiv, abs/2302.13971.
  29. Neural discrete representation learning. ArXiv, abs/1711.00937.
  30. Transfertransfo: A transfer learning approach for neural network based conversational agents.
  31. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213, Melbourne, Australia. Association for Computational Linguistics.
  32. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In Annual Meeting of the Association for Computational Linguistics.
  33. A pre-training based personalized dialogue generation model with persona-sparse data. Proceedings of the AAAI Conference on Artificial Intelligence, 34:9693–9700.
  34. Less is more: Learning to refine dialogue history for personalized dialogue generation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5808–5820, Seattle, United States. Association for Computational Linguistics.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Yihong Tang (24 papers)
  2. Bo Wang (823 papers)
  3. Dongming Zhao (15 papers)
  4. Xiaojia Jin (2 papers)
  5. Jijun Zhang (2 papers)
  6. Ruifang He (7 papers)
  7. Yuexian Hou (23 papers)