Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PersonalityChat: Conversation Distillation for Personalized Dialog Modeling with Facts and Traits (2401.07363v1)

Published 14 Jan 2024 in cs.CL

Abstract: The new wave of LLMs (LLM) has offered an efficient tool to curate sizeable conversational datasets. So far studies have mainly focused on task-oriented or generic open-domain dialogs, and have not fully explored the ability of LLMs in following complicated prompts. In this work, we focus on personalization, and employ LLMs to curate a dataset which is difficult and costly to crowd-source: PersonalityChat is a synthetic conversational dataset based upon the popular PersonaChat dataset, but conditioned on both personas and (Big-5) personality traits. Evaluating models fine-tuned on this dataset, we show that the personality trait labels can be used for trait-based personalization of generative dialogue models. We also perform a head-to-head comparison between PersonalityChat and PersonaChat, and show that training on the distilled dataset results in more fluent and coherent dialog agents in the small-model regime.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc.
  2. Zero-shot transfer learning with synthesized data for multi-domain dialogue state tracking. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 122–132, Online. Association for Computational Linguistics.
  3. Places: Prompting language models for social conversation synthesis. In EACL 2023.
  4. Weakly supervised data augmentation through prompting for dialogue understanding. In NeurIPS 2022 Workshop on Synthetic Data for Empowering ML Research.
  5. Deborah A. Cobb-Clark and Stefanie Schurer. 2012. The stability of big-five personality traits. Economics Letters, 115(1):11–15.
  6. Augmenting neural response generation with context-aware topical attention. In Proceedings of the First Workshop on NLP for Conversational AI, pages 18–31, Florence, Italy. Association for Computational Linguistics.
  7. F Galton. 1884. Measurement of character. fortnightly review, 36, 179-185. Galton17936Fortnightly Review1884.
  8. Perceptions of alignment and personality in generated dialogue. In INLG 2012 Proceedings of the Seventh International Natural Language Generation Conference, pages 40–48, Utica, IL. Association for Computational Linguistics.
  9. PANDORA talks: Personality and demographics on Reddit. In Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media, pages 138–152, Online. Association for Computational Linguistics.
  10. Lewis R. Goldberg. 1990. An alternative "description of personality": the big-five factor structure. Journal of personality and social psychology, 59 6:1216–29.
  11. Ctrl: A conditional transformer language model for controllable generation. ArXiv, abs/1909.05858.
  12. SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization. arXiv e-prints, page arXiv:2212.10465.
  13. PERSONACHATGEN: Generating personalized dialogues using GPT-3. In Proceedings of the 1st Workshop on Customized Chat Grounding Persona and Knowledge, pages 29–48, Gyeongju, Republic of Korea. Association for Computational Linguistics.
  14. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online. Association for Computational Linguistics.
  15. A persona-based neural conversation model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 994–1003, Berlin, Germany. Association for Computational Linguistics.
  16. Pierre Lison and Jörg Tiedemann. 2016. OpenSubtitles2016: Extracting large parallel corpora from movie and TV subtitles. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 923–929, Portorož, Slovenia. European Language Resources Association (ELRA).
  17. The Ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 285–294, Prague, Czech Republic. Association for Computational Linguistics.
  18. Mathdial: A dialogue tutoring dataset with rich pedagogical properties grounded in math reasoning problems.
  19. François Mairesse and Marilyn Walker. 2007. PERSONAGE: Personality generation for dialogue. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 496–503, Prague, Czech Republic. Association for Computational Linguistics.
  20. François Mairesse and Marilyn Walker. 2008a. A personality-based framework for utterance generation in dialogue applications. pages 80–87.
  21. François Mairesse and Marilyn A. Walker. 2008b. Can conversational agents express big five personality traits through language ? : Evaluating a psychologically-informed language generator.
  22. Philip M. McCarthy and Scott Jarvis. 2010. Mtld, vocd-d, and hd-d: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42:381–392.
  23. Robert R. McCrae and Oliver P. John. 1992. An introduction to the five-factor model and its applications. Journal of Personality, 60(2):175–215.
  24. LAD: Language models as data for zero-shot dialog. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 595–604, Edinburgh, UK. Association for Computational Linguistics.
  25. Warren T. Norman. 1963. Toward an adequate taxonomy of personality attributes: replicated factors structure in peer nomination personality ratings. Journal of abnormal and social psychology, 66:574–83.
  26. Training language models to follow instructions with human feedback.
  27. S. V. Paunonen and Douglas N. Jackson. 2000. What is beyond the big five? plenty! Journal of personality, 68 5:821–35.
  28. James W. Pennebaker and Lynda A. King. 1999. Linguistic styles: language use as an individual difference. Journal of personality and social psychology, 77 6:1296–312.
  29. Towards empathetic open-domain conversation models: A new benchmark and dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5370–5381, Florence, Italy. Association for Computational Linguistics.
  30. William Revelle. 2003. Personality processes. Annual Review of Psychology, 46:295–328.
  31. LINGUIST: Language model instruction tuning to generate annotated utterances for intent classification and slot tagging. In Proceedings of the 29th International Conference on Computational Linguistics, pages 218–241, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  32. Stylistic response generation by controlling personality traits and intent. In Proceedings of the 4th Workshop on NLP for Conversational AI, pages 197–211, Dublin, Ireland. Association for Computational Linguistics.
  33. Data augmentation for intent classification with off-the-shelf large language models. In Proceedings of the 4th Workshop on NLP for Conversational AI, pages 47–57, Dublin, Ireland. Association for Computational Linguistics.
  34. Agenda-based user simulation for bootstrapping a POMDP dialogue system. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers, pages 149–152, Rochester, New York. Association for Computational Linguistics.
  35. Bootstrapping a neural conversational agent with dialogue self-play, crowdsourcing and on-line reinforcement learning. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers), pages 41–51, New Orleans - Louisiana. Association for Computational Linguistics.
  36. Human evaluation of conversations is an open problem: comparing the sensitivity of various methods for evaluating dialogue agents. In Proceedings of the 4th Workshop on NLP for Conversational AI, pages 77–97, Dublin, Ireland. Association for Computational Linguistics.
  37. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
  38. Learning new skills after deployment: Improving open-domain internet-driven dialogue with human feedback.
  39. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213, Melbourne, Australia. Association for Computational Linguistics.
  40. Opt: Open pre-trained transformer language models.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Ehsan Lotfi (9 papers)
  2. Maxime De Bruyn (5 papers)
  3. Jeska Buhmann (5 papers)
  4. Walter Daelemans (31 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now