Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generation (2306.07206v1)

Published 12 Jun 2023 in cs.CL and cs.AI

Abstract: Endowing chatbots with a consistent persona is essential to an engaging conversation, yet it remains an unresolved challenge. In this work, we propose a new retrieval-enhanced approach for personalized response generation. Specifically, we design a hierarchical transformer retriever trained on dialogue domain data to perform personalized retrieval and a context-aware prefix encoder that fuses the retrieved information to the decoder more effectively. Extensive experiments on a real-world dataset demonstrate the effectiveness of our model at generating more fluent and personalized responses. We quantitatively evaluate our model's performance under a suite of human and automatic metrics and find it to be superior compared to state-of-the-art baselines on English Reddit conversations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Weighted Krippendorff’s alpha is a more reliable metrics for multi-coders ordinal annotations: experimental studies on emotion, opinion and coreference annotation. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 550–559, Gothenburg, Sweden. Association for Computational Linguistics.
  2. JinYeong Bak and Alice Oh. 2019. Variational hierarchical user-based conversation model. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1941–1950, Hong Kong, China. Association for Computational Linguistics.
  3. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, Ann Arbor, Michigan. Association for Computational Linguistics.
  4. The pushshift reddit dataset. In Proceedings of the Fourteenth International AAAI Conference on Web and Social Media, ICWSM 2020, Held Virtually, Original Venue: Atlanta, Georgia, USA, June 8-11, 2020, pages 830–839. AAAI Press.
  5. Pearson correlation coefficient. In Noise reduction in speech processing, pages 1–4. Springer.
  6. Latent dirichlet allocation. Journal of machine Learning research, 3(Jan):993–1022.
  7. Isabel Briggs-Myers and Peter B. Myers. 1995. Gifts differing: Understanding personality type. Davies-Black Publishing.
  8. Robert M Capraro and Mary Margaret Capraro. 2002. Myers-briggs type indicator score reliability across: Studies a meta-analytic reliability generalization study. Educational and Psychological Measurement, 62(4):590–602.
  9. Modeling personalization in continuous space for response generation via augmented Wasserstein autoencoders. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1931–1940, Hong Kong, China. Association for Computational Linguistics.
  10. Hyundong Cho and Jonathan May. 2020. Grounding conversations with improvised dialogues. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2398–2413, Online. Association for Computational Linguistics.
  11. Refocusing on relevance: Personalization in NLG. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5190–5202, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  12. PANDORA talks: Personality and demographics on Reddit. In Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media, pages 138–152, Online. Association for Computational Linguistics.
  13. Macro-average: Rare types are important too. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1138–1157, Online. Association for Computational Linguistics.
  14. The curious case of neural text degeneration. In International Conference on Learning Representations.
  15. Poly-encoders: Architectures and pre-training strategies for fast and accurate multi-sentence scoring. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
  16. Cobot in lambdamoo: An adaptive social statistics agent. Autonomous Agents and Multi-Agent Systems, 13:327–354.
  17. Construction of MBTI personality estimation model considering emotional information. In Proceedings of the 35th Pacific Asia Conference on Language, Information and Computation, pages 262–269, Shanghai, China. Association for Computational Lingustics.
  18. Optimus: Organizing sentences via pre-trained modeling of a latent space. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4678–4699, Online. Association for Computational Linguistics.
  19. A persona-based neural conversation model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 994–1003, Berlin, Germany. Association for Computational Linguistics.
  20. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582–4597, Online. Association for Computational Linguistics.
  21. Chin-Yew Lin and Franz Josef Och. 2004. Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), pages 605–612, Barcelona, Spain.
  22. A Three-Stage Learning Framework for Low-Resource Knowledge-Grounded Dialogue Generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2262–2272, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  23. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 61–68, Dublin, Ireland. Association for Computational Linguistics.
  24. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692.
  25. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
  26. Luna: Linear unified nested attention. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 2441–2453.
  27. One chatbot per person: Creating personalized chatbots based on implicit user profiles. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21, page 555–564, New York, NY, USA. Association for Computing Machinery.
  28. Training millions of personalized dialogue agents. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2775–2779, Brussels, Belgium. Association for Computational Linguistics.
  29. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
  30. Hierarchical transformers for long document classification. CoRR, abs/1910.10781.
  31. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830.
  32. David J Pittenger. 2005. Cautionary comments regarding the myers-briggs type indicator. Consulting Psychology Journal: Practice and Research, 57(3):210.
  33. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.
  34. Recipes for building an open-domain chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 300–325, Online. Association for Computational Linguistics.
  35. MBTI personality prediction for fictional characters using movie scripts. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6715–6724, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  36. BLEURT: Learning robust metrics for text generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881–7892, Online. Association for Computational Linguistics.
  37. Building end-to-end dialogue systems using generative hierarchical neural network models. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1).
  38. Hierarchical neural network generative models for movie dialogues. CoRR, abs/1507.04808.
  39. Blenderbot 3: a deployed conversational agent that continually learns to responsibly engage.
  40. BoB: BERT over BERT for training persona-based dialogue models from limited personalized data. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 167–177, Online. Association for Computational Linguistics.
  41. Exploiting persona information for diverse generation of conversational responses. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 5190–5196. International Joint Conferences on Artificial Intelligence Organization.
  42. Same author or just same topic? towards content-independent style representations. In Proceedings of the 7th Workshop on Representation Learning for NLP, pages 249–268, Dublin, Ireland. Association for Computational Linguistics.
  43. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
  44. Getting to know you: User attribute extraction from dialogues. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 581–589, Marseille, France. European Language Resources Association.
  45. Starspace: Embed all the things! Proceedings of the AAAI Conference on Artificial Intelligence, 32(1).
  46. Personalized response generation via generative split memory network. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1956–1970, Online. Association for Computational Linguistics.
  47. Yujie Xing and Raquel Fernández. 2018. Automatic evaluation of neural personality-based chatbots. In Proceedings of the 11th International Conference on Natural Language Generation, pages 189–194, Tilburg University, The Netherlands. Association for Computational Linguistics.
  48. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213, Melbourne, Australia. Association for Computational Linguistics.
  49. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations.
  50. HIBERT: Document level pre-training of hierarchical bidirectional transformers for document summarization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5059–5069, Florence, Italy. Association for Computational Linguistics.
  51. DIALOGPT : Large-scale generative pre-training for conversational response generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 270–278, Online. Association for Computational Linguistics.
  52. Personalized dialogue generation with diversified traits. CoRR, abs/1901.09672.
  53. Less is more: Learning to refine dialogue history for personalized dialogue generation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5808–5820, Seattle, United States. Association for Computational Linguistics.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Shuai Liu (215 papers)
  2. Hyundong J. Cho (1 paper)
  3. Marjorie Freedman (12 papers)
  4. Xuezhe Ma (50 papers)
  5. Jonathan May (76 papers)
Citations (14)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now