Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A LLM-based Controllable, Scalable, Human-Involved User Simulator Framework for Conversational Recommender Systems (2405.08035v1)

Published 13 May 2024 in cs.HC and cs.AI

Abstract: Conversational Recommender System (CRS) leverages real-time feedback from users to dynamically model their preferences, thereby enhancing the system's ability to provide personalized recommendations and improving the overall user experience. CRS has demonstrated significant promise, prompting researchers to concentrate their efforts on developing user simulators that are both more realistic and trustworthy. The emergence of LLMs has marked the onset of a new epoch in computational capabilities, exhibiting human-level intelligence in various tasks. Research efforts have been made to utilize LLMs for building user simulators to evaluate the performance of CRS. Although these efforts showcase innovation, they are accompanied by certain limitations. In this work, we introduce a Controllable, Scalable, and Human-Involved (CSHI) simulator framework that manages the behavior of user simulators across various stages via a plugin manager. CSHI customizes the simulation of user behavior and interactions to provide a more lifelike and convincing user interaction experience. Through experiments and case studies in two conversational recommendation scenarios, we show that our framework can adapt to a variety of conversational recommendation settings and effectively simulate users' personalized preferences. Consequently, our simulator is able to generate feedback that closely mirrors that of real users. This facilitates a reliable assessment of existing CRS studies and promotes the creation of high-quality conversational recommendation datasets.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web, pages 173–182, 2017.
  2. Matrix factorization techniques for recommender systems. Computer, 42(8):30–37, 2009.
  3. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems, pages 7–10, 2016.
  4. Advances and challenges in conversational recommender systems: A survey. AI open, 2:100–126, 2021.
  5. Yueming Sun and Yi Zhang. Conversational recommender system. In The 41st international acm sigir conference on research & development in information retrieval, pages 235–244, 2018.
  6. Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. In Proceedings of the 13th International Conference on Web Search and Data Mining, pages 304–312, 2020.
  7. Interactive path reasoning on graph for conversational recommendation. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2073–2083, 2020.
  8. Unified conversational recommendation policy learning via graph-based reinforcement learning. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1431–1441, 2021.
  9. Adapting user preference to online feedback in multi-round conversational recommendation. In Proceedings of the 14th ACM international conference on web search and data mining, pages 364–372, 2021.
  10. Knowledge graph-enhanced sampling for conversational recommendation system. IEEE Transactions on Knowledge and Data Engineering, 2022.
  11. Towards knowledge-based recommender dialog system. arXiv preprint arXiv:1908.05391, 2019.
  12. Improving conversational recommender systems via knowledge graph based semantic fusion. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1006–1014, 2020.
  13. Barcor: Towards a unified framework for conversational recommendation systems. arXiv preprint arXiv:2203.14257, 2022.
  14. Towards unified conversational recommender systems via knowledge-enhanced prompt learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1929–1937, 2022.
  15. Learning neural templates for recommender dialogue system. arXiv preprint arXiv:2109.12302, 2021.
  16. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023.
  17. Uncovering chatgpt’s capabilities in recommender systems. In Proceedings of the 17th ACM Conference on Recommender Systems, pages 1126–1132, 2023.
  18. Chat-rec: Towards interactive and explainable llms-augmented recommender system. arXiv preprint arXiv:2303.14524, 2023.
  19. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022.
  20. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 2024.
  21. Testing the general deductive reasoning capacity of large language models using ood examples. Advances in Neural Information Processing Systems, 36, 2024.
  22. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864, 2023.
  23. A survey on large language model based autonomous agents. corr abs/2308.11432 (2023), 2023.
  24. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents. arXiv preprint arXiv:2308.10848, 2023.
  25. Unveiling the truth and facilitating change: Towards agent-based large-scale social movement simulation. arXiv preprint arXiv:2402.16333, 2024.
  26. Rethinking the evaluation for conversational recommendation in the era of large language models. arXiv preprint arXiv:2305.13112, 2023.
  27. Recagent: A novel simulation paradigm for recommender systems. arXiv preprint arXiv:2306.02552, 2023.
  28. On generative agents in recommendation. arXiv preprint arXiv:2310.10108, 2023.
  29. Recommender ai agent: Integrating large language models for interactive recommendations. arXiv preprint arXiv:2308.16505, 2023.
  30. How reliable is your simulator? analysis on the limitations of current llm-based user simulators for conversational recommendation. arXiv preprint arXiv:2403.16416, 2024.
  31. Towards deep conversational recommendations. Advances in neural information processing systems, 31, 2018.
  32. Opendialkg: Explainable conversational reasoning with attention-based walks over knowledge graphs. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 845–854, 2019.
  33. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019.
  34. Large language models for generative recommendation: A survey and visionary discussions. arXiv preprint arXiv:2309.01157, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Lixi Zhu (4 papers)
  2. Xiaowen Huang (12 papers)
  3. Jitao Sang (71 papers)