Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
72 tokens/sec
GPT-4o
61 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
8 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language Models (2402.15265v1)

Published 23 Feb 2024 in cs.HC and cs.CL

Abstract: LLMs have facilitated significant strides in generating conversational agents, enabling seamless, contextually relevant dialogues across diverse topics. However, the existing LLM-driven conversational agents have fixed personalities and functionalities, limiting their adaptability to individual user needs. Creating personalized agent personas with distinct expertise or traits can address this issue. Nonetheless, we lack knowledge of how people customize and interact with agent personas. In this research, we investigated how users customize agent personas and their impact on interaction quality, diversity, and dynamics. To this end, we developed CloChat, an interface supporting easy and accurate customization of agent personas in LLMs. We conducted a study comparing how participants interact with CloChat and ChatGPT. The results indicate that participants formed emotional bonds with the customized agents, engaged in more dynamic dialogues, and showed interest in sustaining interactions. These findings contribute to design implications for future systems with conversational agents using LLMs.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (100)
  1. [n. d.]. F/awesome-CHATGPT-prompts. https://github.com/f/awesome-chatgpt-prompts
  2. [n. d.]. Google Bard. https://bard.google.com/
  3. [n. d.]. Introducing ChatGPT. https://openai.com/blog/chatgpt/
  4. [n. d.]. Introducing GPTs. https://openai.com/blog/introducing-gpts
  5. KITLM: Domain-Specific Knowledge InTegration into Language Models for Question Answering. arXiv:2308.03638 [cs.CL]
  6. Open AI. 2023. Introducing GPTs. https://openai.com/blog/introducing-gpts
  7. Fine-Tuning GPT-3 for Russian Text Summarization. In Data Science and Intelligent Systems, Radek Silhavy, Petr Silhavy, and Zdenka Prokopova (Eds.). Springer International Publishing, Cham, 748–757.
  8. Toxicity in chatGPT: Analyzing persona-assigned language models. arXiv preprint arXiv: 2304.05335 (2023). https://doi.org/10.48550/arXiv.2304.05335
  9. An empirical investigation of the influence of persona with personality traits on conceptual design. Journal of Systems and Software 134 (2017), 324–339. https://doi.org/10.1016/j.jss.2017.09.020
  10. A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity. arXiv. https://doi.org/10.48550/arXiv.2302.04023
  11. The Chatbot Usability Scale: the design and pilot of a usability scale for interaction with AI-based conversational agents. Personal and Ubiquitous Computing 26 (2022), 95–119. https://doi.org/10.1007/s00779-021-01582-9
  12. Petter Bae Brandtzaeg and Asbjørn Følstad. 2018. Chatbots: changing user needs and motivations. interactions 25, 5 (2018), 38–43. https://doi.org/10.1145/3236669
  13. At your service: Designing voice assistant personalities to improve automotive user interfaces. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–11. https://doi.org/10.1145/3290605.3300270
  14. Virginia Braun and Victoria Clarke. 2012. Thematic analysis. American Psychological Association.
  15. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712 (2023). https://doi.org/10.48550/arXiv.2303.12712
  16. Personas: From Theory to Practices (NordiCHI ’08). Association for Computing Machinery, New York, NY, USA, 439–442. https://doi.org/10.1145/1463160.1463214
  17. Personas: from theory to practices. In Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges. 439–442. https://doi.org/10.1145/1463160.1463214
  18. Ana Paula Chaves and Marco Aurelio Gerosa. 2021. How Should My Chatbot Interact? A Survey on Social Characteristics in Human–Chatbot Interaction Design. International Journal of Human–Computer Interaction 37, 8 (2021), 729–758. https://doi.org/10.1080/10447318.2020.1841438
  19. Salesbot: Transitioning from chit-chat to task-oriented dialogues. arXiv preprint arXiv:2204.10591 (2022). https://doi.org/10.48550/arXiv.2204.10591
  20. ” What can i help you with?” infrequent users’ experiences of intelligent personal assistants. In Proceedings of the 19th international conference on human-computer interaction with mobile devices and services. 1–12. https://doi.org/10.1145/3098279.3098539
  21. Carolyn E Cutrona and Julie A Suhr. 1992. Controllability of stressful events and satisfaction with spouse support behaviors. Communication research 19, 2 (1992), 154–174. https://doi.org/10.1177/009365092019002002
  22. Chatbot personality and customer satisfaction. Info Support Research (2018). https://research.infosupport.com/wp-content/uploads/Chatbot-Personality-and-Customer-Satisfaction-Bachelor-Thesis-Information-Sciences-Hayco-de-Haan.pdf
  23. Toxicity in chatgpt: Analyzing persona-assigned language models. arXiv preprint arXiv:2304.05335 (2023).
  24. Wizard of wikipedia: Knowledge-powered conversational agents. arXiv preprint arXiv:1811.01241 (2018). https://doi.org/10.48550/arXiv.1811.01241
  25. Parallel Prototyping Leads to Better Design Results, More Divergence, and Increased Self-Efficacy. ACM Trans. Comput.-Hum. Interact. 17, 4, Article 18 (dec 2011), 24 pages. https://doi.org/10.1145/1879831.1879836
  26. Human conversational behavior. Human nature 8 (1997), 231–246. https://doi.org/10.1007/BF02912493
  27. Nature Editorial. 2023. ChatGPT is a black box: how AI research can break it open. https://www.nature.com/articles/d41586-023-02366-2
  28. Towards automated dialog personalization using mbti personality indicators. In Proc. Interspeech. 1968–1972. https://doi.org/10.21437/Interspeech.2022-376
  29. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR mental health 4, 2 (2017), e7785. https://doi.org/10.2196/mental.7785
  30. Social support in a virtual community: analysis of a clinic-affiliated online support group for persons living with HIV/AIDS. AIDS and Behavior 21 (2017), 3087–3099. https://doi.org/10.1007/s10461-016-1587-3
  31. Luciano Floridi and Massimo Chiriatti. 2020. GPT-3: Its nature, scope, limits, and consequences. Minds and Machines 30 (2020), 681–694. https://doi.org/10.1007/s11023-020-09548-1
  32. Neural approaches to conversational AI. In The 41st international ACM SIGIR conference on research & development in information retrieval. 1371–1374. https://doi.org/10.1145/3209978.3210183
  33. A knowledge-grounded neural conversation model. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32. https://doi.org/10.1609/aaai.v32i1.11977
  34. Anand Gokul. 2023. LLMs and AI: Understanding Its Reach and Impact. (2023).
  35. Generative language models and automated influence operations: Emerging threats and potential mitigations. arXiv preprint arXiv:2301.04246 (2023). https://doi.org/10.48550/arXiv.2301.04246
  36. Understanding the user experience of customer service chatbots: An experimental study of chatbot interaction design. International Journal of Human-Computer Studies 161 (2022), 102788. https://doi.org/10.1016/j.ijhcs.2022.102788
  37. hu.ma.ne. 2023. Ai Pin Overview. https://hu.ma.ne/aipin
  38. Linguistic Features to Consider When Applying Persona of the Real Person to the Text-Based Agent (MobileHCI ’20). Association for Computing Machinery, New York, NY, USA, Article 23, 4 pages. https://doi.org/10.1145/3406324.3410723
  39. Measuring and Explaining the Inter-Cluster Reliability of Multidimensional Projections. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2022), 551–561. https://doi.org/10.1109/TVCG.2021.3114833
  40. Uniform Manifold Approximation with Two-phase Optimization. In 2022 IEEE Visualization and Visual Analytics (VIS). IEEE, 80–84. https://doi.org/10.1109/VIS54862.2022.00025
  41. CLAMS: A Cluster Ambiguity Measure for Estimating Perceptual Variability in Visual Clustering. arXiv preprint arXiv:2308.00284 (2023). https://doi.org/10.48550/arXiv.2308.00284
  42. Personallm: Investigating the ability of gpt-3.5 to express personality traits and gender differences. arXiv preprint arXiv:2305.02547 (2023). https://doi.org/10.48550/arXiv.2305.02547
  43. Marius Kaminskas and Derek Bridge. 2016. Diversity, Serendipity, Novelty, and Coverage: A Survey and Empirical Analysis of Beyond-Accuracy Objectives in Recommender Systems. ACM Trans. Interact. Intell. Syst. 7, 1, Article 2 (dec 2016), 42 pages. https://doi.org/10.1145/2926720
  44. Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses. Review of Educational Research 68, 3 (1998), 350–386. https://doi.org/10.3102/00346543068003350
  45. Explaining the user experience of recommender systems. User modeling and user-adapted interaction 22 (2012), 441–504. https://doi.org/10.1007/s11257-011-9118-4
  46. A Baki Kocaballi. 2023. Conversational ai-powered design: Chatgpt as designer, user, and product. arXiv preprint arXiv:2302.07406 (2023). https://doi.org/10.48550/arXiv.2302.07406
  47. ” What does your Agent look like?” A Drawing Study to Understand Users’ Perceived Persona of Conversational Agent. In Extended abstracts of the 2019 CHI conference on human factors in computing systems. 1–6. https://doi.org/10.1145/3290607.3312796
  48. Nadine Lessio and Alexis Morris. 2020. Toward Design Archetypes for Conversational Agent Personality. In 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 3221–3228. https://doi.org/10.1109/SMC42975.2020.9283254
  49. A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv:1510.03055 (2015). https://doi.org/10.48550/arXiv.1510.03055
  50. You Truly Understand What I Need: Intellectual and Friendly Dialogue Agents grounding Knowledge and Persona. arXiv preprint arXiv:2301.02401 (2023). https://doi.org/10.48550/arXiv.2301.02401
  51. Li Liu and Vincent G Duffy. 2023. Exploring the Future Development of Artificial Intelligence (AI) Applications in Chatbots: A Bibliometric Analysis. International Journal of Social Robotics 15, 5 (2023), 703–716. https://doi.org/10.1007/s12369-022-00956-0
  52. Physician voice characteristics and patient satisfaction in online health consultation. Information & Management 57, 5 (2020), 103233. https://doi.org/10.1016/j.im.2019.103233
  53. Training millions of personalized dialogue agents. arXiv preprint arXiv:1809.01984 (2018). https://doi.org/10.48550/arXiv.1809.01984
  54. Robert R McCrae and Oliver P John. 1992. An introduction to the five-factor model and its applications. Journal of personality 60, 2 (1992), 175–215. https://doi.org/10.1111/j.1467-6494.1992.tb00970.x
  55. Sara Moussawi and Raquel Benbunan-Fich. 2021. The effect of voice and humour on users’ perceptions of personal intelligent agents. Behaviour & Information Technology 40, 15 (2021), 1603–1626. https://doi.org/10.1080/0144929X.2020.1772368
  56. How perceptions of intelligence and anthropomorphism affect adoption of personal intelligent agents. Electronic Markets 31 (2021), 343–364. https://doi.org/10.1007/s12525-020-00411-w
  57. A review of current trends in the development of chatbot systems. In 2020 6th International conference on advanced computing and communication systems (ICACCS). IEEE, 706–710. https://doi.org/10.1109/ICACCS48705.2020.9074420
  58. Ha Nguyen. 2022. Examining Teenagers’ Perceptions of Conversational Agents in Learning Settings. In Proceedings of the 21st Annual ACM Interaction Design and Children Conference (Braga, Portugal) (IDC ’22). Association for Computing Machinery, New York, NY, USA, 374–381. https://doi.org/10.1145/3501712.3529740
  59. S Nithuna and CA Laseena. 2020. Review on implementation techniques of chatbot. In 2020 International Conference on Communication and Signal Processing (ICCSP). IEEE, 0157–0161. https://doi.org/10.1109/ICCSP48568.2020.9182168
  60. Capabilities of gpt-4 on medical challenge problems. arXiv preprint arXiv:2303.13375 (2023). https://doi.org/10.48550/arXiv.2303.13375
  61. Inc Open AI. 2023. Introducing chatgpt. https://openai.com/blog/chatgpt/
  62. OpenAI. 2023. GPT-4 Technical Report. https://doi.org/10.48550/arXiv.2303.08774 arXiv:2303.08774 [cs.CL]
  63. Keyu Pan and Yawen Zeng. 2023. Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models. arXiv preprint arXiv:2307.16180 (2023). https://doi.org/10.48550/arXiv.2307.16180
  64. Towards understanding human similarity perception in the analysis of large sets of scatter plots. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 3659–3669. https://doi.org/10.1145/2858036.2858155
  65. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442 (2023). https://doi.org/10.48550/arXiv.2304.03442
  66. Alisha Pradhan and Amanda Lazar. 2021a. Hey Google, Do You Have a Personality? Designing Personality and Personas for Conversational Agents (CUI ’21). Association for Computing Machinery, New York, NY, USA, Article 12, 4 pages. https://doi.org/10.1145/3469595.3469607
  67. Alisha Pradhan and Amanda Lazar. 2021b. Hey Google, do you have a personality? Designing personality and personas for conversational agents. In Proceedings of the 3rd Conference on Conversational User Interfaces. 1–4. https://doi.org/10.1145/3469595.3469607
  68. ” Alexa is my new BFF” social roles, user satisfaction, and personification of the Amazon Echo. In Proceedings of the 2017 CHI conference extended abstracts on human factors in computing systems. 2853–2859. https://doi.org/10.1145/3027063.3053246
  69. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 1, 2 (2022), 3. https://doi.org/10.48550/arXiv.2204.06125
  70. Directed Diversity: Leveraging Language Embedding Distances for Collective Creativity in Crowd Ideation. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 393, 35 pages. https://doi.org/10.1145/3411764.3445782
  71. Personality traits in large language models. arXiv preprint arXiv:2307.00184 (2023). https://doi.org/10.48550/arXiv.2307.00184
  72. The Effect of Experience on Persona Perceptions. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3334480.3382786
  73. Persona Perception Scale: Developing and Validating an Instrument for Human-Like Representations of Data. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA ’18). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3170427.3188461
  74. Persona Transparency: Analyzing the Impact of Explanations on Perceptions of Data-Driven Personas. International Journal of Human–Computer Interaction 36, 8 (2020), 788–800. https://doi.org/10.1080/10447318.2019.1688946
  75. Dale Schuurmans. 2023. Memory Augmented Large Language Models are Computationally Universal. arXiv:2301.04589 [cs.CL]
  76. Revealing persona biases in dialogue systems. arXiv preprint arXiv:2104.08728 (2021). https://doi.org/10.48550/arXiv.2104.08728
  77. Towards controllable biases in language generation. arXiv preprint arXiv:2005.00268 (2020). https://doi.org/10.48550/arXiv.2005.00268
  78. Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems 25 (2012). https://doi.org/10.48550/arXiv.1206.2944
  79. Viriya Taecharungroj. 2023. “What Can ChatGPT Do?” Analyzing Early Reactions to the Innovative AI Chatbot on Twitter. Big Data and Cognitive Computing 7, 1 (2023), 35. https://doi.org/10.3390/bdcc7010035
  80. Deborah Tannen. 1984. Conversational Style: Analyzing Talk Among Friends. Vol. 61. 188 pages. https://doi.org/10.2307/414501
  81. John W. Tukey. 1949. Comparing Individual Means in the Analysis of Variance. Biometrics 5, 2 (1949), 99–114. http://www.jstor.org/stable/3001913
  82. Stanford University. 2023. Dialogue distillery: Crafting interpolable, interpretable, and introspectable dialogue from LLMs. In Alexa Prize SocialBot Grand Challenge 5 Proceedings. https://www.amazon.science/alexa-prize/proceedings/chirpy-cardinal-dialogue-distillery-crafting-interpolable-interpretable-and-introspectable-dialogue-from-llms
  83. Aleksandra Urman and Mykola Makhortykh. 2023. The Silence of the LLMs: Cross-Lingual Analysis of Political Bias and False Information Prevalence in ChatGPT, Google Bard, and Bing Chat. (2023).
  84. Eliciting and analysing users’ envisioned dialogues with perfect voice assistants. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15. https://doi.org/10.1145/3411764.3445536
  85. Developing a Personality Model for Speech-Based Conversational Agents Using the Psycholexical Approach (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376210
  86. User perceptions of extraversion in chatbots after repeated use. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–18. https://doi.org/10.1145/3491102.3502058
  87. Jieyu Wang and Anita Komlodi. 2012. Children’s Formal and Informal Definition of Technology. In Proceedings of the 2012 IConference (Toronto, Ontario, Canada) (iConference ’12). Association for Computing Machinery, New York, NY, USA, 587–588. https://doi.org/10.1145/2132176.2132299
  88. Persuasion for good: Towards a personalized persuasive dialogue system for social good. arXiv preprint arXiv:1906.06725 (2019). https://doi.org/10.48550/arXiv.1906.06725
  89. Philip Weber and Thomas Ludwig. 2020. (Non-) Interacting with conversational agents: perceptions and motivations of using chatbots and voice assistants. In Proceedings of Mensch und Computer 2020. 321–331. https://doi.org/10.1145/3404983.3405513
  90. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359 (2021). https://doi.org/10.48550/arXiv.2112.04359
  91. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382 (2023). https://doi.org/10.48550/arXiv.2302.11382
  92. Chatbot for Health Care and Oncology Applications Using Artificial Intelligence and Machine Learning: Systematic Review. JMIR Cancer 7, 4 (29 Nov 2021), e27850. https://doi.org/10.2196/27850
  93. Generating stylistic and personalized dialogues for virtual agents in narratives. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems. 737–746. https://doi.org/10.5555/3545946.3598706
  94. Shanshan Yang and Chris Evans. 2019. Opportunities and challenges in using AI chatbots in higher education. In Proceedings of the 2019 3rd International Conference on Education and E-Learning. 79–83. https://doi.org/10.1145/3371647.3371659
  95. Llm lies: Hallucinations are not bugs, but features as adversarial examples. arXiv preprint arXiv:2310.01469 (2023).
  96. GPT-4 outperforms ChatGPT in answering non-English questions related to cirrhosis. medRxiv (2023), 2023–05. https://doi.org/10.1101/2023.05.04.23289482
  97. User engagement study with virtual agents under different cultural contexts. In Intelligent Virtual Agents: 16th International Conference, IVA 2016, Los Angeles, CA, USA, September 20–23, 2016, Proceedings 16. Springer, 364–368. https://doi.org/10.1007/978-3-319-47665-0_34
  98. Personalizing Dialogue Agents: I have a dog, do you have pets too? https://doi.org/10.48550/arXiv.1801.07243 arXiv:1801.07243 [cs.AI]
  99. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910 (2022). https://doi.org/10.48550/arXiv.2211.01910
  100. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593 (2019). https://doi.org/10.48550/arXiv.1909.08593
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Juhye Ha (1 paper)
  2. Hyeon Jeon (26 papers)
  3. DaEun Han (1 paper)
  4. Jinwook Seo (30 papers)
  5. Changhoon Oh (4 papers)
Citations (5)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets