Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Task Supportive and Personalized Human-Large Language Model Interaction: A User Study (2402.06170v1)

Published 9 Feb 2024 in cs.HC and cs.IR

Abstract: LLM applications, such as ChatGPT, are a powerful tool for online information-seeking (IS) and problem-solving tasks. However, users still face challenges initializing and refining prompts, and their cognitive barriers and biased perceptions further impede task completion. These issues reflect broader challenges identified within the fields of IS and interactive information retrieval (IIR). To address these, our approach integrates task context and user perceptions into human-ChatGPT interactions through prompt engineering. We developed a ChatGPT-like platform integrated with supportive functions, including perception articulation, prompt suggestion, and conversation explanation. Our findings of a user study demonstrate that the supportive functions help users manage expectations, reduce cognitive loads, better refine prompts, and increase user engagement. This research enhances our comprehension of designing proactive and user-centric systems with LLMs. It offers insights into evaluating human-LLM interactions and emphasizes potential challenges for under served users.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. Guidelines for human-AI interaction. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–13.
  2. Leif Azzopardi. 2021. Cognitive biases in search: a review and reflection of cognitive biases in Information Retrieval. In Proceedings of the 2021 conference on human information interaction and retrieval. 27–37.
  3. ChatGPT: Applications, opportunities, and threats. In 2023 Systems and Information Engineering Design Symposium (SIEDS). IEEE, 274–279.
  4. David Baidoo-Anu and Leticia Owusu Ansah. 2023. Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI 7, 1 (2023), 52–62.
  5. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
  6. Barbara J Ford. 1995. Information literacy as a barrier. IFLA journal 21, 2 (1995), 99–101.
  7. Struggling or exploring? Disambiguating long search sessions. In Proceedings of the 7th ACM international conference on Web search and data mining. 53–62.
  8. ChatGPT for good? On opportunities and challenges of large language models for education. Learning and individual differences 103 (2023), 102274.
  9. ” Help Me Help the AI”: Understanding How Explainability Can Support Human-AI Interaction. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17.
  10. Evaluating human-language model interaction. arXiv preprint arXiv:2212.09746 (2022).
  11. Proactive conversational agents. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining. 1244–1247.
  12. Proactive Conversational Agents in the Post-ChatGPT World. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 3452–3455.
  13. Task, information seeking intentions, and user behavior: Toward a multi-level understanding of Web search. In Proceedings of the 2019 conference on human information interaction and retrieval. 123–132.
  14. Identifying and predicting the states of complex search tasks. In Proceedings of the 2020 conference on human information interaction and retrieval. 193–202.
  15. Jiqun Liu and Chirag Shah. 2019. Proactive identification of query failure. Proceedings of the Association for Information Science and Technology 56, 1 (2019), 176–185.
  16. Jiqun Liu and Chirag Shah. 2022. Leveraging user interaction signals and task state information in adaptively optimizing usefulness-oriented search sessions. In Proceedings of the 22nd ACM/IEEE joint conference on digital libraries. 1–11.
  17. ” Satisfaction with Failure” or” Unsatisfied Success” Investigating the Relationship between Search Success and User Satisfaction. In Proceedings of the 2018 world wide web conference. 1533–1542.
  18. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. Comput. Surveys 55, 9 (2023), 1–35.
  19. Struggling and success in web search. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 1551–1560.
  20. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (2022), 27730–27744.
  21. Who are the crowdworkers? Shifting demographics in Mechanical Turk. In CHI’10 extended abstracts on Human factors in computing systems. 2863–2872.
  22. Representing Tasks with a Graph-Based Method for Supporting Users in Complex Search Tasks. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval. 378–382.
  23. Reijo Savolainen. 2015. Cognitive barriers to information seeking: A conceptual analysis. Journal of Information Science 41, 5 (2015), 613–623.
  24. CHIRAG SHAH and EMILY M BENDER. 2023. Envisioning Information Access Systems: What Makes for Good Tools and a Healthy Web? (2023).
  25. Taking search to task. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval. 1–13.
  26. The user experience of ChatGPT: Findings from a questionnaire study of early users. In Proceedings of the 5th International Conference on Conversational User Interfaces. 1–10.
  27. Crowdsourcing Backstories for Complex Task-Based Search. In Proceedings of the 25th Australasian Document Computing Symposium. 1–6.
  28. Large language models can accurately predict searcher preferences. arXiv preprint arXiv:2309.10621 (2023).
  29. What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments 10, 1 (2023), 15.
  30. Ben Wang and Jiqun Liu. 2022. Investigating the Relationship between In-Situ User Expectations and Web Search Behavior. Proceedings of the Association for Information Science and Technology 59, 1 (2022), 827–829.
  31. Ben Wang and Jiqun Liu. 2023. Investigating the role of in-situ user expectations in Web search. Information Processing & Management 60, 3 (2023), 103300.
  32. Explainable reasoning over knowledge graphs for recommendation. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 5329–5336.
  33. Explainable AI: A brief survey on history, research areas, approaches and challenges. In Natural Language Processing and Chinese Computing: 8th CCF International Conference, NLPCC 2019, Dunhuang, China, October 9–14, 2019, Proceedings, Part II 8. Springer, 563–574.
  34. Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21.
  35. Xiaoming Zhai. 2022. ChatGPT user experience: Implications for education. Available at SSRN 4312418 (2022).
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Ben Wang (42 papers)
  2. Jiqun Liu (15 papers)
  3. Jamshed Karimnazarov (2 papers)
  4. Nicolas Thompson (1 paper)
Citations (9)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets