Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deconstructing Human-AI Collaboration: Agency, Interaction, and Adaptation (2404.12056v1)

Published 18 Apr 2024 in cs.HC and cs.AI

Abstract: As full AI-based automation remains out of reach in most real-world applications, the focus has instead shifted to leveraging the strengths of both human and AI agents, creating effective collaborative systems. The rapid advances in this area have yielded increasingly more complex systems and frameworks, while the nuance of their characterization has gotten more vague. Similarly, the existing conceptual models no longer capture the elaborate processes of these systems nor describe the entire scope of their collaboration paradigms. In this paper, we propose a new unified set of dimensions through which to analyze and describe human-AI systems. Our conceptual model is centered around three high-level aspects - agency, interaction, and adaptation - and is developed through a multi-step process. Firstly, an initial design space is proposed by surveying the literature and consolidating existing definitions and conceptual frameworks. Secondly, this model is iteratively refined and validated by conducting semi-structured interviews with nine researchers in this field. Lastly, to illustrate the applicability of our design space, we utilize it to provide a structured description of selected human-AI systems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Effects of communication directionality and AI agent differences in human-AI interaction. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (2021), ACM, pp. 1–15.
  2. Guidelines for human-AI interaction. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (2019), ACM, pp. 1–13.
  3. Comparing visual-interactive labeling with active learning: An experimental study. IEEE Trans. on Visualization and Computer Graphics 24, 1 (2018), 298–308.
  4. Brehmer M., Munzner T.: A multi-level typology of abstract visualization tasks. IEEE Trans. on Visualization and Computer Graphics 19, 12 (2013), 2376–2385.
  5. A provenance task abstraction framework. IEEE Trans. on Visualization and Computer Graphics 39, 6 (2019), 46–60.
  6. Characterizing guidance in visual analytics. IEEE Trans. on Visualization and Computer Graphics 23, 1 (2017), 111–120.
  7. A review of guidance approaches in visual data analysis: A multifocal perspective. Computer Graphics Forum 38, 3 (2019), 861–879.
  8. What did my AI learn? How data scientists make sense of model behavior. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (2023), vol. 30, ACM, pp. 1–27.
  9. Intellingo: An intelligible translation environment. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (2018), ACM, pp. 1–13.
  10. “Hello AI”: Uncovering the onboarding needs of medical practitioners for human-AI collaborative decision-making. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (2019), vol. 3, ACM, pp. 1–24.
  11. Hybrid intelligence. Business & Information Systems Engineering 61, 5 (2019), 637–643.
  12. Dudley J. J., Kristensson P. O.: A review of user interface design for interactive machine learning. ACM Trans. on Interactive Intelligent Systems 8, 2 (2018), 1–37.
  13. El-Assady M., Moruzzi C.: Which biases and reasoning pitfalls do explanations trigger? Decomposing communication processes in human-AI interaction. IEEE Trans. on Visualization and Computer Graphics 42, 6 (2022), 11–23.
  14. A grammar-based approach for applying visualization taxonomies to interaction logs. Computer Graphics Forum 41, 3 (2022), 489–500.
  15. Gotz D., Zhou M. X.: Characterizing users’ visual analytic activity for insight provenance. Proc. of IEEE Conf. on Visual Analytics Science and Technology (2008), 123–130.
  16. VATLD : A visual analytics system to assess, understand and improve traffic light detection. IEEE Trans. on Visualization and Computer Graphics 27, 2 (2021), 261–271.
  17. Gamut: A design probe to understand how data scientists understand machine learning models. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (2019), ACM, pp. 1–13.
  18. Horvitz E.: Principles of mixed-initiative user interfaces. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (1999), ACM, pp. 159–166.
  19. Artificial intelligence and machine learning. Electronic Markets 32, 4 (2022), 2235–2244.
  20. Towards a science of human-AI decision making: An overview of design space in empirical human-subject studies. Proc. of the ACM Conf. on Fairness, Accountability, and Transparency (2023), 1369–1385.
  21. The state-of-the-art in predictive visual analytics. Computer Graphics Forum 36, 3 (2017), 539–562.
  22. A human-AI collaborative approach for clinical decision making on rehabilitation assessment. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (2021), ACM, pp. 1–14.
  23. Bridging from goals to tasks with design study analysis reports. IEEE Trans. on Visualization and Computer Graphics 24 (2018), 435–445.
  24. Where are we so far? Understanding data storytelling tools from the perspective of human-AI collaboration. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (2024), ACM.
  25. A review on human–AI interaction in machine learning and insights for medical applications. Int. Journal of Environmental Research and Public Health 18, 4 (2021), 2121.
  26. Human-computer collaboration for visual analytics: An agent-based framework. Computer Graphics Forum (2023).
  27. RLHF-blender: A configurable interactive interface for learning from diverse human feedback. Interactive Learning with Implicit Human Feedback Workshop at ICML (2023).
  28. RuleMatrix: Visualizing and understanding classifiers with rules. IEEE Trans. on Visualization and Computer Graphics 25, 1 (2019), 342–352.
  29. Mou Y., Xu K.: The media inequality: Comparing the initial human-human and human-AI social interactions. Computers in Human Behavior 72 (2017), 432–440.
  30. ProtoSteer: Steering deep sequence model with prototypes. IEEE Trans. on Visualization and Computer Graphics 26, 1 (2020), 238–248.
  31. Believe it or not: Designing a human-AI partnership for mixed-initiative fact-checking. In Proc. of the Annual ACM Symposium on User Interface Software and Technology (2018), ACM, pp. 189–199.
  32. A typology of guidance tasks in mixed-initiative visual analytics environments. Computer Graphics Forum 41, 3 (2022), 465–476.
  33. A methodology for task-driven guidance design. EuroVis Workshop on Visual Analytics (2023), 6 pages.
  34. Experimental evidence of effective human–AI collaboration in medical decision-making. Scientific Reports 12, 1 (2022), 14952.
  35. Towards effective human-AI decision-making: The role of human learning in appropriate reliance on AI advice. Int. Conf. on Information Systems (2023).
  36. Lotse: A practical framework for guidance in visual analytics. IEEE Trans. on Visualization and Computer Graphics (2022), 1124–1134.
  37. A survey of human-centered evaluations in human-centered machine learning. Computer Graphics Forum 40, 3 (2021), 543–568.
  38. Learning and teaching in co-adaptive guidance for mixed-initiative visual analytics. In EuroVis Workshop on Visual Analytics (Goslar, 2020), Turkay C., Vrotsou K., (Eds.), Eurographics Association, pp. 61–65.
  39. Co-adaptive visual data analysis and guidance processes. Computers & Graphics 100 (2021), 93–105.
  40. Cobots in knowledge work. Journal of Business Research 125 (2021), 135–142.
  41. Learning contextualized user preferences for co-adaptive guidance in mixed-initiative topic model refinement. Computer Graphics Forum 40, 3 (2021), 215–226.
  42. Knowledge generation model for visual analytics. IEEE Trans. on Visualization and Computer Graphics 20, 12 (2014), 1604–1613.
  43. Sundar S. S.: Rise of machine agency: A framework for studying the psychology of human-AI interaction (haii). Journal of Computer-Mediated Communication 25, 1 (2020), 74–88.
  44. From human-human collaboration to human-AI collaboration: Designing AI systems that can work together with people. In Extended Abstracts in the Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (2020), ACM, pp. 1–6.
  45. A survey on ML4VIS: Applying machine learning advances to data visualization. IEEE Trans. on Visualization and Computer Graphics 28, 12 (2022), 5134–5153.
  46. Podium: Ranking data using mixed-initiative visual analytics. IEEE Trans. on Visualization and Computer Graphics 24, 1 (2018), 288–297.
  47. The What-If tool: Interactive probing of machine learning models. IEEE Trans. on Visualization and Computer Graphics (2019), 1–1.
  48. Xu W., Gao Z.: Applying human-centered AI in developing effective human-AI teaming: A perspective of human-AI joint cognitive systems. Interactions 31, 1 (2024).
  49. Re-examining whether, why, and how human-AI interaction is uniquely difficult to design. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (2020), ACM, pp. 1–13.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Steffen Holter (4 papers)
  2. Mennatallah El-Assady (54 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets