Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Optimizing Risk-averse Human-AI Hybrid Teams (2403.08386v1)

Published 13 Mar 2024 in cs.AI and cs.LG

Abstract: We anticipate increased instances of humans and AI systems working together in what we refer to as a hybrid team. The increase in collaboration is expected as AI systems gain proficiency and their adoption becomes more widespread. However, their behavior is not error-free, making hybrid teams a very suitable solution. As such, we consider methods for improving performance for these teams of humans and AI systems. For hybrid teams, we will refer to both the humans and AI systems as agents. To improve team performance over that seen for agents operating individually, we propose a manager which learns, through a standard Reinforcement Learning scheme, how to best delegate, over time, the responsibility of taking a decision to any of the agents. We further guide the manager's learning so they also minimize how many changes in delegation are made resulting from undesirable team behavior. We demonstrate the optimality of our manager's performance in several grid environments which include failure states which terminate an episode and should be avoided. We perform our experiments with teams of agents with varying degrees of acceptable risk, in the form of proximity to a failure state, and measure the manager's ability to make effective delegation decisions with respect to its own risk-based constraints, then compare these to the optimal decisions. Our results show our manager can successfully learn desirable delegations which result in team paths near/exactly optimal with respect to path length and number of delegations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. M. Carroll, R. Shah, M. K. Ho, T. Griffiths, S. Seshia, P. Abbeel, and A. Dragan, “On the utility of learning about humans for human-ai coordination,” Advances in neural information processing systems, vol. 32, 2019.
  2. M. Chen, S. Nikolaidis, H. Soh, D. Hsu, and S. Srinivasa, “Planning with trust for human-robot collaboration,” in Proceedings of the 2018 ACM/IEEE international conference on human-robot interaction, 2018, pp. 307–315.
  3. Z. R. Khavas, S. R. Ahmadzadeh, and P. Robinette, “Modeling trust in human-robot interaction: A survey,” in Social Robotics, A. R. Wagner, D. Feil-Seifer, K. S. Haring, S. Rossi, T. Williams, H. He, and S. Sam Ge, Eds.   Cham: Springer International Publishing, 2020, pp. 529–541.
  4. S. Westby and C. Riedl, “Collective intelligence in human-ai teams: A bayesian theory of mind approach,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 5, 2023, pp. 6119–6127.
  5. S. A. Wu, R. E. Wang, J. A. Evans, J. B. Tenenbaum, D. C. Parkes, and M. Kleiman-Weiner, “Too many cooks: Bayesian inference for coordinating multi-agent collaboration,” Topics in Cognitive Science, vol. 13, no. 2, pp. 414–432, 2021.
  6. U. Agudo, K. G. Liberal, M. Arrese, and H. Matute, “The impact of ai errors in a human-in-the-loop process,” Cognitive Research: Principles and Implications, vol. 9, no. 1, p. 1, 2024.
  7. Á. A. Cabrera, A. J. Druck, J. I. Hong, and A. Perer, “Discovering and validating ai errors with crowdsourced failure reports,” Proceedings of the ACM on Human-Computer Interaction, vol. 5, no. CSCW2, pp. 1–22, 2021.
  8. K. Haegler, R. Zernecke, A. M. Kleemann, J. Albrecht, O. Pollatos, H. Brückmann, and M. Wiesmann, “No fear no risk! human risk behavior is affected by chemosensory anxiety signals,” Neuropsychologia, vol. 48, no. 13, pp. 3901–3908, 2010.
  9. A. Mahmood, J. W. Fung, I. Won, and C.-M. Huang, “Owning mistakes sincerely: Strategies for mitigating ai errors,” in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 2022, pp. 1–11.
  10. S. Russell, I. S. Moskowitz, and A. Raglin, “Human information interaction, artificial intelligence, and errors,” Autonomy and Artificial Intelligence: A Threat or Savior?, pp. 71–101, 2017.
  11. J. Afanador, M. Baptista, and N. Oren, “An adversarial algorithm for delegation,” in Agreement Technologies: 6th International Conference, AT 2018, Bergen, Norway, December 6-7, 2018, Revised Selected Papers 6.   Springer, 2019, pp. 130–145.
  12. G. Chen, X. Li, C. Sun, and H. Wang, “Learning to make adherence-aware advice,” arXiv preprint arXiv:2310.00817, 2023.
  13. A. Fuchs, A. Passarella, and M. Conti, “Optimizing delegation in collaborative human-ai hybrid teams,” 2024.
  14. ——, “A cognitive framework for delegation between error-prone ai and human agents,” in 2022 IEEE International Conference on Smart Computing (SMARTCOMP).   IEEE, 2022, pp. 317–322.
  15. ——, “Compensating for sensing failures via delegation in human–ai hybrid systems,” Sensors, vol. 23, no. 7, p. 3409, 2023.
  16. ——, “Optimizing delegation between human and ai collaborative agents,” in Workshop on Hybrid Human-Machine Learning and Decision Making, 2023.
  17. C. C. Aggarwal, “Instance-based learning: A survey.” Data classification: algorithms and applications, vol. 157, 2014.
  18. F. M. Garcia, C. Nota, and P. S. Thomas, “Learning reusable options for multi-task reinforcement learning,” arXiv preprint arXiv:2001.01577, 2020.
  19. J. Erskine and C. Lehnert, “Developing cooperative policies for multi-stage reinforcement learning tasks,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 6590–6597, 2022.
  20. J. Chakravorty, N. Ward, J. Roy, M. Chevalier-Boisvert, S. Basu, A. Lupu, and D. Precup, “Option-critic in cooperative multi-agent systems,” arXiv preprint arXiv:1911.12825, 2019.
  21. J. Chen, M. Haliem, T. Lan, and V. Aggarwal, “Multi-agent deep covering option discovery,” arXiv preprint arXiv:2210.03269, 2022.
  22. K. Kurzer, C. Zhou, and J. M. Zöllner, “Decentralized cooperative planning for automated vehicles with hierarchical monte carlo tree search,” in 2018 IEEE intelligent vehicles symposium (IV).   IEEE, 2018, pp. 529–536.
  23. Q. P. Lau, M.-L. Lee, and W. Hsu, “Coordination guided reinforcement learning.” in AAMAS, 2012, pp. 215–222.
  24. K. Menda, Y.-C. Chen, J. Grana, J. W. Bono, B. D. Tracey, M. J. Kochenderfer, and D. Wolpert, “Deep reinforcement learning for event-driven multi-agent decision processes,” IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 4, pp. 1259–1268, 2018.
  25. K. Rohanimanesh and S. Mahadevan, “Learning to take concurrent actions,” Advances in neural information processing systems, vol. 15, 2002.
  26. A. J. Singh, A. Kumar, and H. C. Lau, “Hierarchical multiagent reinforcement learning for maritime traffic management,” 2020.
  27. M. Yang, J. Zhao, X. Hu, W. Zhou, and H. Li, “Ldsa: Learning dynamic subtask assignment in cooperative multi-agent reinforcement learning,” arXiv preprint arXiv:2205.02561, 2022.
  28. A. Jacq, J. Ferret, O. Pietquin, and M. Geist, “Lazy-mdps: Towards interpretable rl by learning when to act,” in Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, 2022, pp. 669–677.
  29. V. B. Meresht, A. De, A. Singla, and M. Gomez-Rodriguez, “Learning to switch between machines and humans,” CoRR, vol. abs/2002.04258, 2020. [Online]. Available: https://arxiv.org/abs/2002.04258
  30. E. Straitouri, A. Singla, V. B. Meresht, and M. Gomez-Rodriguez, “Reinforcement learning under algorithmic triage,” CoRR, vol. abs/2109.11328, 2021. [Online]. Available: https://arxiv.org/abs/2109.11328
  31. D. Richards and A. Stedmon, “To delegate or not to delegate: A review of control frameworks for autonomous cars,” Applied ergonomics, vol. 53, pp. 383–388, 2016.
  32. S. Palmer, D. Richards, G. Shelton-Rayner, K. Izzetoglu, and D. Inch, “Assessing variable levels of delegated control–a novel measure of trust,” in HCI International 2020–Late Breaking Papers: Cognition, Learning and Games: 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings 22.   Springer, 2020, pp. 202–215.
  33. C. Candrian and A. Scherer, “Rise of the machines: Delegating decisions to autonomous ai,” Computers in Human Behavior, vol. 134, p. 107308, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Andrew Fuchs (8 papers)
  2. Andrea Passarella (91 papers)
  3. Marco Conti (73 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets