Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Innate-Values-driven Reinforcement Learning based Cooperative Multi-Agent Cognitive Modeling (2401.05572v2)

Published 10 Jan 2024 in cs.LG, cs.AI, cs.MA, and cs.RO

Abstract: In multi-agent systems (MAS), the dynamic interaction among multiple decision-makers is driven by their innate values, affecting the environment's state, and can cause specific behavioral patterns to emerge. On the other hand, innate values in cognitive modeling reflect individual interests and preferences for specific tasks and drive them to develop diverse skills and plans, satisfying their various needs and achieving common goals in cooperation. Therefore, building the awareness of AI agents to balance the group utilities and system costs and meet group members' needs in their cooperation is a crucial problem for individuals learning to support their community and even integrate into human society in the long term. However, the current MAS reinforcement learning domain lacks a general intrinsic model to describe agents' dynamic motivation for decision-making and learning from an individual needs perspective in their cooperation. To address the gap, this paper proposes a general MAS innate-values reinforcement learning (IVRL) architecture from the individual preferences angle. We tested the Multi-Agent IVRL Actor-Critic Model in different StarCraft Multi-Agent Challenge (SMAC) settings, which demonstrated its potential to organize the group's behaviours to achieve better performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. Alderfer, C. P. 1972. Existence, relatedness, and growth: Human needs in organizational settings.
  2. Intrinsically motivated learning systems: an overview. Intrinsically motivated learning in natural and artificial systems, 1–14.
  3. Intrinsically motivated learning of hierarchical collections of skills. In Proceedings of the 3rd International Conference on Development and Learning, volume 112, 19. Citeseer.
  4. A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 38(2): 156–172.
  5. Motivation and action. Springer.
  6. A survey and critique of multiagent deep reinforcement learning. Autonomous Agents and Multi-Agent Systems, 33(6): 750–797.
  7. Littman, M. L. 1994. Markov games as a framework for multi-agent reinforcement learning. In Machine learning proceedings 1994, 157–163. Elsevier.
  8. A real-time novelty detector for a mobile robot. EUREL European Advanced Robotics Systems Masterclass and Conference.
  9. Maslow, A. H. 1958. A Dynamic Theory of Human Motivation.
  10. Merrick, K. E. 2013. Novelty and beyond: Towards combined motivation models and integrated learning architectures. Intrinsically motivated learning in natural and artificial systems, 209–233.
  11. Motivated reinforcement learning: curious characters for multiuser games. Springer Science & Business Media.
  12. Puterman, M. L. 2014. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons.
  13. Multi-objective multi-agent decision making: a utility-based analysis and survey. Autonomous Agents and Multi-Agent Systems, 34(1): 10.
  14. Monotonic value function factorisation for deep multi-agent reinforcement learning. The Journal of Machine Learning Research, 21(1): 7234–7284.
  15. The StarCraft Multi-Agent Challenge. CoRR, abs/1902.04043.
  16. Evolution and learning in an intrinsically motivated reinforcement learning robot. In Advances in Artificial Life: 9th European Conference, ECAL 2007, Lisbon, Portugal, September 10-14, 2007. Proceedings 9, 294–303. Springer.
  17. Schiefele, U. 1996. Motivation und Lernen mit Texten. Hogrefe Göttingen.
  18. Schmidhuber, J. 1991. Curious model-building control systems. In Proc. international joint conference on neural networks, 1458–1463.
  19. Schmidhuber, J. 2010. Formal theory of creativity, fun, and intrinsic motivation (1990–2010). IEEE transactions on autonomous mental development, 2(3): 230–247.
  20. Shapley, L. S. 1953. Stochastic games. Proceedings of the national academy of sciences, 39(10): 1095–1100.
  21. Qtran: Learning to factorize with transformation for cooperative multi-agent reinforcement learning. In International conference on machine learning, 5887–5896. PMLR.
  22. Multiagent cooperation and competition with deep reinforcement learning. PloS one, 12(4): e0172395.
  23. Yang, Q. 2023. Hierarchical Needs-driven Agent Learning Systems: From Deep Reinforcement Learning To Diverse Strategies. The 37th AAAI 2023 Conference on Artificial Intelligence and Robotics Bridge Program.
  24. Understanding the Application of Utility Theory in Robotics and Artificial Intelligence: A Survey. arXiv preprint arXiv:2306.09445.
  25. Self-reactive planning of multi-robots with dynamic task assignments. In 2019 International Symposium on Multi-Robot and Multi-Agent Systems (MRS), 89–91. IEEE.
  26. Hierarchical needs based self-adaptive framework for cooperative multi-robot system. In 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2991–2998. IEEE.
  27. Needs-driven heterogeneous multi-robot cooperation in rescue missions. In 2020 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), 252–259. IEEE.
  28. How can robots trust each other for better cooperation? a relative needs entropy based robot-robot trust assessment model. In 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2656–2663. IEEE.
  29. Game-theoretic utility tree for multi-robot cooperative pursuit strategy. In ISR Europe 2022; 54th International Symposium on Robotics, 1–7. VDE.
  30. A Hierarchical Game-Theoretic Decision-Making for Cooperative Multiagent Systems Under the Presence of Adversarial Agents. In Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing, SAC ’23, 773–782. New York, NY, USA: Association for Computing Machinery. ISBN 9781450395175.
  31. A Strategy-Oriented Bayesian Soft Actor-Critic Model. Procedia Computer Science, 220: 561–566.
  32. Bayesian Soft Actor-Critic: A Directed Acyclic Strategy Graph Based Deep Reinforcement Learning. the 39th ACM/SIGAPP Symposium on Applied Computing, SAC ’24.
  33. Quality assessment of MORL algorithms: A utility-based approach. In Benelearn 2015: proceedings of the 24th annual machine learning conference of Belgium and the Netherlands.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now