Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Aquarium: A Comprehensive Framework for Exploring Predator-Prey Dynamics through Multi-Agent Reinforcement Learning Algorithms (2401.07056v1)

Published 13 Jan 2024 in cs.AI and cs.MA

Abstract: Recent advances in Multi-Agent Reinforcement Learning have prompted the modeling of intricate interactions between agents in simulated environments. In particular, the predator-prey dynamics have captured substantial interest and various simulations been tailored to unique requirements. To prevent further time-intensive developments, we introduce Aquarium, a comprehensive Multi-Agent Reinforcement Learning environment for predator-prey interaction, enabling the study of emergent behavior. Aquarium is open source and offers a seamless integration of the PettingZoo framework, allowing a quick start with proven algorithm implementations. It features physics-based agent movement on a two-dimensional, edge-wrapping plane. The agent-environment interaction (observations, actions, rewards) and the environment settings (agent speed, prey reproduction, predator starvation, and others) are fully customizable. Besides a resource-efficient visualization, Aquarium supports to record video files, providing a visual comprehension of agent behavior. To demonstrate the environment's capabilities, we conduct preliminary studies which use PPO to train multiple prey agents to evade a predator. In accordance to the literature, we find Individual Learning to result in worse performance than Parameter Sharing, which significantly improves coordination and sample-efficiency.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (32)
  1. Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press.
  2. Alfred, L. (1925). Elements of Physical Biology. Nature, 116(2917):461–461.
  3. Implicit coordination for 3d underwater collective behaviors in a fish-inspired robot swarm. Science Robotics, 6(50):eabd8668.
  4. Is independent learning all you need in the starcraft multi-agent challenge? arxiv preprint, abs/2011.09533.
  5. Predator–Prey Models: A Review of Some Recent Advances. Mathematics, 9(15):1783.
  6. How to Manage Nature? Strategies, Predator-Prey Models, and Chaos. Marine Resource Economics, 12(2):127–143.
  7. Cooperative multi-agent control using deep reinforcement learning. In Autonomous Agents and Multiagent Systems, pages 66–83. Springer International.
  8. Learning to walk via deep reinforcement learning.
  9. Nash equilibria in multi-agent swarms. In ICAART: Proceedings of the 12th International Conference on Agents and Artificial Intelligence, pages 234–241.
  10. Emergent Escape-based Flocking Behavior using Multi-Agent Reinforcement Learning. In Proceedings of the 2019 Conference on Artificial Life, pages 598–605.
  11. Foraging swarms using multi-agent reinforcement learning. In Proceedings of the 2020 Conference on Artificial Life, pages 333–340.
  12. Deep reinforcement learning for swarm systems. Journal of Machine Learning Research, 20(54):1–31.
  13. Kuchkuda, R. (1988). An Introduction to Ray Tracing. Theoretical Foundations of Computer Graphics and CAD, pages 1039–1060.
  14. Predator–prey survival pressure is sufficient to evolve swarming behaviors. New Journal of Physics, 25(9):092001.
  15. Multi-agent actor-critic for mixed cooperative-competitive environments. In Advances in Neural Information Processing Systems (NeurIPS), volume 30, pages 6379–6390.
  16. Human-level control through deep reinforcement learning. Nature, 518(7540):529–533.
  17. Emergence of grounded compositional language in multi-agent populations. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence.
  18. Exploring the evolution of a trade-off between vigilance and foraging in group-living organisms. Royal Society Open Science, 2(9).
  19. Resilient multi-agent reinforcement learning with adversarial value decomposition. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 11308–11316.
  20. Reynolds, C. W. (1987). Flocks, herds and schools: A distributed behavioral model. ACM SIGGRAPH Computer Graphics, 21(4):25–34.
  21. Reynolds, C. W. (1999). Steering behaviors for autonomous characters. Game developers conference, pages 763–782.
  22. Towards Ecosystem Management from Greedy Reinforcement Learning in a Predator-Prey Setting. In Proceedings of the 2020 Conference on Artificial Life, pages 518–525.
  23. A Sustainable Ecosystem through Emergent Cooperation in Multi-Agent Reinforcement Learning. In Proceedings of the 2021 Conference on Artificial Life, pages 74–84.
  24. Proximal policy optimization algorithms. arXiv preprint, abs/1707.06347.
  25. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362(6419):1140–1144.
  26. Multiagent systems: A survey from a machine learning perspective. Autonomous Robots, 8(3):345–383.
  27. Reinforcement Learning: An Introduction. A Bradford Book, Cambridge, MA, USA.
  28. Pettingzoo: Gym for multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 34:15032–15043.
  29. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575(7782):350–354.
  30. Volterra, V. (1926). Fluctuations in the Abundance of a Species considered Mathematically1. Nature, 118(2972):558–560.
  31. A study of ai population dynamics with million-agent reinforcement learning. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS), page 2133–2135.
  32. An acceleration technique for 2d ray tracing simulation based on the research of diffraction in urban environment. In Proceedings of the 9th International Symposium on Antennas, Propagation and EM Theory, pages 493–496.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Michael Kölle (45 papers)
  2. Yannick Erpelding (1 paper)
  3. Fabian Ritz (18 papers)
  4. Thomy Phan (29 papers)
  5. Steffen Illium (17 papers)
  6. Claudia Linnhoff-Popien (105 papers)