Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

RobocupGym: A challenging continuous control benchmark in Robocup (2407.14516v1)

Published 3 Jul 2024 in cs.RO and cs.LG

Abstract: Reinforcement learning (RL) has progressed substantially over the past decade, with much of this progress being driven by benchmarks. Many benchmarks are focused on video or board games, and a large number of robotics benchmarks lack diversity and real-world applicability. In this paper, we aim to simplify the process of applying reinforcement learning in the 3D simulation league of Robocup, a robotic football competition. To this end, we introduce a Robocup-based RL environment based on the open source rcssserver3d soccer server, simple pre-defined tasks, and integration with a popular RL library, Stable Baselines 3. Our environment enables the creation of high-dimensional continuous control tasks within a robotics football simulation. In each task, an RL agent controls a simulated Nao robot, and can interact with the ball or other agents. We open-source our environment and training code at https://github.com/Michael-Beukman/RobocupGym.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015a.
  2. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
  3. Mujoco: A physics engine for model-based control. In International Conference on Intelligent Robots and Systems, pages 5026–5033. IEEE, 2012. doi: 10.1109/IROS.2012.6386109. URL https://doi.org/10.1109/IROS.2012.6386109.
  4. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47:253–279, 2013.
  5. Human-level control through deep reinforcement learning. nature, 518(7540):529–533, 2015.
  6. Transfer learning in deep reinforcement learning: A survey. CoRR, abs/2009.07888, 2020. URL https://arxiv.org/abs/2009.07888.
  7. An empirical investigation of the challenges of real-world reinforcement learning. CoRR, abs/2003.11881, 2020. URL https://arxiv.org/abs/2003.11881.
  8. Deep reinforcement multi-directional kick-learning of a simulated robot with toes. In International Conference on Autonomous Robot Systems and Competitions, pages 104–110. IEEE, 2021.
  9. RMA: rapid motor adaptation for legged robots. In Dylan A. Shell, Marc Toussaint, and M. Ani Hsieh, editors, Robotics: Science and Systems XVII, Virtual Event, July 12-16, 2021, 2021. doi: 10.15607/RSS.2021.XVII.011. URL https://doi.org/10.15607/RSS.2021.XVII.011.
  10. Humanoid-gym: Reinforcement learning for humanoid robot with zero-shot sim2real transfer. arXiv preprint arXiv:2404.05695, 2024.
  11. Robocup: A challenge problem for ai. AI magazine, 18(1):73–73, 1997.
  12. BahiaRT Setplays Collecting Toolkit and BahiaRT Gym. Software Impacts, 14:100401, November 2022. ISSN 26659638. doi: 10.1016/j.simpa.2022.100401. URL https://linkinghub.elsevier.com/retrieve/pii/S2665963822000938.
  13. Designing a skilled soccer team for robocup: Exploring skill-set-primitives through reinforcement learning. 2023. URL https://doi.org/10.48550/arXiv.2312.14360.
  14. Simspark–concepts and application in the robocup 3d soccer simulation league. Autonomous Robots, 174:181, 2008.
  15. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22(268):1–8, 2021. URL http://jmlr.org/papers/v22/20-1364.html.
  16. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 1856–1865. PMLR, 2018. URL http://proceedings.mlr.press/v80/haarnoja18b.html.
  17. Openai gym. arXiv preprint arXiv:1606.01540, 2016.
  18. Boxes: An experiment in adaptive control. Machine intelligence, 2(2):137–152, 1968.
  19. Andrew William Moore. Efficient memory-based learning for robot control. Technical report, University of Cambridge, Computer Laboratory, 1990.
  20. Kenji Doya. Reinforcement learning in continuous time and space. Neural computation, 12(1):219–245, 2000.
  21. Swinging up the acrobot: An example of intelligent control. In Proceedings of 1994 American Control Conference-ACC’94, volume 2, pages 2158–2162. IEEE, 1994.
  22. Computer control of a double inverted pendulum. Computers & Electrical Engineering, 5(1):67–84, 1978.
  23. Rémi Coulom. Reinforcement learning using neural networks, with applications to motor control. PhD thesis, Institut National Polytechnique de Grenoble-INPG, 2002.
  24. Trust region policy optimization. In International conference on machine learning, pages 1889–1897. PMLR, 2015b.
  25. Guided policy search. In International conference on machine learning, pages 1–9. PMLR, 2013.
  26. Reinforcement learning: An introduction. MIT press, 2018.
  27. Deep learning for real-time atari game play using offline monte-carlo tree search planning. Advances in neural information processing systems, 27, 2014.
  28. Revisiting the arcade learning environment: Evaluation protocols and open problems for general agents. Journal of Artificial Intelligence Research, 61:523–562, 2018.
  29. Leveraging procedural generation to benchmark reinforcement learning. In International conference on machine learning, pages 2048–2056. PMLR, 2020.
  30. Leveraging procedural generation to benchmark reinforcement learning. arXiv preprint arXiv:1912.01588, 2019.
  31. Deepmind control suite. arXiv preprint arXiv:1801.00690, 2018.
  32. Safety gymnasium: A unified safe reinforcement learning benchmark. Advances in Neural Information Processing Systems, 36, 2023.
  33. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on robot learning, pages 1094–1100. PMLR, 2020.
  34. D4rl: Datasets for deep data-driven reinforcement learning. arXiv preprint arXiv:2004.07219, 2020.
  35. JAX: composable transformations of Python+NumPy programs, 2018. URL http://github.com/google/jax.
  36. Brax - a differentiable physics engine for large scale rigid body simulation, 2021. URL http://github.com/google/brax.
  37. Robert Tjarko Lange. gymnax: A JAX-based reinforcement learning environment library, 2022. URL http://github.com/RobertTLange/gymnax.
  38. XLand-minigrid: Scalable meta-reinforcement learning environments in JAX. In Intrinsically-Motivated and Open-Ended Learning Workshop, NeurIPS2023, 2023. URL https://openreview.net/forum?id=xALDC4aHGz.
  39. Jaxmarl: Multi-agent rl environments in jax. arXiv preprint arXiv:2311.10090, 2023.
  40. Craftax: A lightning-fast benchmark for open-ended reinforcement learning. arXiv preprint, 2024.
  41. On progress in robocup: the simulation league showcase. In RoboCup 2010: Robot Soccer World Cup XIV 14, pages 36–47. Springer, 2011.
  42. Ut austin villa 2011: a champion agent in the robocup 3d soccer simulation competition. In AAMAS, pages 129–136, 2012.
  43. Ut austin villa 2014: Robocup 3d simulation league champion via overlapping layered learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015.
  44. Ut austin villa: Robocup 2017 3d simulation league competition and technical challenges champions. In RoboCup 2017: Robot World Cup XXI 11, pages 473–485. Springer, 2018.
  45. Fc portugal: Robocup 2022 3d simulation league and technical challenge champions. In Robot World Cup, pages 313–324. Springer, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Michael Beukman (19 papers)
  2. Branden Ingram (2 papers)
  3. Geraud Nangue Tasse (10 papers)
  4. Benjamin Rosman (41 papers)
  5. Pravesh Ranchod (3 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - Michael-Beukman/RobocupGym (24 stars)