Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Versatile Multi-Agent Reinforcement Learning Benchmark for Inventory Management (2306.07542v1)

Published 13 Jun 2023 in cs.AI

Abstract: Multi-agent reinforcement learning (MARL) models multiple agents that interact and learn within a shared environment. This paradigm is applicable to various industrial scenarios such as autonomous driving, quantitative trading, and inventory management. However, applying MARL to these real-world scenarios is impeded by many challenges such as scaling up, complex agent interactions, and non-stationary dynamics. To incentivize the research of MARL on these challenges, we develop MABIM (Multi-Agent Benchmark for Inventory Management) which is a multi-echelon, multi-commodity inventory management simulator that can generate versatile tasks with these different challenging properties. Based on MABIM, we evaluate the performance of classic operations research (OR) methods and popular MARL algorithms on these challenging tasks to highlight their weaknesses and potential.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Reinforcement Learning: An Introduction. MIT press, 2018.
  2. Mastering the game of go without human knowledge. Nature, 550(7676):354–359, 2017.
  3. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575(7782):350–354, 2019.
  4. Starcraft II: A new challenge for reinforcement learning. arXiv preprint arXiv:1708.04782, 2017.
  5. Starcraft micromanagement with reinforcement learning and curriculum transfer learning. IEEE Transactions on Emerging Topics in Computational Intelligence, 3(1):73–84, 2018.
  6. Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv:1912.06680, 2019.
  7. Towards playing full moba games with deep reinforcement learning. Advances in Neural Information Processing Systems, 33:621–632, 2020.
  8. Deep reinforcement learning approaches for process control. In 2017 6th international symposium on advanced control of industrial processes, pages 201–206. IEEE, 2017.
  9. A review on reinforcement learning: Introduction and applications in industrial process control. Computers & Chemical Engineering, 139:106886, 2020.
  10. Experience replay for real-time reinforcement learning control. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 42(2):201–212, 2011.
  11. Applications of reinforcement learning in energy systems. Renewable and Sustainable Energy Reviews, 137:110618, 2021.
  12. Multiadvisor reinforcement learning for multiagent multiobjective smart home energy control. IEEE Transactions on Artificial Intelligence, 3(4):581–594, 2021.
  13. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 2021.
  14. A decision-making method for autonomous vehicles based on simulation and reinforcement learning. In 2013 International Conference on Machine Learning and Cybernetics, volume 1, pages 362–369. IEEE, 2013.
  15. Decision-making method for vehicle longitudinal automatic driving based on reinforcement q-learning. International Journal of Advanced Robotic Systems, 16(3):1729881419853185, 2019.
  16. Modern perspectives on reinforcement learning in finance. Modern Perspectives on Reinforcement Learning in Finance. The Journal of Machine Learning in Finance, 1(1), 2020.
  17. Reinforcement learning in economics and finance. Computational Economics, pages 1–38, 2021.
  18. Deep reinforcement learning for optimizing finance portfolio management. In 2019 Amity International Conference on Artificial Intelligence, pages 14–20. IEEE, 2019.
  19. Generative adversarial user model for reinforcement learning based recommendation system. In International Conference on Machine Learning, pages 1052–1061. PMLR, 2019.
  20. A reinforcement learning approach to personalized learning recommendation systems. British Journal of Mathematical and Statistical Psychology, 72(1):108–135, 2019.
  21. Cooperative multi-agent control using deep reinforcement learning. In Autonomous Agents and Multiagent Systems: AAMAS 2017 Workshops, pages 66–83. Springer, 2017.
  22. CityLearn v1.0: An OpenAI gym environment for demand response with deep reinforcement learning. In Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, pages 356–357, 2019.
  23. Daniel Krajzewicz. Traffic simulation with sumo–simulation of urban mobility. Fundamentals of traffic simulation, pages 269–293, 2010.
  24. Qlib: An AI-oriented quantitative investment platform. arXiv preprint arXiv:2009.11189, 2020.
  25. Recogym: A reinforcement learning environment for the problem of product recommendation in online advertising. arXiv preprint arXiv:1808.00720, 2018.
  26. Multi-agent reinforcement learning: An overview. Innovations in multi-agent systems and applications-1, pages 183–221, 2010.
  27. A survey and critique of multiagent deep reinforcement learning. Autonomous Agents and Multi-Agent Systems, 33(6):750–797, 2019.
  28. Mean field multi-agent reinforcement learning. In International conference on machine learning, pages 5571–5580. PMLR, 2018.
  29. Learning to communicate with deep multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 29, 2016.
  30. Multi-agent reinforcement learning for networked system control. arXiv preprint arXiv:2004.01339, 2020.
  31. Robust adversarial reinforcement learning. In International Conference on Machine Learning, pages 2817–2826. PMLR, 2017.
  32. OpenAI Gym. arXiv preprint arXiv:1606.01540, 2016.
  33. Discovering and removing exogenous state variables and rewards for reinforcement learning. In International Conference on Machine Learning, pages 1262–1270. PMLR, 2018.
  34. Towards generalizable reinforcement learning for trade execution. In IJCAI, 2023.
  35. The starcraft multi-agent challenge. arXiv preprint arXiv:1902.04043, 2019.
  36. Google research football: A novel reinforcement learning environment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 4501–4510, 2020.
  37. Gobigger: A scalable platform for cooperative-competitive multi-agent interactive simulation. In The Eleventh International Conference on Learning Representations, 2023.
  38. Emergence of grounded compositional language in multi-agent populations. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  39. Facmac: Factored multi-agent centralised policy gradients. Advances in Neural Information Processing Systems, 34:12208–12221, 2021.
  40. Collective intelligence for deep learning: A survey of recent developments. Collective Intelligence, 1(1):26339137221114874, 2022.
  41. Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations. arXiv preprint arXiv:2107.14483, 2021.
  42. Or-gym: A reinforcement learning library for operations research problems. arXiv preprint arXiv:2008.06319, 2020.
  43. Marlim: Multi-agent reinforcement learning for inventory management. Advances in Neural Information Processing RL4RealLife Workshop, 2022.
  44. Simulation of inventory management systems in retail stores: A case study. Materials Today: Proceedings, 47:5130–5134, 2021.
  45. Quantitative models for supply chain management, volume 17. Springer Science & Business Media, 2012.
  46. John W Toomey. Inventory management: principles, concepts and techniques, volume 12. Springer Science & Business Media, 2000.
  47. Hartmut Stadtler. Supply chain management: An overview. Supply chain management and advanced planning: Concepts, models, software, and case studies, pages 3–28, 2014.
  48. Optimal inventory policy. Econometrica: Journal of the Econometric Society, pages 250–272, 1951.
  49. Alan S Blinder. Inventory theory and consumer behavior. Harvester Wheatsheaf, 1990.
  50. M-A Dittrich and S Fohlmeister. A deep q-learning-based optimization of the inventory control in a linear process chain. Production Engineering, 15:35–43, 2021.
  51. Monotonic value function factorisation for deep multi-agent reinforcement learning. The Journal of Machine Learning Research, 21(1):7234–7284, 2020.
  52. Qtran: Learning to factorize with transformation for cooperative multi-agent reinforcement learning. In International conference on machine learning, pages 5887–5896. PMLR, 2019.
  53. A deep reinforcement learning approach for inventory management in retail. Industrial Management & Data Systems, 2020.
  54. Multi-agent reinforcement learning with shared resources for inventory management. In Advances in Neural Information Processing RL4RealLife Workshop, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Xianliang Yang (4 papers)
  2. Zhihao Liu (28 papers)
  3. Wei Jiang (343 papers)
  4. Chuheng Zhang (24 papers)
  5. Li Zhao (150 papers)
  6. Lei Song (60 papers)
  7. Jiang Bian (229 papers)
Citations (7)
View on Semantic Scholar