2000 character limit reached
Reinforcement Learning of Multi-robot Task Allocation for Multi-object Transportation with Infeasible Tasks
Published 18 Apr 2024 in cs.RO | (2404.11817v3)
Abstract: Multi-object transport using multi-robot systems has the potential for diverse practical applications such as delivery services owing to its efficient individual and scalable cooperative transport. However, allocating transportation tasks of objects with unknown weights remains challenging. Moreover, the presence of infeasible tasks (untransportable objects) can lead to robot stoppage (deadlock). This paper proposes a framework for dynamic task allocation that involves storing task experiences for each task in a scalable manner with respect to the number of robots. First, these experiences are broadcasted from the cloud server to the entire robot system. Subsequently, each robot learns the exclusion levels for each task based on those task experiences, enabling it to exclude infeasible tasks and reset its task priorities. Finally, individual transportation, cooperative transportation, and the temporary exclusion of tasks considered infeasible are achieved. The scalability and versatility of the proposed method were confirmed through numerical experiments with an increased number of robots and objects, including unlearned weight objects. The effectiveness of the temporary deadlock avoidance was also confirmed by introducing additional robots within an episode. The proposed method enables the implementation of task allocation strategies that are feasible for different numbers of robots and various transport tasks without prior consideration of feasibility.
- B. P. Gerkey and M. J. Matarić, “A formal analysis and taxonomy of task allocation in multi-robot systems,” The International journal of robotics research, vol. 23, no. 9, pp. 939–954, 2004.
- N. Lissandrini, C. K. Verginis, P. Roque, A. Cenedese, and D. V. Dimarogonas, “Decentralized nonlinear mpc for robust cooperative manipulation by heterogeneous aerial-ground robots,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 1531–1536.
- F. Bertoncelli, F. Ruggiero, and L. Sabattini, “Characterization of grasp configurations for multi-robot object pushing,” in 2021 International Symposium on Multi-Robot and Multi-Agent Systems (MRS). IEEE, 2021, pp. 38–46.
- Y. Liu, F. Zhang, P. Huang, and X. Zhang, “Analysis, planning and control for cooperative transportation of tethered multi-rotor uavs,” Aerospace Science and Technology, vol. 113, p. 106673, 2021.
- M. Doakhan, M. Kabganian, and A. Azimi, “Cooperative payload transportation with real-time formation control of multi-quadrotors in the presence of uncertainty,” Journal of the Franklin Institute, vol. 360, no. 2, pp. 1284–1307, 2023.
- H. Chakraa, F. Guérin, E. Leclercq, and D. Lefebvre, “Optimization techniques for multi-robot task allocation problems: Review on the state-of-the-art,” Robotics and Autonomous Systems, p. 104492, 2023.
- L. Liu and D. A. Shell, “Assessing optimal assignment under uncertainty: An interval-based algorithm,” The International Journal of Robotics Research, vol. 30, no. 7, pp. 936–953, 2011.
- L. Sabattini, V. Digani, C. Secchi, and C. Fantuzzi, “Optimized simultaneous conflict-free task assignment and path planning for multi-agv systems,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017, pp. 1083–1088.
- A. Kimmel and K. Bekris, “Decentralized multi-agent path selection using minimal information,” in Distributed Autonomous Robotic Systems, N.-Y. Chong and Y.-J. Cho, Eds. Tokyo: Springer Japan, 2016, pp. 341–356.
- M. B. Dias, R. Zlot, N. Kalra, and A. Stentz, “Market-based multirobot coordination: A survey and analysis,” Proceedings of the IEEE, vol. 94, no. 7, pp. 1257–1270, 2006.
- H.-L. Choi, L. Brunet, and J. P. How, “Consensus-based decentralized auctions for robust task allocation,” IEEE transactions on robotics, vol. 25, no. 4, pp. 912–926, 2009.
- M. Braquet and E. Bakolas, “Greedy decentralized auction-based task allocation for multi-agent systems,” IFAC-PapersOnLine, vol. 54, no. 20, pp. 675–680, 2021.
- Y. Yang and J. Wang, “An overview of multi-agent reinforcement learning from game theoretical perspective,” arXiv preprint arXiv:2011.00583, 2020.
- K. Shibata, T. Jimbo, T. Odashima, K. Takeshita, and T. Matsubara, “Learning locally, communicating globally: Reinforcement learning of multi-robot task allocation for cooperative transport,” IFAC-PapersOnLine, vol. 56, no. 2, pp. 11 436–11 443, 2023.
- M. Wen, J. Kuba, R. Lin, W. Zhang, Y. Wen, J. Wang, and Y. Yang, “Multi-agent reinforcement learning is a sequence modeling problem,” in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 2022, pp. 16 509–16 521.
- R. Lowe, Y. WU, A. Tamar, J. Harb, O. Pieter Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environments,” in Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30. Curran Associates, Inc., 2017.
- J. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson, “Counterfactual multi-agent policy gradients,” in Proceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018.
- C. D. Hsu, H. Jeong, G. J. Pappas, and P. Chaudhari, “Scalable reinforcement learning policies for multi-agent control,” 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4785–4791, 2020.
- T. S. Dahl, M. J. Mataric, and G. S. Sukhatme, “Adaptive spatio-temporal organization in groups of robots,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1. IEEE, 2002, pp. 1044–1049.
- G. Theraulaz, E. Bonabeau, and J. Denuebourg, “Response threshold reinforcements and division of labour in insect societies,” Proceedings of the Royal Society of London. Series B: Biological Sciences, vol. 265, no. 1393, pp. 327–332, 1998.
- M. J. Krieger and J.-B. Billeter, “The call of duty: Self-organised task allocation in a population of up to twelve mobile robots,” Robotics and Autonomous Systems, vol. 30, no. 1-2, pp. 65–84, 2000.
- C. J. Watkins and P. Dayan, “Q-learning,” Machine learning, vol. 8, pp. 279–292, 1992.
- A. M. Kwasnica, J. O. Ledyard, D. Porter, and C. DeMartini, “A new and improved design for multiobject iterative auctions,” Management science, vol. 51, no. 3, pp. 419–434, 2005.
- Y.-T. Tian, M. Yang, X.-Y. Qi, and Y.-M. Yang, “Multi-robot task allocation for fire-disaster response based on reinforcement learning,” in 2009 International Conference on Machine Learning and Cybernetics, vol. 4. IEEE, 2009, pp. 2312–2317.
- X. Zhao, Q. Zong, B. Tian, B. Zhang, and M. You, “Fast task allocation for heterogeneous unmanned aerial vehicles through reinforcement learning,” Aerospace Science and Technology, vol. 92, pp. 588–594, 2019.
- Y. Wang, H. Liu, W. Zheng, Y. Xia, Y. Li, P. Chen, K. Guo, and H. Xie, “Multi-objective workflow scheduling with deep-q-network-based multi-agent reinforcement learning,” IEEE access, vol. 7, pp. 39 974–39 982, 2019.
- H. Qie, D. Shi, T. Shen, X. Xu, Y. Li, and L. Wang, “Joint optimization of multi-uav target assignment and path planning based on multi-agent reinforcement learning,” IEEE access, vol. 7, pp. 146 264–146 272, 2019.
- H. Tang, A. Wang, F. Xue, J. Yang, and Y. Cao, “A novel hierarchical soft actor-critic algorithm for multi-logistics robots task allocation,” Ieee Access, vol. 9, pp. 42 568–42 582, 2021.
- T. Niwa, K. Shibata, and T. Jimbo, “Multi-agent reinforcement learning and individuality analysis for cooperative transportation with obstacle removal,” in Distributed Autonomous Robotic Systems: 15th International Symposium. Springer, 2022, pp. 202–213.
- R. Lowe, Y. WU, A. Tamar, J. Harb, O. Pieter Abbeel, and I. Mordatch, “Maddpg algorithm,” github. [Online]. Available:https://github.com/openai/maddpg [Accessed: 3-Nov-2021].
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
