Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GCBF+: A Neural Graph Control Barrier Function Framework for Distributed Safe Multi-Agent Control (2401.14554v1)

Published 25 Jan 2024 in cs.RO and math.OC

Abstract: Distributed, scalable, and safe control of large-scale multi-agent systems (MAS) is a challenging problem. In this paper, we design a distributed framework for safe multi-agent control in large-scale environments with obstacles, where a large number of agents are required to maintain safety using only local information and reach their goal locations. We introduce a new class of certificates, termed graph control barrier function (GCBF), which are based on the well-established control barrier function (CBF) theory for safety guarantees and utilize a graph structure for scalable and generalizable distributed control of MAS. We develop a novel theoretical framework to prove the safety of an arbitrary-sized MAS with a single GCBF. We propose a new training framework GCBF+ that uses graph neural networks (GNNs) to parameterize a candidate GCBF and a distributed control policy. The proposed framework is distributed and is capable of directly taking point clouds from LiDAR, instead of actual state information, for real-world robotic applications. We illustrate the efficacy of the proposed method through various hardware experiments on a swarm of drones with objectives ranging from exchanging positions to docking on a moving target without collision. Additionally, we perform extensive numerical experiments, where the number and density of agents, as well as the number of obstacles, increase. Empirical results show that in complex environments with nonlinear agents (e.g., Crazyflie drones) GCBF+ outperforms the handcrafted CBF-based method with the best performance by up to 20% for relatively small-scale MAS for up to 256 agents, and leading reinforcement learning (RL) methods by up to 40% for MAS with 1024 agents. Furthermore, the proposed method does not compromise on the performance, in terms of goal reaching, for achieving high safety rates, which is a common trade-off in RL-based methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (80)
  1. A. Dorri, S. S. Kanhere, and R. Jurdak, “Multi-agent systems: A survey,” IEEE Access, vol. 6, pp. 28 573–28 593, 2018.
  2. B. Li and H. Ma, “Double-deck multi-agent pickup and delivery: Multi-robot rearrangement in large-scale warehouses,” IEEE Robotics and Automation Letters, vol. 8, no. 6, pp. 3701–3708, 2023.
  3. A. Kattepur, H. K. Rath, A. Simha, and A. Mukherjee, “Distributed optimization in multi-agent robotics for industry 4.0 warehouses,” in Proceedings of the 33rd Annual ACM Symposium on Applied Computing, 2018, pp. 808–815.
  4. L. M. Schmidt, J. Brosig, A. Plinge, B. M. Eskofier, and C. Mutschler, “An introduction to multi-agent reinforcement learning and review of its application to autonomous mobility,” in 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC).   IEEE, 2022, pp. 1342–1349.
  5. P. Palanisamy, “Multi-agent connected autonomous driving using deep reinforcement learning,” in 2020 International Joint Conference on Neural Networks (IJCNN).   IEEE, 2020, pp. 1–7.
  6. M. Zhou, J. Luo, J. Villella, Y. Yang, D. Rusu, J. Miao, W. Zhang, M. Alban, I. Fadakar, Z. Chen et al., “Smarts: An open-source scalable multi-agent rl training school for autonomous driving,” in Conference on Robot Learning.   PMLR, 2021, pp. 264–285.
  7. S. Zhang, Y. Xiu, G. Qu, and C. Fan, “Compositional neural certificates for networked dynamical systems,” in Learning for Dynamics and Control Conference.   PMLR, 2023, pp. 272–285.
  8. Y. Tian, K. Liu, K. Ok, L. Tran, D. Allen, N. Roy, and J. P. How, “Search and rescue under the forest canopy using multiple uavs,” The International Journal of Robotics Research, vol. 39, no. 10-11, pp. 1201–1221, 2020.
  9. K. A. Ghamry, M. A. Kamel, and Y. Zhang, “Multiple uavs in forest fire fighting mission using particle swarm optimization,” in 2017 International Conference on Unmanned Aircraft Systems (ICUAS).   IEEE, 2017, pp. 1404–1409.
  10. C. Ju, J. Kim, J. Seol, and H. I. Son, “A review on multirobot systems in agriculture,” Computers and Electronics in Agriculture, vol. 202, p. 107336, 2022.
  11. J. Chen, J. Li, C. Fan, and B. C. Williams, “Scalable and safe multi-agent motion planning with nonlinear dynamics and bounded disturbances,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, 2021, pp. 11 237–11 245.
  12. R. J. Afonso, M. R. Maximo, and R. K. Galvão, “Task allocation and trajectory planning for multiple agents in the presence of obstacle and connectivity constraints with mixed-integer linear programming,” International Journal of Robust and Nonlinear Control, vol. 30, no. 14, pp. 5464–5491, 2020.
  13. J. Netter, G. P. Kontoudis, and K. G. Vamvoudakis, “Bounded rational rrt-qx: Multi-agent motion planning in dynamic human-like environments using cognitive hierarchy and q-learning,” in 2021 60th IEEE Conference on Decision and Control (CDC).   IEEE, 2021, pp. 3597–3602.
  14. A. D. Saravanos, Y. Aoyama, H. Zhu, and E. A. Theodorou, “Distributed differential dynamic programming architectures for large-scale multiagent control,” IEEE Transactions on Robotics, vol. 39, no. 6, pp. 4387–4407, 2023.
  15. K. Garg, S. Zhang, O. So, C. Dawson, and C. Fan, “Learning safe control for multi-robot systems: Methods, verification, and open challenges,” 2023, arXiv:2311.13714.
  16. C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. Wu, “The surprising effectiveness of ppo in cooperative multi-agent games,” Advances in Neural Information Processing Systems, vol. 35, pp. 24 611–24 624, 2022.
  17. A. D. Ames, S. Coogan, M. Egerstedt, G. Notomista, K. Sreenath, and P. Tabuada, “Control barrier functions: Theory and applications,” in 2019 18th European Control Conference (ECC).   IEEE, 2019, pp. 3420–3431.
  18. A. D. Ames, X. Xu, J. W. Grizzle, and P. Tabuada, “Control barrier function based quadratic programs for safety critical systems,” IEEE Transactions on Automatic Control, vol. 62, no. 8, pp. 3861–3876, 2017.
  19. P. Glotfelter, J. Cortés, and M. Egerstedt, “Nonsmooth barrier functions with applications to multi-robot systems,” IEEE Control Systems Letters, vol. 1, no. 2, pp. 310–315, 2017.
  20. M. Jankovic and M. Santillo, “Collision avoidance and liveness of multi-agent systems with cbf-based controllers,” in 2021 60th IEEE Conference on Decision and Control (CDC).   IEEE, 2021, pp. 6822–6828.
  21. R. Cheng, M. J. Khojasteh, A. D. Ames, and J. W. Burdick, “Safe multi-agent interaction through robust control barrier functions with learned uncertainties,” in 2020 59th IEEE Conference on Decision and Control (CDC).   IEEE, 2020, pp. 777–783.
  22. K. Garg and D. Panagou, “Robust control barrier and control lyapunov functions with fixed-time convergence guarantees,” in 2021 American Control Conference (ACC).   IEEE, 2021, pp. 2292–2297.
  23. L. Wang, A. D. Ames, and M. Egerstedt, “Safety barrier certificates for collisions-free multirobot systems,” IEEE Transactions on Robotics, vol. 33, no. 3, pp. 661–674, 2017.
  24. D. R. Agrawal and D. Panagou, “Safe control synthesis via input constrained control barrier functions,” in 2021 60th IEEE Conference on Decision and Control (CDC).   IEEE, 2021, pp. 6113–6118.
  25. Y. Chen, M. Jankovic, M. Santillo, and A. D. Ames, “Backup control barrier functions: Formulation and comparative study,” in 2021 60th IEEE Conference on Decision and Control (CDC).   IEEE, 2021, pp. 6835–6841.
  26. J. Breeden and D. Panagou, “High relative degree control barrier functions under input constraints,” in 2021 60th IEEE Conference on Decision and Control (CDC).   IEEE, 2021, pp. 6119–6124.
  27. Y. Chen, A. Singletary, and A. D. Ames, “Guaranteed obstacle avoidance for multi-robot operations with limited actuation: A control barrier function approach,” IEEE Control Systems Letters, vol. 5, no. 1, pp. 127–132, 2020.
  28. S. Zhang, K. Garg, and C. Fan, “Neural graph control barrier functions guided distributed collision-avoidance multi-agent control,” in 7th Annual Conference on Robot Learning, 2023.
  29. S. Nayak, K. Choi, W. Ding, S. Dolan, K. Gopalakrishnan, and H. Balakrishnan, “Scalable multi-agent reinforcement learning through intelligent information aggregation,” in International Conference on Machine Learning.   PMLR, 2023, pp. 25 817–25 833.
  30. H. Ma, D. Harabor, P. J. Stuckey, J. Li, and S. Koenig, “Searching with consistent prioritization for multi-agent path finding,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp. 7643–7650.
  31. G. Sharon, R. Stern, A. Felner, and N. R. Sturtevant, “Conflict-based search for optimal multi-agent pathfinding,” Artificial Intelligence, vol. 219, pp. 40–66, 2015.
  32. S. H. Arul and D. Manocha, “V-rvo: Decentralized multi-agent collision avoidance using voronoi diagrams and reciprocal velocity obstacles,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 8097–8104.
  33. L. Zheng, J. Yang, H. Cai, M. Zhou, W. Zhang, J. Wang, and Y. Yu, “Magent: A many-agent reinforcement learning platform for artificial collective intelligence,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018.
  34. S. Prajna, A. Papachristodoulou, and P. A. Parrilo, “Introducing sostools: A general purpose sum of squares programming solver,” in Proceedings of the 41st IEEE Conference on Decision and Control, 2002., vol. 1.   IEEE, 2002, pp. 741–746.
  35. X. Xu, J. W. Grizzle, P. Tabuada, and A. D. Ames, “Correctness guarantees for the composition of lane keeping and adaptive cruise control,” IEEE Transactions on Automation Science and Engineering, vol. 15, no. 3, pp. 1216–1229, 2017.
  36. M. Srinivasan, M. Abate, G. Nilsson, and S. Coogan, “Extent-compatible control barrier functions,” Systems & Control Letters, vol. 150, p. 104895, 2021.
  37. P. Zhao, R. Ghabcheloo, Y. Cheng, H. Abdi, and N. Hovakimyan, “Convex synthesis of control barrier functions under input constraints,” IEEE Control Systems Letters, 2023.
  38. A. A. Ahmadi and A. Majumdar, “Some applications of polynomial optimization in operations research and real-time decision making,” Optimization Letters, vol. 10, pp. 709–729, 2016.
  39. Z. Cai, H. Cao, W. Lu, L. Zhang, and H. Xiong, “Safe multi-agent reinforcement learning through decentralized multiple control barrier functions,” arXiv preprint arXiv:2103.12553, 2021.
  40. Z. Qin, K. Zhang, Y. Chen, J. Chen, and C. Fan, “Learning safe multi-agent control with decentralized neural barrier certificates,” in International Conference on Learning Representations, 2021. [Online]. Available: https://openreview.net/forum?id=P6_q1BRxY8Q
  41. C. Dawson, S. Gao, and C. Fan, “Safe control with learned certificates: A survey of neural lyapunov, barrier, and contraction methods for robotics and control,” IEEE Transactions on Robotics, 2023.
  42. C. Dawson, Z. Qin, S. Gao, and C. Fan, “Safe nonlinear control using robust neural lyapunov-barrier functions,” in Conference on Robot Learning.   PMLR, 2022, pp. 1724–1735.
  43. Z. Qin, D. Sun, and C. Fan, “Sablas: Learning safe control for black-box dynamical systems,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 1928–1935, 2022.
  44. W. Zhao, T. He, and C. Liu, “Model-free safe control for zero-violation reinforcement learning,” in 5th Annual Conference on Robot Learning, 2021.
  45. O. So, Z. Serlin, M. Mann, J. Gonzales, K. Rutledge, N. Roy, and C. Fan, “How to train your neural control barrier function: Learning safety filters for complex input-constrained systems,” arXiv preprint arXiv:2310.15478, 2023.
  46. Y. Meng, Z. Qin, and C. Fan, “Reactive and safe road user simulations using neural barrier certificates,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 6299–6306.
  47. J. Dinneweth, A. Boubezoul, R. Mandiau, and S. Espié, “Multi-agent reinforcement learning for autonomous vehicles: a survey,” Autonomous Intelligent Systems, vol. 2, no. 1, p. 27, 2022.
  48. W. Zhang, O. Bastani, and V. Kumar, “Mamps: Safe multi-agent reinforcement learning via model predictive shielding,” arXiv preprint arXiv:1910.12639, 2019.
  49. H. Qie, D. Shi, T. Shen, X. Xu, Y. Li, and L. Wang, “Joint optimization of multi-uav target assignment and path planning based on multi-agent reinforcement learning,” IEEE access, vol. 7, pp. 146 264–146 272, 2019.
  50. M. Everett, Y. F. Chen, and J. P. How, “Motion planning among dynamic, decision-making agents with deep reinforcement learning,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2018, pp. 3052–3059.
  51. X. Xiao, B. Liu, G. Warnell, and P. Stone, “Motion planning and control for mobile robot navigation using machine learning: a survey,” Autonomous Robots, vol. 46, no. 5, pp. 569–597, 2022.
  52. Z. Dai, T. Zhou, K. Shao, D. H. Mguni, B. Wang, and H. Jianye, “Socially-attentive policy optimization in multi-agent self-driving system,” in Conference on Robot Learning.   PMLR, 2023, pp. 946–955.
  53. X. Pan, M. Liu, F. Zhong, Y. Yang, S.-C. Zhu, and Y. Wang, “Mate: Benchmarking multi-agent reinforcement learning in distributed target coverage control,” Advances in Neural Information Processing Systems, vol. 35, pp. 27 862–27 879, 2022.
  54. B. Wang, J. Xie, and N. Atanasov, “Darl1n: Distributed multi-agent reinforcement learning with one-hop neighbors,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2022, pp. 9003–9010.
  55. Y. Wang, M. Damani, P. Wang, Y. Cao, and G. Sartoretti, “Distributed reinforcement learning for robot teams: a review,” Current Robotics Reports, vol. 3, no. 4, pp. 239–257, 2022.
  56. C. Yu, H. Yu, and S. Gao, “Learning control admissibility models with graph neural networks for multi-agent navigation,” in Conference on Robot Learning.   PMLR, 2023, pp. 934–945.
  57. E. Tolstaya, J. Paulos, V. Kumar, and A. Ribeiro, “Multi-robot coverage and exploration using spatial graph neural networks,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 8944–8950.
  58. J. Blumenkamp, S. Morad, J. Gielis, Q. Li, and A. Prorok, “A framework for real-world multi-robot systems running decentralized gnn-based policies,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 8772–8778.
  59. X. Jia, L. Sun, H. Zhao, M. Tomizuka, and W. Zhan, “Multi-agent trajectory prediction by combining egocentric and allocentric views,” in Conference on Robot Learning.   PMLR, 2022, pp. 1434–1443.
  60. X. Ji, H. Li, Z. Pan, X. Gao, and C. Tu, “Decentralized, unlabeled multi-agent navigation in obstacle-rich environments using graph neural networks,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 8936–8943.
  61. C. Yu and S. Gao, “Reducing collision checking for sampling-based motion planning using graph neural networks,” Advances in Neural Information Processing Systems, vol. 34, pp. 4274–4289, 2021.
  62. Q. Li, F. Gama, A. Ribeiro, and A. Prorok, “Graph neural networks for decentralized multi-robot path planning,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2020, pp. 11 785–11 792.
  63. Y. Li, C. Gu, T. Dullien, O. Vinyals, and P. Kohli, “Graph matching networks for learning the similarity of graph structured objects,” in International Conference on Machine Learning.   PMLR, 2019, pp. 3835–3845.
  64. F. Blanchini, “Set invariance in control,” Automatica, vol. 35, no. 11, pp. 1747–1767, 1999.
  65. M. A. Pereira, A. D. Saravanos, O. So, and E. A. Theodorou, “Decentralized safe multi-agent stochastic optimal control using deep FBSDEs and ADMM,” in Robotics: Science and Systems, 2022.
  66. Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel, “Gated graph sequence neural networks,” arXiv preprint arXiv:1511.05493, 2015.
  67. I. M. Mitchell, “The flexible, extensible and efficient toolbox of level set methods,” Journal of Scientific Computing, vol. 35, pp. 300–329, 2008.
  68. K.-C. Hsu, V. Rubies-Royo, C. Tomlin, and J. F. Fisac, “Safety and Liveness Guarantees through Reach-Avoid Reinforcement Learning,” in Proceedings of Robotics: Science and Systems, Virtual, July 2021.
  69. J. F. Fisac, N. F. Lugovoy, V. Rubies-Royo, S. Ghosh, and C. J. Tomlin, “Bridging hamilton-jacobi safety analysis and reinforcement learning,” in 2019 International Conference on Robotics and Automation (ICRA).   IEEE, 2019, pp. 8550–8556.
  70. L. Schäfer, F. Gruber, and M. Althoff, “Scalable computation of robust control invariant sets of nonlinear systems,” IEEE Transactions on Automatic Control, 2023.
  71. C. Dawson, B. Lowenkamp, D. Goff, and C. Fan, “Learning safe, generalizable perception-based hybrid control with certificates,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 1904–1911, 2022.
  72. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  73. S. H. Semnani, H. Liu, M. Everett, A. De Ruiter, and J. P. How, “Multi-agent motion planning for dense and dynamic environments via deep reinforcement learning,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3221–3226, 2020.
  74. Y. F. Chen, M. Liu, M. Everett, and J. P. How, “Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning,” in 2017 IEEE International Conference on Robotics and Automation.   IEEE, 2017, pp. 285–292.
  75. Q. Nguyen and K. Sreenath, “Exponential control barrier functions for enforcing high relative-degree safety-critical constraints,” in 2016 American Control Conference (ACC).   IEEE, 2016, pp. 322–328.
  76. W. Xiao and C. Belta, “Control barrier functions for systems with high relative degree,” in 2019 IEEE 58th conference on decision and control (CDC).   IEEE, 2019, pp. 474–479.
  77. G. Katz, C. Barrett, D. L. Dill, K. Julian, and M. J. Kochenderfer, “Reluplex: An efficient smt solver for verifying deep neural networks,” in Computer Aided Verification: 29th International Conference, CAV 2017, Heidelberg, Germany, July 24-28, 2017, Proceedings, Part I 30.   Springer, 2017, pp. 97–117.
  78. L. Lindemann, H. Hu, A. Robey, H. Zhang, D. Dimarogonas, S. Tu, and N. Matni, “Learning hybrid control barrier functions from data,” in Conference on Robot Learning.   PMLR, 2021, pp. 1351–1370.
  79. H. K. Khalil, “Nonlinear systems third edition,” Patience Hall, vol. 115, 2002.
  80. C. Budaciu, N. Botezatu, M. Kloetzer, and A. Burlacu, “On the evaluation of the crazyflie modular quadcopter system,” in 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA).   IEEE, 2019, pp. 1189–1195.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Songyuan Zhang (10 papers)
  2. Oswin So (24 papers)
  3. Kunal Garg (37 papers)
  4. Chuchu Fan (81 papers)
Citations (11)

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com