Scalability and stability of multi‑agent reinforcement learning for multi‑robot interaction

Establish scalable and stable multi‑agent reinforcement learning algorithms and analysis for multi‑robot interaction that can handle large numbers of agents, guarantee convergence or robustness, and enable reliable real‑world deployment.

Background

Multi‑robot interaction problems rapidly increase in complexity as team size grows, and current MARL methods often struggle with scalability and stability, limiting real‑world applicability.

The survey identifies the scalability and stability of MARL as explicit open questions that hinder broader use in multi‑robot coordination, collision avoidance, and cooperative tasks.

References

The scalability and stability of MARL remain open questions that hinder RL's application for multi-robot interaction.

— Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes  (2408.03539 - Tang et al., 2024) in Improving Stability and Sample‑Efficiency in RL Algorithms (Section "General Trends and Open Challenges")

Several research questions remain open; we discuss two here. First, when a vanishing-AMCR path does converge, its limit is an MPE under state-space coverage, raising the question of when standard multi-agent reinforcement-learning algorithms possess such last-iterate guarantees. Existing results obtain convergence guarantees under additional game structure such as for policy-gradient methods in Markov potential games , but last-iterate results remain limited.

— Equilibrium in Multi-Agent Reinforcement Learning  (2608.22840 - D'Andrea et al., 24 Aug 2026) in Section Conclusions

Realizing this potential, however, requires robust autonomous swarm navigation, which remains an active research challenge.

— SwarmNxt: Open-source Software-Hardware Platform for Fast and Agile Aerial Swarms  (2609.11382 - Toumieh et al., 10 Sep 2026) in Abstract; Section 1, INTRODUCTION

This motivates distributed predefined-time IRL with a common prescribed deadline preserved under graph coupling, heterogeneity, partial measurements, delays, switching information structures, and network growth, together with explicit network-dependent constants.

— Predefined-Time Resilient Integral Reinforcement Learning for Input-Constrained Unknown Nonlinear Systems Under FDI Attacks and Disturbances: A Fully Data-Driven Approach  (2609.11815 - Vu et al., 10 Sep 2026) in Section Discussion, subsection Scalability