Multi-Agent LLM Actor-Critic Framework for Social Robot Navigation (2503.09758v1)

Abstract: Recent advances in robotics and LLMs have sparked growing interest in human-robot collaboration and embodied intelligence. To enable the broader deployment of robots in human-populated environments, socially-aware robot navigation (SAN) has become a key research area. While deep reinforcement learning approaches that integrate human-robot interaction (HRI) with path planning have demonstrated strong benchmark performance, they often struggle to adapt to new scenarios and environments. LLMs offer a promising avenue for zero-shot navigation through commonsense inference. However, most existing LLM-based frameworks rely on centralized decision-making, lack robust verification mechanisms, and face inconsistencies in translating macro-actions into precise low-level control signals. To address these challenges, we propose SAMALM, a decentralized multi-agent LLM actor-critic framework for multi-robot social navigation. In this framework, a set of parallel LLM actors, each reflecting distinct robot personalities or configurations, directly generate control signals. These actions undergo a two-tier verification process via a global critic that evaluates group-level behaviors and individual critics that assess each robot's context. An entropy-based score fusion mechanism further enhances self-verification and re-query, improving both robustness and coordination. Experimental results confirm that SAMALM effectively balances local autonomy with global oversight, yielding socially compliant behaviors and strong adaptability across diverse multi-robot scenarios. More details and videos about this work are available at: https://sites.google.com/view/SAMALM.

• The paper introduces SAMALM, a decentralized multi-agent LLM actor-critic framework that enables socially-compliant navigation via dual-layer critic evaluation.
• It employs parallel LLM-actors with personalized world models to dynamically adapt low-level control actions based on both local context and global oversight.
• Evaluation in simulated scenarios demonstrates enhanced navigation success rates and social compliance in human-populated environments.

Multi-Agent LLM Actor-Critic Framework for Social Robot Navigation

Introduction

The "Multi-Agent LLM Actor-Critic Framework for Social Robot Navigation" paper addresses the challenges of deploying socially-aware robots in human-populated environments. The research introduces SAMALM, a decentralized multi-agent LLM actor-critic framework designed to enable socially-compliant multi-robot navigation. The paper recognizes the limitations of existing approaches, such as centralized decision-making and macro-action inconsistencies, while proposing a solution that integrates decentralized LLMs for better social interaction and adaptability in diverse scenarios.

Figure 1: SAMALM procedure in a multi-robot social navigation scenario: The decentralized decision-making system supports self-verification and re-query, consisting of multiple parallel LLM-actors, evaluated by both the individual LLM-critic and a global LLM-critic.

SAMALM Architecture

SAMALM employs a set of parallel LLM-actors that generate low-level control signals for distinct robot personalities and configurations. Each robot within this framework acts autonomously while submitting its decisions to a two-tier evaluation process involving global and local LLM-critics.

Figure 2: SAMALM architecture: A decentralized multi-agent LLM actor-critic framework designed for multi-robot social navigation.

The framework enhances coordination and robustness through this decentralized architecture, where a global critic provides overarching behavioral evaluations, and individual critics ensure context-based adjustments. This structure is aimed at bridging the gap between macro-action inference by LLMs and the precise control actions needed by robots.

Multi-Robot World Model

SAMALM introduces a multi-robot world model to represent the dynamic environment concerning both human-robot interactions (HRI) and robot-robot interactions (RRI). This model constructs personalized knowledge bases for each robot using spatio-temporal graphs derived from local observations.

Figure 3: An illustration of multi-robot world model construction.

These models enable robots to navigate through social spaces intelligently by capturing environmental dynamics effectively, thereby facilitating precise and reliable path planning.

Multi-Agent LLM Actor-Critic Framework

SAMALM's primary contribution lies in its actor-critic framework, wherein LLM-actors process world model observations to derive semantically correlated control actions. These actions undergo evaluation by an entropy-based fusion mechanism that integrates critic feedback to refine decision-making.

Figure 4: Multi-Agent LLM Actor-Critic Framework: Facilitates social navigation through parallel LLM actors working with global and local critics.

This approach ensures that robot actions are not only individually rational but also collectively cohesive, maintaining social norms across complex navigation scenarios.

Evaluation and Results

SAMALM was tested in simulated environments with various robot and human configurations. The results indicate a significant improvement in navigation success rates and social compliance scores compared to baseline models and ablation variants.

Figure 5: Illustration of multi-robot social navigation simulation scenario, where 3 robots navigate to distinct destinations across a crowd with 10 humans.

The framework's decentralized decision-making paradigm and its adaptation of personalized LLM actors contribute to more adaptive and socially-aware navigation behavior, handling the complexities of dynamic and populated environments effectively.

Conclusion

The research provides a robust framework that advances multi-robot social navigation by integrating LLMs with actor-critic methodologies. SAMALM's self-verification, re-query procedures, and decentralized architecture offer an adaptable, resilient, and socially-complaint navigation solution. These innovations present opportunities for deploying more adaptive robotic systems in dynamic human environments, opening avenues for future AI research in socially-aware robotics.