Learning Neural Traffic Rules (2312.01498v1)

Abstract: Extensive research has been devoted to the field of multi-agent navigation. Recently, there has been remarkable progress attributed to the emergence of learning-based techniques with substantially elevated intelligence and realism. Nonetheless, prevailing learned models face limitations in terms of scalability and effectiveness, primarily due to their agent-centric nature, i.e., the learned neural policy is individually deployed on each agent. Inspired by the efficiency observed in real-world traffic networks, we present an environment-centric navigation policy. Our method learns a set of traffic rules to coordinate a vast group of unintelligent agents that possess only basic collision-avoidance capabilities. Our method segments the environment into distinct blocks and parameterizes the traffic rule using a Graph Recurrent Neural Network (GRNN) over the block network. Each GRNN node is trained to modulate the velocities of agents as they traverse through. Using either Imitation Learning (IL) or Reinforcement Learning (RL) schemes, we demonstrate the efficacy of our neural traffic rules in resolving agent congestion, closely resembling real-world traffic regulations. Our method handles up to $240$ agents at real-time and generalizes across diverse agent and environment configurations.

• The paper introduces a decentralized, environment-centric navigation paradigm that employs a GRNN to learn traffic rules, enabling scalable, low-resource multi-agent coordination.
• It combines imitation and reinforcement learning to dynamically modulate agent velocities, ensuring collision avoidance and reduced congestion.
• The approach successfully coordinates up to 240 agents, demonstrating its potential for efficient autonomous robotics and smart traffic systems.

Introduction to Neural Traffic Rules

In the domain of robotics, the tasks concerned with guiding multiple agents or robots through environments without collisions are known as multi-agent navigation challenges. This task is crucial across a variety of industries, ranging from automated warehousing systems to the development of autonomous vehicles and the construction of smart cities.

Addressing Scalability for Agents with Limited Resources

When it comes to navigating multiple agents, a central issue is ensuring that each can compute and follow a complex policy individually. Typically, such policies are learned and require each agent to have the computational ability to execute deep neural network inferences. However, given that many robot systems operate with limited computational resources, executing these inferences can be impractical due to high costs and efficiency constraints.

Emulating Real-World Traffic Networks

This paper puts forward a novel environment-centric navigation policy that draws inspiration from the rule-based nature of real-world traffic systems. Unlike most current agent-centric approaches which require heavy computational resources, this work suggests learning predefined traffic rules at an environmental level. By applying a Graph Recurrent Neural Network (GRNN) over a segmented environment into blocks, the method focuses on learning and implementing traffic rules that can be followed by agents with minimal computational capabilities. This echoes the real-world scenario where, for example, drivers follow traffic rules that don't require complex individual decision-making and navigation plans.

Training Environment-Centric Navigation Policies

The proposed approach involves training the GRNN to modulate agents' velocities to ensure collision avoidance and minimize congestion, using a combination of Imitation Learning (IL) and Reinforcement Learning (RL). In cases where groundtruth traffic rules are known, IL can be used to mimic expert behavior. Alternatively, when such expert rules are not available, evolutionary RL is employed to allow the system to seek out and identify effective traffic rules to minimize congestion. Furthermore, the approach demonstrated the ability to coordinate up to 240 agents in simulated environments, proving both scalable and generalizable across different agent and environment configurations.

Conclusion

The key contributions of the paper are threefold: it presents a decentralized navigation paradigm using learned environment-encoded traffic rules, it encapsulates environment-centric navigation policies using GRNN, and it offers a new reward design and training algorithms for these policies within IL and RL settings. The outcome is a scalable and efficient multi-agent navigation method that promises reduced computational demands and negates the necessity for comprehensive inter-agent communication. This paper paves the way for future research directions such as real-world deployment and enhancements to account for time-dependent traffic rules and heterogeneous agent behaviors.