CAMAR: Continuous Actions Multi-Agent Routing Benchmark
- CAMAR is a benchmark for continuous multi-agent routing that evaluates coordination using continuous state, action, and physics-based dynamics.
- It supports cooperative and competitive interactions with heterogeneous agents across diverse, procedurally generated maps.
- CAMAR integrates classical planning methods like RRT and RRT* with MARL algorithms to test policy optimization and planning integration at high simulation speeds.
CAMAR, short for Continuous Actions Multi-Agent Routing, is a benchmark for multi-agent reinforcement learning in multi-agent pathfinding settings with continuous state and action spaces. It is designed for environments that require coordination and planning under physically plausible motion, supports both cooperative and competitive interactions, and is implemented for high-throughput simulation at up to 100,000 environment steps per second. The benchmark also formalizes a three-tier evaluation protocol, includes RRT and RRT* as standalone and hybrid planning components, and provides test scenarios and benchmarking tools intended to support reproducibility and fair comparison (Pshenitsyn et al., 18 Aug 2025).
1. Benchmark scope and research role
CAMAR targets a specific gap in MARL benchmarking: many established environments emphasize discrete grids or discrete action abstractions, whereas CAMAR is organized around continuous actions for multi-agent pathfinding. Its stated motivation is to bridge the gap between grid or discrete simulated MARL environments and settings that emphasize smooth motion, realistic collision avoidance, and large-scale simulation. In that sense, CAMAR is positioned as a benchmark for studying coordination under continuous control rather than as a narrowly specialized path planner (Pshenitsyn et al., 18 Aug 2025).
The benchmark is also framed as a platform for comparing three methodological families within a common environment definition. The first family is standard MARL algorithms, including IPPO, MAPPO, MADDPG, ISAC, and MASAC. The second is classical planning, represented by RRT and RRT*. The third is hybrid systems that inject planning outputs into learned policies. This combination makes CAMAR relevant not only to benchmark design, but also to the broader question of how planning priors and policy optimization interact in continuous multi-agent routing.
A plausible implication is that CAMAR is intended to serve simultaneously as a stress test for coordination, a generalization benchmark, and an integration test for planning-learning pipelines. That interpretation follows from the presence of procedural map generation, heterogeneous agents, a multi-tier evaluation scheme, and explicit support for hybrid baselines.
2. Continuous state, action, and interaction model
CAMAR uses true continuous 2D state and action spaces. Two dynamics models are described. Under holonomic dynamics, each agent applies a continuous 2D force and evolves according to semi-implicit Euler integration, with damping and speed clamping. Under DiffDrive dynamics, agents follow a differential-drive model with heading, clipped linear and angular controls, and corresponding position and orientation updates (Pshenitsyn et al., 18 Aug 2025).
A notable feature of the environment model is its treatment of collisions. Instead of hard contacts, CAMAR uses smooth, differentiable forces for collision handling. The collision term depends on the inter-agent distance and a minimum allowed distance determined by the sum of radii. This makes the environment closer to a differentiable continuous-control simulator than to a discrete occupancy-based MAPF abstraction. The use of customizable radii also allows agents and obstacles to represent different footprints.
Observations are local and continuous. Each agent receives LIDAR-inspired but differentiable and vectorized local observations, including penetration vectors to nearby agents and obstacles together with a normalized vector to its goal. This observation design is structurally important: it preserves locality while avoiding discretized ray-casting representations. The benchmark therefore combines local sensing, continuous motion, and collision-sensitive interaction in a single vectorized environment model (Pshenitsyn et al., 18 Aug 2025).
This organization distinguishes CAMAR from benchmarks in which continuous actions are layered on top of otherwise coarse interaction rules. Here, motion, collision response, and observations are jointly formulated in continuous space.
3. Scenario classes, heterogeneity, and map generation
CAMAR supports both cooperative tasks and competitive or heterogeneous tasks. In cooperative settings, multiple agents must reach separate goals while avoiding collisions and are often rewarded for collective success. In competitive or heterogeneous settings, agents may have different dynamics, different sizes, or conflicting objectives. The benchmark therefore permits experiments on homogeneous coordination as well as on mixed-team control (Pshenitsyn et al., 18 Aug 2025).
Map diversity is a central part of the benchmark design. CAMAR includes procedural generators for random grids, mazes (LabMaze), caves (Perlin noise), textual layouts (string grid), and MovingAI maps. These are mapped into continuous space through circle-based discretization. The resulting environment family spans narrow passages, cluttered layouts, and map structures inspired by real-world navigation corpora. Because map structures can be batched and varied during training and evaluation, CAMAR is not restricted to a fixed set of static arenas.
The benchmark also supports heterogeneous agents with different dynamics and sizes. This matters for two reasons. First, it expands the pathfinding problem beyond identical-particle routing. Second, it allows the same benchmark to be used for questions about specialization, parameter sharing, and team composition. The paper reports that learning specialized policies for heterogeneous agents can outperform parameter-shared approaches, which indicates that heterogeneity in CAMAR is not merely representational but behaviorally consequential (Pshenitsyn et al., 18 Aug 2025).
This suggests that CAMAR treats multi-agent routing as a family of control problems rather than a single benchmark task. Cooperative pathfinding, competitive interaction, heterogeneous embodiment, and procedural geometry are all native to the environment.
4. Evaluation protocol and reported metrics
CAMAR introduces a three-level evaluation protocol intended to track algorithmic progress and analyze generalization under increasing distribution shift. The protocol is summarized below (Pshenitsyn et al., 18 Aug 2025).
| Tier | Train/test difference | What it tests |
|---|---|---|
| Easy | Same map type and agent count; test seeds randomize start/goal | Robustness on a fixed scenario |
| Medium | Train on one map type; test on 12 variants | Generalization across team sizes and local map variations |
| Hard | Train on any maps except some withheld benchmarks; test on fully unseen map types and different agent counts | Generalization to novel map structures and team scales |
The reported metrics are explicitly multi-faceted. Success Rate (SR) measures the fraction of agents that reach their goals. Flowtime (FT) is the mean per-agent time-to-goal. Makespan (MS) is the maximum time any agent takes. Coordination (CO) is defined as . Aggregate reporting uses IQM and 95\% confidence intervals. The benchmark also uses thousands of evaluation episodes with fixed seeds for reproducibility (Pshenitsyn et al., 18 Aug 2025).
The protocol is significant because it prevents performance from being reduced to a single episodic return. SR, FT, MS, and CO separate completion, efficiency, latency bottlenecks, and collision-sensitive coordination. Easy, Medium, and Hard then probe different failure modes: seed robustness, local generalization, and transfer to withheld map families. This makes CAMAR useful for diagnosing where a method succeeds rather than only whether it achieves a high score.
A common misunderstanding would be to treat CAMAR as a throughput-oriented simulator with standard RL evaluation. That is incomplete. The three-tier protocol and metric suite indicate that the benchmark is equally concerned with reproducibility and structured comparison.
5. Classical planning baselines and hybrid learning pipelines
A distinctive feature of CAMAR is its explicit integration of classical planning methods such as RRT and RRT* into MARL pipelines. These planners are used both as standalone baselines and as components in hybrid approaches (Pshenitsyn et al., 18 Aug 2025).
As standalone baselines, RRT+PD and RRT*+PD assign each agent a path produced by a single-robot planner and then execute that path with a simple proportional-derivative controller. These baselines are non-learnable and serve as strong references for what can be achieved by direct planning without policy optimization.
Hybridization is implemented in two forms. In policy guidance, each agent receives a path and cost-to-go sample from RRT* at the start of an episode; these are exposed as additional observation features such as waypoints and estimated costs. In learning integration, the RRT*-derived features are concatenated with the agent observation before entering the policy network. The intended effect is to let the policy combine local sensory input with a global planning hint and potentially adapt the planner’s suggestions to dynamic or partially observed conditions (Pshenitsyn et al., 18 Aug 2025).
This design has methodological importance beyond baseline construction. It operationalizes a specific hypothesis: global geometric planning and local reactive control need not be competing paradigms in continuous MARL. CAMAR turns that hypothesis into a benchmarkable interface by exposing planner outputs through an observation wrapper compatible with on-policy and off-policy algorithms.
Another misconception is that CAMAR evaluates only end-to-end learned coordination. In fact, the benchmark is explicitly structured so that planning-only, learning-only, and planning-plus-learning systems can be compared within the same task distribution.
6. Empirical findings, scalability, and benchmark significance
The reported experiments identify MAPPO as the best overall performer among the MARL baselines, particularly in Success Rate and path efficiency. The hybrid RRT*+MAPPO configuration often improves over pure MARL, especially on Flowtime and Makespan, which indicates that planning hints can improve route efficiency even when a learned controller is already competent (Pshenitsyn et al., 18 Aug 2025).
The behavior of non-learnable baselines is more nuanced. RRT*+PD can achieve high Success Rate, but it struggles with Coordination in crowded or narrow maps because it lacks explicit multi-agent awareness. By contrast, off-policy hybrids such as RRT*+MADDPG and RRT*+MASAC show mixed results; the paper attributes some of this difficulty to the enlarged observation vector, which can make centralized critics unstable. These findings are relevant because they separate the value of planning priors from the optimization stability of the downstream MARL algorithm.
On scalability, CAMAR is reported to maintain more than 100,000 steps per second with 16 agents and 2,000 parallel environments on a modern GPU, and to remain over an order of magnitude faster than VMAS and other continuous-space MARL testbeds as agent and obstacle counts increase (Pshenitsyn et al., 18 Aug 2025). That throughput is not merely an implementation detail. In MARL, benchmark feasibility is strongly constrained by sample requirements, parallel rollout capacity, and the cost of multi-scenario evaluation. CAMAR’s performance profile is therefore part of its scientific contribution.
The benchmark is also integrated with Gymnax/JAX, TorchRL, and BenchMARL, and exposes a standardized API. Combined with its test scenarios, fixed-seed evaluation, and metric definitions, this makes CAMAR a reproducibility-oriented benchmark rather than a simulator without protocol.
From a broader perspective, CAMAR defines multi-agent routing as a continuous-control benchmark in which geometry, coordination, classical planning, and generalization are all first-class concerns. This suggests a shift away from evaluating MARL only on stylized discrete environments and toward evaluating whether algorithms can coordinate under continuous dynamics, local sensing, heterogeneous embodiment, and structured distribution shift (Pshenitsyn et al., 18 Aug 2025).
