Sequential Pathfinder (SePar) for Multi-Agent Delivery

• Sequential Pathfinder (SePar) is a Transformer-based policy architecture designed for lifelong multi-agent pickup and delivery in warehouse environments.
• It leverages autoregressive sequence modeling with a Multi-Agent Transformer to reduce the exponential joint-action complexity to linear-time action generation.
• The method combines reinforcement learning with imitation learning to significantly improve throughput and collision avoidance in high-complexity layouts.

Searching arXiv for the SePar paper and closely related MAPF/MAPD baselines to ground the article in current literature. First, locating the primary paper "Sequence Pathfinder for Multi-Agent Pickup and Delivery in the Warehouse" and core comparative methods. Sequential Pathfinder (SePar) is a Transformer-based policy architecture for Multi-Agent Pickup and Delivery (MAPD) in warehouse-like environments, introduced by Zhao et al. as a sequence-modeling reformulation of lifelong Multi-Agent Path Finding (MAPF) (Zhao et al., 28 Sep 2025). In this formulation, a fleet of agents repeatedly receives fixed-location pickup and delivery requests and must execute collision-free paths under partial observability, while maintaining throughput in layouts characterized by long corridors, narrow pathways, and frequent single-file traffic. SePar addresses the coordination bottleneck by factorizing the joint policy autoregressively over agents and implementing that factorization with a Multi-Agent Transformer, with the central claim that sequence modeling preserves order-invariant optimality while reducing the effective decision complexity from an exponential joint-action space to linear-time action generation (Zhao et al., 28 Sep 2025).

1. Problem setting and formalization

SePar is defined on an undirected grid graph $\mathcal G=(V,E)$ of size $h\times w$ with obstacle set $O\subset V$, free nodes $V\setminus O$, and $n$ agents (Zhao et al., 28 Sep 2025). At time $t$, each agent $i$ observes $o^i_t$ from its $m\times m$ field of view, so the joint observation is $\bm o_t=(o^1_t,\dots,o^n_t)$. The action space is discrete,

$h\times w$0

and a joint action $h\times w$1 induces a transition $h\times w$2.

The MAPD problem is cast as a Dec-POMDP,

$h\times w$3

where each agent’s reward $h\times w$4 penalizes collisions and step-costs ($h\times w$5), and awards $h\times w$6 for pickups and $h\times w$7 for deliveries (Zhao et al., 28 Sep 2025). The setting is lifelong MAPF: when an agent arrives at its current goal $h\times w$8, a new request $h\times w$9 is assigned and the agent must immediately proceed.

A distinctive element in the paper is the Path-Finding Complexity Index (PFCI),

$O\subset V$0

Here $O\subset V$1 captures typical corridor length, and $O\subset V$2 measures “narrowness” (Zhao et al., 28 Sep 2025). In the reported experiments, high-PFCI warehouse maps serve as the regime in which purely local decentralized policies tend to suffer from deadlocks and gridlocks, while classical planners incur high planning latency as $O\subset V$3 grows.

2. Sequence modeling and order-invariant optimality

The conceptual core of SePar is the recasting of MAPD as a sequence-modeling problem. For any permutation $O\subset V$4 of the agents, the policy is written autoregressively as

$O\subset V$5

Proposition 1, termed order-invariant optimality, states that for any two permutations $O\subset V$6, the induced joint distributions coincide:

$O\subset V$7

where $O\subset V$8 can be makespan or lifelong throughput (Zhao et al., 28 Sep 2025).

The key proof step is that the different autoregressive factorizations are merely re-orderings of the same underlying joint policy. Within the paper’s formalism, this is used to reject a common concern about autoregressive multi-agent control: that the imposed decoding order necessarily injects an intrinsic order-bias into the optimum. The claim is narrower than a blanket statement about all implementations; it establishes that an autoregressive decomposition does not lose optimality nor induce order-bias at the level of the modeled joint policy (Zhao et al., 28 Sep 2025).

This theoretical move is significant because it allows SePar to treat joint decision-making as sequential token prediction without abandoning global coordination. A plausible implication is that the architecture inherits the representational convenience of sequence models while remaining aligned with MAPD objectives such as throughput and collision avoidance.

3. Architecture and information flow

SePar combines an on-line feature extractor with a Multi-Agent Transformer (MAT) (Zhao et al., 28 Sep 2025). The observation feature extractor consists of two VGG-style CNN blocks, each with 3 convolutions and 1 max-pool, followed by one extra $O\subset V$9 convolution and then two fully connected layers. Temporal context is propagated by an LSTM carrying $V\setminus O$0, producing a per-agent embedding $V\setminus O$1.

The MAT has an encoder-decoder structure. The encoder applies self-attention over the sequence $V\setminus O$2 with no masking, yielding refined observations $V\setminus O$3 that encode high-level agent interactions. A tiny value-head MLP on each $V\setminus O$4 approximates $V\setminus O$5. The decoder is autoregressive and agent-by-agent: at step $V\setminus O$6 it attends to $V\setminus O$7 and to the past decoded actions $V\setminus O$8 via triangular-mask self-attention, ensuring no peeking ahead; a final MLP maps the decoder state to $V\setminus O$9 (Zhao et al., 28 Sep 2025).

The tokenization is explicit. Input tokens are the per-agent embeddings $n$0, and output tokens are learned embeddings for action symbols fed autoregressively. The encoder implements implicit communication: full self-attention makes every agent aware of every other agent’s embedding in $n$1, but in a parallelizable form. The decoder then conditions each agent’s action on all predecessor actions, propagating commitments through the action sequence.

The paper’s pseudocode sketch is:

1. collect $n$2 and extract $n$3 via CNN+LSTM;
2. encode $n$4;
3. for $n$5, decode $n$6;
4. execute $n$7 and observe reward and next observations (Zhao et al., 28 Sep 2025).

4. Training regime, imitation learning, and computational scaling

SePar is trained with a mixed reinforcement-learning and imitation-learning pipeline. The paper states that pure RL with PPO struggles in high-PFCI maps because of sparse successes and frequent deadlocks, so SePar interleaves on-policy PPO updates with off-policy behavioral cloning from a heuristic planner, LaCAM2 (Zhao et al., 28 Sep 2025). The behavioral-cloning loss is

$n$8

where $n$9 is the expert action.

Within the reported results, this mixed RL + IL pipeline provides high-quality demos in narrow corridors, dramatically accelerating convergence and raising final throughput by up to 50% (Zhao et al., 28 Sep 2025). The ablation evidence is also direct: SePar-Follower, which omits IL, fails to learn in high-PFCI layouts, and removing IL halves performance on warehouse layouts. This is the paper’s strongest empirical basis for the claim that imitation learning is not merely helpful but necessary in complex warehouse maps.

The complexity argument is one of the architecture’s defining features. A classical joint-action policy must consider $t$0 action combinations, which is exponential in $t$1. A decentralized approach runs $t$2 independent planners but cannot coordinate globally. By contrast, SePar’s autoregressive decoder attends to $t$3 tokens and, at each step, selects one action from $t$4 possibilities, for total complexity $t$5 per timestep. The encoder incurs $t$6 attention, but this is described as amortized via parallel GPU kernels, and the paper concludes that end-to-end inference scales linearly with $t$7 in wall-time (Zhao et al., 28 Sep 2025).

A second misconception addressed in the ablations concerns masked decoding. The paper reports that removing masked decoding destroys collision-avoidance guarantees. In SePar’s design, triangular masking is therefore not an implementation detail but a structural requirement for the autoregressive semantics of coordinated action generation.

5. Empirical evaluation and observed behavior

The evaluation spans both warehouse MAPD and POGEMA benchmarks (Zhao et al., 28 Sep 2025). In the Warehouse Simulator, the layouts are warehouse_small with PFCI $t$8, warehouse_large with PFCI $t$9, and three unseen huge maps with PFCI up to $i$0. The metric is throughput, defined as total deliveries divided by time. In POGEMA, the tasks include one-shot MAPF and lifelong MAPF on mazes with PFCI $i$1, using Success Rate, Sum of Costs (SoC), and Throughput.

On warehouse_small and warehouse_large with 32–512 agents, SePar surpasses PRIMAL2 and SCRIMP by $i$2–$i$3 throughput for $i$4 (Zhao et al., 28 Sep 2025). At $i$5, SePar retains $i$6–$i$7 of LaCAM2’s throughput, whereas the other learning-based methods collapse to $i$8–$i$9. The out-of-distribution results on warehouse_huge maps are more qualified: SePar achieves $o^i_t$0–$o^i_t$1 of LaCAM2’s throughput on two similar layouts, but performance drops on a structurally distinct third map.

On POGEMA mazes with 8–64 agents, SePar achieves a MAPF Success Rate exceeding DCC, MAMBA, and SCRIMP by $o^i_t$2–$o^i_t$3, and it has the lowest total path length for $o^i_t$4 in terms of SoC (Zhao et al., 28 Sep 2025). For lifelong MAPF throughput, it is second only to Follower and the planning solver RHCR. Taken together, these results position SePar as strongest in the regime where local observation is insufficient and where dense interaction structure makes explicit pairwise communication costly.

The reported empirical pattern is internally consistent with the paper’s training argument. In open-space benchmarks, pure RL is said to suffice; in high-PFCI warehouse layouts, IL becomes critical. This suggests that SePar’s advantage is not simply a generic benefit of Transformers, but the conjunction of sequence modeling, implicit all-to-all information exchange, masked autoregressive decoding, and expert-guided training.

6. Scope, limitations, and terminological ambiguity

The paper’s conclusions are explicitly bounded. SePar is presented as a globally informed, scalable MAPD solver whose inference cost grows linearly with $o^i_t$5 while preserving optimality guarantees, but its limitations include sensitivity to drastically out-of-distribution layouts and an $o^i_t$6 encoder in compute (Zhao et al., 28 Sep 2025). The stated future directions are sparse attention or graph-sparsification, along with extensions to dynamic disturbances such as obstacle appearances and communication dropouts.

A separate source of ambiguity is terminological rather than algorithmic. The acronym “SePar” is also used in the autonomous microscopy paper “PATHFINDER: Multi-objective discovery in structural and spectral spaces,” where “Sequential PATHFINDER (SePar)” denotes a sequential decision algorithm that combines a VAE latent representation, a deep-kernel Gaussian-process surrogate, and a Pareto-hypervolume-improvement acquisition rule for autonomous measurement selection (Barakati et al., 5 Apr 2026). That use concerns multi-objective discovery in structural and spectral spaces, including STEM-EELS and ferroelectric scanning probe microscopy, rather than MAPD in warehouses.

Accordingly, in current usage the label “Sequential Pathfinder (SePar)” is ambiguous across at least two technically unrelated contexts. In multi-agent robotics, it denotes Zhao et al.’s sequence-modeling architecture for lifelong MAPD (Zhao et al., 28 Sep 2025). In autonomous microscopy, it denotes the sequential acquisition algorithm embedded in the PATHFINDER framework (Barakati et al., 5 Apr 2026). The shared acronym does not imply methodological continuity between the two.