---
title: Sequential Pathfinder (SePar) for Multi-Agent Delivery
url: https://www.emergentmind.com/topics/sequential-pathfinder-separ
type: topic
---

# Sequential Pathfinder (SePar) for Multi-Agent Delivery

Searching arXiv for the SePar paper and closely related MAPF/MAPD baselines to ground the article in current literature.
First, locating the primary paper "Sequence Pathfinder for Multi-Agent Pickup and Delivery in the Warehouse" and core comparative methods.
Sequential Pathfinder (SePar) is a Transformer-based policy architecture for Multi-Agent Pickup and Delivery (MAPD) in warehouse-like environments, introduced by Zhao et al. as a sequence-modeling reformulation of lifelong Multi-Agent Path Finding (MAPF) [2509.23778]. In this formulation, a fleet of agents repeatedly receives fixed-location pickup and delivery requests and must execute collision-free paths under partial observability, while maintaining throughput in layouts characterized by long corridors, narrow pathways, and frequent single-file traffic. SePar addresses the coordination bottleneck by factorizing the joint policy autoregressively over agents and implementing that factorization with a Multi-Agent Transformer, with the central claim that sequence modeling preserves order-invariant optimality while reducing the effective decision complexity from an exponential joint-action space to linear-time action generation [2509.23778].

## 1. Problem setting and formalization

SePar is defined on an undirected grid graph $\mathcal G=(V,E)$ of size $h\times w$ with obstacle set $O\subset V$, free nodes $V\setminus O$, and $n$ agents [2509.23778]. At time $t$, each agent $i$ observes $o^i_t$ from its $m\times m$ field of view, so the joint observation is $\bm o_t=(o^1_t,\dots,o^n_t)$. The action space is discrete,
$$
A=\{\mathsf{NOOP},\mathsf{L},\mathsf{R},\mathsf{U},\mathsf{D}\},
$$
and a joint action $\bm a_t\in A^n$ induces a transition $P(s_{t+1}\mid s_t,\bm a_t)$.

The MAPD problem is cast as a Dec-POMDP,
$$
M=\langle S,A,\mathbb N,P,R,O,\gamma\rangle,
$$
where each agent’s reward $R^i(\bm o_t,\bm a_t)$ penalizes collisions and step-costs ($-0.3$), and awards $+5.0$ for pickups and $+10.0$ for deliveries [2509.23778]. The setting is lifelong MAPF: when an agent arrives at its current goal $\tau^i_t$, a new request $(\mathrm{pickup},\mathrm{drop})$ is assigned and the agent must immediately proceed.

A distinctive element in the paper is the Path-Finding Complexity Index (PFCI),
$$
PFCI=(l_{\mathrm{corr},v_e),\quad v_e=\alpha\bigl(\rho_e\,\rho_t\bigr)^{-1},\quad \rho_e=\frac{2|E|}{|V|(|V|-1)},\ \rho_t=1-\tfrac{|O|}{|V|}.
$$
Here $l_{\mathrm{corr}$ captures typical corridor length, and $v_e$ measures “narrowness” [2509.23778]. In the reported experiments, high-PFCI warehouse maps serve as the regime in which purely local decentralized policies tend to suffer from deadlocks and gridlocks, while classical planners incur high planning latency as $n$ grows.

## 2. Sequence modeling and order-invariant optimality

The conceptual core of SePar is the recasting of MAPD as a sequence-modeling problem. For any permutation $\sigma\in S_n$ of the agents, the policy is written autoregressively as
$$
\pi^\sigma_\theta(a^{1:n}\mid \bm o)\;=\;\prod_{k=1}^n
\pi_\theta\bigl(a^{\sigma[k]}\mid \bm o,\;a^{\sigma[1]},\dots,a^{\sigma[k-1]}\bigr).
$$
Proposition 1, termed order-invariant optimality, states that for any two permutations $\sigma,\nu$, the induced joint distributions coincide:
$$
\forall \bm o,\quad \pi^\sigma_\theta(\cdot\mid\bm o)
=\pi^\nu_\theta(\cdot\mid\bm o)
\quad\Longrightarrow\quad
\sup_\theta f(\pi^\sigma_\theta)=\sup_\theta f(\pi^\nu_\theta),
$$
where $f(\cdot)$ can be makespan or lifelong throughput [2509.23778].

The key proof step is that the different autoregressive factorizations are merely re-orderings of the same underlying joint policy. Within the paper’s formalism, this is used to reject a common concern about autoregressive multi-agent control: that the imposed decoding order necessarily injects an intrinsic order-bias into the optimum. The claim is narrower than a blanket statement about all implementations; it establishes that an autoregressive decomposition does not lose optimality nor induce order-bias at the level of the modeled joint policy [2509.23778].

This theoretical move is significant because it allows SePar to treat joint decision-making as sequential token prediction without abandoning global coordination. A plausible implication is that the architecture inherits the representational convenience of sequence models while remaining aligned with MAPD objectives such as throughput and collision avoidance.

## 3. Architecture and information flow

SePar combines an on-line feature extractor with a Multi-Agent Transformer (MAT) [2509.23778]. The observation feature extractor consists of two VGG-style CNN blocks, each with 3 convolutions and 1 max-pool, followed by one extra $3\times 3$ convolution and then two fully connected layers. Temporal context is propagated by an LSTM carrying $(h_{t-1},c_{t-1})$, producing a per-agent embedding $z^i_t\in\mathbb R^d$.

The MAT has an encoder-decoder structure. The encoder applies self-attention over the sequence $(z^{1}_t,\dots,z^{n}_t)$ with no masking, yielding refined observations $(\hat o^1_t,\dots,\hat o^n_t)$ that encode high-level agent interactions. A tiny value-head MLP on each $\hat o^i_t$ approximates $V(\hat o^i_t)$. The decoder is autoregressive and agent-by-agent: at step $k$ it attends to $\hat o^{1:n}_t$ and to the past decoded actions $(a^{1}_{t},\dots,a^{k-1}_{t})$ via triangular-mask self-attention, ensuring no peeking ahead; a final MLP maps the decoder state to $\pi^k_\theta(a^k_t\mid \hat o^{1:n}_t,a^{1:k-1}_t)$ [2509.23778].

The tokenization is explicit. Input tokens are the per-agent embeddings $z^i_t$, and output tokens are learned embeddings for action symbols fed autoregressively. The encoder implements implicit communication: full self-attention makes every agent aware of every other agent’s embedding in $O(n^2)$, but in a parallelizable form. The decoder then conditions each agent’s action on all predecessor actions, propagating commitments through the action sequence.

The paper’s pseudocode sketch is:
1. collect $\bm o_t$ and extract $z^{i}_t$ via CNN+LSTM;
2. encode $(z^1_t,\dots,z^n_t)\to(\hat o^1_t,\dots,\hat o^n_t)$;
3. for $k=1\ldots n$, decode $a^k_t\sim\pi^k_\theta(\,\cdot\mid \hat o^{1:n}_t,a^{1:k-1}_t)$;
4. execute $\bm a_t=(a^1_t,\dots,a^n_t)$ and observe reward and next observations [2509.23778].

## 4. Training regime, imitation learning, and computational scaling

SePar is trained with a mixed reinforcement-learning and imitation-learning pipeline. The paper states that pure RL with PPO struggles in high-PFCI maps because of sparse successes and frequent deadlocks, so SePar interleaves on-policy PPO updates with off-policy behavioral cloning from a heuristic planner, LaCAM2 [2509.23778]. The behavioral-cloning loss is
$$
L_{\mathrm{bc}(\phi,\theta)
= -\frac1{nT}\sum_{i=1}^n\sum_{t=0}^{T-1}
\log \pi^i_\theta\bigl(a^{\star,i}_t\mid \hat o^{1:n}_t,a^{\star,1:i-1}_t\bigr),
$$
where $a^{\star,i}_t$ is the expert action.

Within the reported results, this mixed RL + IL pipeline provides high-quality demos in narrow corridors, dramatically accelerating convergence and raising final throughput by up to 50% [2509.23778]. The ablation evidence is also direct: SePar-Follower, which omits IL, fails to learn in high-PFCI layouts, and removing IL halves performance on warehouse layouts. This is the paper’s strongest empirical basis for the claim that imitation learning is not merely helpful but necessary in complex warehouse maps.

The complexity argument is one of the architecture’s defining features. A classical joint-action policy must consider $|A|^n$ action combinations, which is exponential in $n$. A decentralized approach runs $n$ independent planners but cannot coordinate globally. By contrast, SePar’s autoregressive decoder attends to $n$ tokens and, at each step, selects one action from $|A|$ possibilities, for total complexity $O(n\,|A|)$ per timestep. The encoder incurs $O(n^2)$ attention, but this is described as amortized via parallel GPU kernels, and the paper concludes that end-to-end inference scales linearly with $n$ in wall-time [2509.23778].

A second misconception addressed in the ablations concerns masked decoding. The paper reports that removing masked decoding destroys collision-avoidance guarantees. In SePar’s design, triangular masking is therefore not an implementation detail but a structural requirement for the autoregressive semantics of coordinated action generation.

## 5. Empirical evaluation and observed behavior

The evaluation spans both warehouse MAPD and POGEMA benchmarks [2509.23778]. In the Warehouse Simulator, the layouts are `warehouse_small` with PFCI $\approx 11.4$, `warehouse_large` with PFCI $\approx 21.7$, and three unseen huge maps with PFCI up to $26.1$. The metric is throughput, defined as total deliveries divided by time. In POGEMA, the tasks include one-shot MAPF and lifelong MAPF on mazes with PFCI $\approx 0.33$, using Success Rate, Sum of Costs (SoC), and Throughput.

On `warehouse_small` and `warehouse_large` with 32–512 agents, SePar surpasses PRIMAL2 and SCRIMP by $2$–$3\times$ throughput for $n\ge 128$ [2509.23778]. At $n=256$, SePar retains $50$–$90\%$ of LaCAM2’s throughput, whereas the other learning-based methods collapse to $7$–$15\%$. The out-of-distribution results on `warehouse_huge` maps are more qualified: SePar achieves $70$–$85\%$ of LaCAM2’s throughput on two similar layouts, but performance drops on a structurally distinct third map.

On POGEMA mazes with 8–64 agents, SePar achieves a MAPF Success Rate exceeding DCC, MAMBA, and SCRIMP by $10$–$20\%$, and it has the lowest total path length for $n\ge 32$ in terms of SoC [2509.23778]. For lifelong MAPF throughput, it is second only to Follower and the planning solver RHCR. Taken together, these results position SePar as strongest in the regime where local observation is insufficient and where dense interaction structure makes explicit pairwise communication costly.

The reported empirical pattern is internally consistent with the paper’s training argument. In open-space benchmarks, pure RL is said to suffice; in high-PFCI warehouse layouts, IL becomes critical. This suggests that SePar’s advantage is not simply a generic benefit of Transformers, but the conjunction of sequence modeling, implicit all-to-all information exchange, masked autoregressive decoding, and expert-guided training.

## 6. Scope, limitations, and terminological ambiguity

The paper’s conclusions are explicitly bounded. SePar is presented as a globally informed, scalable MAPD solver whose inference cost grows linearly with $n$ while preserving optimality guarantees, but its limitations include sensitivity to drastically out-of-distribution layouts and an $O(n^2)$ encoder in compute [2509.23778]. The stated future directions are sparse attention or graph-sparsification, along with extensions to dynamic disturbances such as obstacle appearances and communication dropouts.

A separate source of ambiguity is terminological rather than algorithmic. The acronym “SePar” is also used in the autonomous microscopy paper “PATHFINDER: Multi-objective discovery in structural and spectral spaces,” where “Sequential PATHFINDER (SePar)” denotes a sequential decision algorithm that combines a VAE latent representation, a deep-kernel Gaussian-process surrogate, and a Pareto-hypervolume-improvement acquisition rule for autonomous measurement selection [2604.04194]. That use concerns multi-objective discovery in structural and spectral spaces, including STEM-EELS and ferroelectric scanning probe microscopy, rather than MAPD in warehouses.

Accordingly, in current usage the label “Sequential Pathfinder (SePar)” is ambiguous across at least two technically unrelated contexts. In multi-agent robotics, it denotes Zhao et al.’s sequence-modeling architecture for lifelong MAPD [2509.23778]. In autonomous microscopy, it denotes the sequential acquisition algorithm embedded in the PATHFINDER framework [2604.04194]. The shared acronym does not imply methodological continuity between the two.

Source: https://www.emergentmind.com/topics/sequential-pathfinder-separ