Online MAPD: Pickup & Delivery

• Online Multi-Agent Pickup and Delivery is a dynamic framework where multiple agents continuously execute pickup and delivery tasks in a shared, collision-aware environment.
• It employs mechanisms such as token passing, integrated task assignment, and kinematic path planning to optimize service times and manage congestion.
• Recent advancements incorporate deadline awareness, human-robot collaboration, and learning-based schedulers to enhance real-time performance in complex warehouse settings.

Searching arXiv for recent and foundational MAPD papers to ground the article. Searching arXiv for "online multi-agent pickup and delivery". Online Multi-Agent Pickup and Delivery (MAPD) is a lifelong, online extension of Multi-Agent Path Finding (MAPF) in which multiple agents repeatedly serve a stream of pickup-and-delivery tasks on a shared environment while avoiding collisions. In the canonical formulation, one agent is assigned to each task, the agent must first move to the pickup location and then to the delivery location, and new tasks can appear at any timestep rather than being fixed in advance (Ma et al., 2017). Across warehouse automation, intelligent storage systems, sortation, and related logistics domains, the literature has progressively expanded this core model to include dynamic task streams, kinematic constraints, deadlines, delays, human-robot collaboration, movable infrastructure, and large-scale scheduling regimes (Lau et al., 2022).

1. Formal model and canonical assumptions

The standard graph-based MAPD formulation uses an undirected, connected graph $G = (V, E)$, where vertices are locations and edges are feasible moves. There are $m$ agents $A = \{a_1, a_2, \ldots, a_m\}$, and tasks arrive online as exogenous events. Each task has a pickup location $s_j$ and a delivery location $g_j$. A free agent may be assigned to an unexecuted task, can be reassigned while en route to the pickup, but not after the item is picked up, and must execute the pickup-before-delivery sequence without colliding with other agents (Ma et al., 2017).

Collision avoidance is usually stated with vertex and edge-swap constraints. In the discrete-time model, for all $t$, agents cannot occupy the same location at the same timestep, and they cannot traverse the same edge in opposite directions in the same step. A common performance metric is service time, defined as the average number of timesteps between a task’s arrival and its completion: $$\text{Service Time} = \frac{1}{|\mathcal{T}|} \sum_j (t_j^{done} - t_j^{add}) .$$ This metric became central in early lifelong MAPD work because it captures both task assignment quality and congestion-sensitive path execution (Ma et al., 2017).

A foundational structural notion is the well-formed MAPD instance. In the cited formulation, a MAPD instance is well-formed if the number of tasks is finite, the number of non-task endpoints is at least the number of agents, and for any two endpoints there is a path between them that does not traverse any other endpoint (Ma et al., 2017). This condition underpins several completeness results because it guarantees that agents can move out of one another’s way without permanently blocking pickup or delivery sites.

The warehouse-oriented literature often presents MAPD as a lifelong cycle rather than a one-shot routing problem. In benchmark formulations, agents repeatedly move from current location to pickup, then to delivery, and then continue with newly issued requests; in MARL formulations, the same process is cast as a partially observable Markov game with local observations, joint action space, and discounted return maximization (Lau et al., 2022). This suggests that online MAPD occupies an intermediate position between classical task allocation and persistent multi-agent motion planning.

2. Token passing and the classical online MAPD lineage

The canonical decoupled baseline is Token Passing (TP). In TP, a shared token stores current agent paths, assignments, and the task set. When an agent’s path ends, it requests the token, selects the closest suitable task, plans a collision-free path from current location to pickup to delivery, and, if no suitable task exists, may move to an available non-task endpoint so that task endpoints remain usable (Ma et al., 2017). Paths are extended to remain forever at the final location, which makes the token a persistent reservation structure rather than a transient dispatch record.

Token Passing with Task Swaps (TPTS) relaxes TP’s fixed pre-pickup assignment rule. An agent holding the token can consider tasks already assigned to other agents if those tasks have not yet been picked up, and it may unassign a slower agent when it can reach the pickup earlier. The reassignment can recurse, producing chains of swaps until all involved agents have either new tasks or parking destinations. Both TP and TPTS are shown to solve all well-formed MAPD instances (Ma et al., 2017). Experimentally, TPTS reduced service time by up to $42\%$ over TP, while a centralized baseline reduced it by up to $48\%$ over TP in some settings, albeit without the same correctness guarantee (Ma et al., 2017).

A major refinement of this lineage is TP-SIPPwRT, which replaces the single-agent planner inside TP with Safe Interval Path Planning with Reservation Table. This variant plans continuous movements with given translational and rotational velocities, supports point turns, models agents as disks with safety margins, and computes kinematically feasible trajectories directly rather than applying post-processing to discrete plans (Ma et al., 2018). TP-SIPPwRT remains complete for well-formed MAPD instances and was reported to compute paths for hundreds of agents and thousands of tasks in seconds; representative runtimes include less than $1$ second for $m$0 agents and $m$1 tasks, and less than $m$2 seconds for $m$3 agents and $m$4 tasks (Ma et al., 2018).

This line of work established several enduring MAPD design principles: decentralized planning mediated by a shared token, greedy but correctness-preserving assignment, and explicit endpoint management to avoid deadlock. It also made clear that single-agent planner quality strongly affects the practical viability of online MAPD.

3. Coupling task assignment with path planning

A central criticism of purely sequential MAPD solvers is that they assign tasks using lower-bound or heuristic costs and only later discover the true cost of collision-free execution. One response is the integrated task-assignment-and-path-planning approach for capacitated MAPD, where assignment decisions are informed by actual delivery costs rather than lower-bound estimates (Chen et al., 2021). The main ingredients are a marginal-cost assignment heuristic, a regret-based variant, and an anytime Large Neighbourhood Search (LNS) improvement phase. This formulation also allows agents with capacity $m$5, so that one robot can carry multiple tasks simultaneously, and optimizes Total Travel Delay (TTD): $m$6 In lifelong experiments, RMCA(r) with LNS achieved significantly lower TTD than TPTS and CENTRAL while maintaining competitive or better makespan and faster per-timestep runtimes; increasing agent capacity reduced both TTD and makespan (Chen et al., 2021).

A second influential direction assigns task sequences, not just individual tasks. LNS-PBS and LNS-wPBS use LNS to assign a sequence of tasks to each agent and then use Priority-Based Search (PBS) or windowed PBS for collision-free execution (Xu et al., 2022). LNS-PBS is complete for well-formed MAPD instances and empirically more effective than CENTRAL; LNS-wPBS gives up completeness but is more efficient and stable, scales to thousands of agents and thousands of tasks in a large warehouse, and was reported to achieve service times $m$7–$m$8 lower than HBH+MLA*, while LNS-PBS yielded service times up to $m$9 lower than CENTRAL (Xu et al., 2022). The same framework also generalizes to Multi-Goal MAPD, where tasks can require visiting more than two goal locations (Xu et al., 2022).

At much larger scales, flow-based task assignment reformulates the assignment subproblem as a minimum-cost flow directly over the environment graph, thereby eliminating pairwise agent-task distance computations and simultaneously extracting guide paths (Zhang et al., 7 Aug 2025). Two congestion-aware edge cost models incorporate either planner-estimated traffic or average waiting times from past execution. The resulting method supports real-time execution, scales to over $A = \{a_1, a_2, \ldots, a_m\}$0 agents and $A = \{a_1, a_2, \ldots, a_m\}$1 tasks within $A = \{a_1, a_2, \ldots, a_m\}$2-second planning time, and achieved throughput gains of up to $A = \{a_1, a_2, \ldots, a_m\}$3 over greedy assignment, together with lower makespan and fewer timeouts than RMCA in reported comparisons (Zhang et al., 7 Aug 2025).

Taken together, these systems mark a shift from token-mediated greedy dispatch toward assignment schemes that explicitly internalize congestion, sequence effects, and execution feasibility. A plausible implication is that the task-assignment layer has become a primary bottleneck in large online MAPD systems, not merely a preprocessing step before path finding.

4. Deadlocks, restricted topologies, and asynchronous execution

Much early MAPD work assumes warehouse-like grids with many endpoints and plentiful detours. Restricted environments expose the limits of those assumptions. In maze-like graphs with few pickup and delivery locations and imbalanced endpoint distribution, Standby-Based Deadlock Avoidance (SBDA) introduces standby nodes selected in real time using an articulation-point-finding algorithm (Yamauchi et al., 2022). A potential standby node is neither an articulation point, nor an endpoint, nor a dead-end. By reserving such nodes when endpoints are occupied, SBDA allows agents to wait without disconnecting the graph. On two maze-like environments with $A = \{a_1, a_2, \ldots, a_m\}$4, it achieved an approximately $A = \{a_1, a_2, \ldots, a_m\}$5 makespan reduction in Environment 1 and approximately $A = \{a_1, a_2, \ldots, a_m\}$6 in Environment 2 relative to HTE, and it remained applicable where PIBT is inapplicable because the map is not bi-connected (Yamauchi et al., 2022).

A related extension addresses environments with tree-shaped dead ends attached to a bi-connected main area. PIBT with Temporary Priority (PIBTTP) gives an agent in a tree a temporary high priority,

$A = \{a_1, a_2, \ldots, a_m\}$7

so that agents leaving a tree can always push others back until they exit (Fujitani et al., 2022). Under assumptions $A = \{a_1, a_2, \ldots, a_m\}$8–$A = \{a_1, a_2, \ldots, a_m\}$9, the paper proves that if the task set is finite, all agents can complete all tasks within a finite number of timesteps. PIBTTP-TA further lets pushed agents temporarily avoid in side branches rather than retreating all the way to the main area, and it outperformed both TP and PIBTTP in the tested environments (Fujitani et al., 2022).

Real deployments also motivate asynchronous rather than globally synchronized execution. The MAPDFS formulation introduces fluctuated speeds and flexible endpoints, together with a distributed planning architecture that combines carrier agents with node agents managing local reservations (Miyashita et al., 2023). The main area is oriented into a strongly connected digraph so that one-way traffic suppresses head-on conflicts, and agents resolve movement conflicts through local communication with neighboring node agents rather than global replanning. In the reported experiments, the method achieved $s_j$0 completion despite delays, small endpoint counts, and dense agent packing, while requiring much less CPU time than HTE and RHCR (Miyashita et al., 2023).

These works show that “online” in MAPD cannot be reduced to task arrival alone. It also includes online management of graph connectivity, waiting rights, local circulation patterns, and synchronization assumptions.

5. Deadlines, delays, human collaboration, and evolving tasks

Several MAPD extensions modify the online objective itself. MAPD-D augments each task with a delivery deadline $s_j$1, defines tardiness as

$s_j$2

and focuses on minimizing cumulative tardiness $s_j$3 (Makino et al., 2024). Deadline-aware Token Passing (D-TP) computes pickup deadlines by planning backward from delivery using a dummy agent, while D-TPTS adds task swaps and task switching. Both algorithms produced significant reductions in task tardiness compared with existing methods; in a dense, long-deadline case, task swapping reduced tardiness by $s_j$4 units and task switching reduced it by a further $s_j$5 (Makino et al., 2024).

Execution uncertainty motivates MAPD with delays (MAPD-d). In this setting, k-TP guarantees $s_j$6-robustness by reserving each occupied vertex for $s_j$7 timesteps before and after nominal occupancy, while p-TP uses a Markov-chain delay model to ensure that the probability of collision-free execution is at least $s_j$8 (Lodigiani et al., 2023). Relative to reactive TP with online replanning, both methods significantly reduced the number of replans caused by delays, with little or no increase in solution cost and running time (Lodigiani et al., 2023).

Human-robot settings introduce a different class of uncertainty. The Human-Swarm Hybrid System Pickup and Delivery framework, denoted $s_j$9, addresses time-varying, heterogeneous, and potentially uncertain tasks executed jointly by human workers and robot swarms (Gong et al., 2022). It uses a two-layer decision framework based on a prediction horizon window: the first layer is a two-level programming problem for mode assignment and task allocation, and the second layer solves mixed-integer programs for exact robot paths. Human unpredictability, fatigue-aware costs, and mode-specific constraints are included in assignment, while robot safety is enforced through collision avoidance, emergency stop behavior when sensors detect a human within a safe distance, and rolling-horizon replanning when the map changes (Gong et al., 2022).

A more dynamics-intensive warehouse formulation is Interleaved Prioritized Planning with Via-Point Star. Here, online order delivery creates a stream of inter-dependent pickup and delivery tasks, and robot trajectories must obey realistic differential-drive dynamics, load-dependent timing, orientation constraints, and reservation-based collision checks (Lehoux-Lebacque et al., 2024). The paper proves a completeness theorem for the planning layer under the assumption that each robot has a designated waiting place that does not interfere with other trajectories. Ablations showed that removing via-point planning made up to $g_j$0 of four-robot scenarios infeasible, while removing waiting place reservation caused failure in up to $g_j$1 of such runs (Lehoux-Lebacque et al., 2024).

Dynamic order evolution pushes the online aspect further. In Robotic Cellular Warehousing Systems, Dynamic Token Passing and Cooperative Token Passing handle the case where new SKUs are appended to an order during execution rather than only at release time (Ren et al., 4 Jun 2026). The first method performs localized replanning for the originally bound robot, whereas the second lets idle robots opportunistically serve newly added pickups if doing so reduces order flowtime. For a $g_j$2 warehouse with $g_j$3 and $g_j$4, average flowtime was reported as $g_j$5 for TP, $g_j$6 for TP-Append, $g_j$7 for Dynamic-TP, and $g_j$8 for Cooperative-TP; reported update runtimes remained in the real-time range, from $g_j$9–$t$0 ms for Dynamic-TP to as much as $t$1 ms at $t$2 robots for Cooperative-TP (Ren et al., 4 Jun 2026).

6. Warehouse-specific generalizations and learning-based schedulers

Warehouse MAPD has increasingly been generalized beyond fixed one-to-one pickup and delivery tasks. Double-Deck MAPD (DD-MAPD) allows agents to move beneath shelves or lift and deliver a shelf to an arbitrary location, thereby changing the warehouse layout (Li et al., 2023). The paper proves that solving DD-MAPD is NP-hard and proposes MAPF-DECOMP, which first computes virtual shelf trajectories via MAPF and then solves a MAPD instance with task dependencies for the agents that carry those shelves. The framework was reported to compute high-quality solutions for instances with over one thousand shelves and hundreds of agents in minutes, and a complete variant is given for well-formed DD-MAPD instances (Li et al., 2023).

Terraforming MAPD (tMAPD) similarly treats non-task pods as movable rather than static obstacles. In the proposed tRHCR framework, free agents may relocate pods during disruptions such as dropped items or agent malfunctions, but terraforming is invoked only when it is likely to have significant impact on solution quality (Vainshtein et al., 2023). In the reported experiments, this raised throughput by over $t$3 and reduced maximum service time by more than $t$4 relative to the standard MAPD setting (Vainshtein et al., 2023).

Many-to-many MAPD departs from the one-to-one assumption by letting tasks specify sets of admissible pickup and delivery locations associated with the same SKU, turning assignment into an NP-hard four-dimensional problem over agent, task, source, and destination (Schneider et al., 8 May 2026). The proposed M2M algorithm and its SKU-aware variant M2M-wSKU were evaluated over $t$5-hour warehouse operations, and M2M was reported to complete up to $t$6 more tasks on average across different environments and inventory densities than prior state of the art (Schneider et al., 8 May 2026).

Benchmarking and learning-based work has widened the algorithmic design space. A comparative study of CBS and SEAC found that CBS yields better path efficiency in small, sparse settings, while SEAC scales more gracefully and delivers more items per agent as agent count grows (Lau et al., 2022). The same study argued that the field’s MAPF and MARL traditions use different metrics and suggested hybrid approaches using lifelong MAPF as expert data for imitation or inverse reinforcement learning (Lau et al., 2022).

More recent systems make that hybridization explicit. SePar formulates path finding as a sequence modeling problem, proves order-invariant optimality for autoregressive policies, and uses a Transformer to achieve implicit information exchange with decision complexity reduced from exponential to linear; the paper also emphasizes that imitation learning is necessary in complex warehouse maps (Zhao et al., 28 Sep 2025). GRAND uses a graph neural network trained by reinforcement learning to output a desired distribution of free agents over an aggregated warehouse graph, then converts that guidance into region-to-region rebalancing through minimum-cost flow and small local assignment problems; on League of Robot Runners benchmarks with up to $t$7 agents, it improved throughput by up to $t$8 over the $t$9 winning scheduler while maintaining a $$\text{Service Time} = \frac{1}{|\mathcal{T}|} \sum_j (t_j^{done} - t_j^{add}) .$$0-second compute budget (Gaber et al., 2 Dec 2025). Neural ATTF combines Priority Guided Task Matching with Neural STA*, a learned Space-Time A* variant, and was reported to maintain sub-second per-timestep runtime for $$\text{Service Time} = \frac{1}{|\mathcal{T}|} \sum_j (t_j^{done} - t_j^{add}) .$$1 agents while achieving up to $$\text{Service Time} = \frac{1}{|\mathcal{T}|} \sum_j (t_j^{done} - t_j^{add}) .$$2 lower service time than TPTS in high task-frequency settings (Shah et al., 21 Apr 2025).

Taken together, these developments suggest that contemporary online MAPD is no longer a single problem class centered on well-formed grids and static one-to-one tasks. It has become a family of dispatch, routing, and execution problems spanning deadlines, uncertainty, human presence, dynamic infrastructure, interdependent orders, and very large fleets, with solutions ranging from complete token-based methods to congestion-aware flow models and learning-optimization hybrids.