Exhaustive-Serve-Longest (ESL) Scheduling

• ESL is a state-dependent scheduling policy for multi-robot, multi-queue systems that enforces exhaustive service and reassigns idle robots to the longest unoccupied queue.
• The policy is proven optimal in symmetric i.i.d. Bernoulli arrival models with one-slot service and switching delays, effectively minimizing discounted system backlog.
• Under heterogeneous arrival scenarios, ESL serves as a baseline while learned strategies like EA-AC can better adjust to varying queue dynamics and arrival rates.

Searching arXiv for papers on Exhaustive-Serve-Longest in multi-robot multi-queue scheduling and closely related serve-the-longest queueing work. Exhaustive-Serve-Longest (ESL) is a state-dependent scheduling policy for slotted multi-robot, multi-queue systems with switching delays. In the formulation studied in recent arXiv work, $M$ mobile robots serve $N$ fixed locations, each location hosts at most one robot per slot, service of one task consumes one slot, switching to another location consumes one slot with no service, and arrivals are Bernoulli. ESL combines two rules: exhaustive service at the current location whenever that location is nonempty, and longest-queue reassignment when a robot becomes idle at an empty location. In the symmetric i.i.d. Bernoulli-arrival model this policy is proved optimal; in heterogeneous-arrival models it remains the canonical exhaustive baseline but is not presented as universally optimal and can be suboptimal (Merati et al., 29 Sep 2025, Merati et al., 4 Apr 2026).

1. Policy definition and internal structure

In the multi-robot scheduling literature, ESL is not merely a longest-queue heuristic. It is a two-layer control law. First, it imposes exhaustive service: once a robot is located at a nonempty queue, it must continue serving that queue and does not switch away until the queue becomes empty. Second, it specifies an idle-robot reassignment rule: when a robot is at an empty queue, it is sent to a longest unoccupied nonempty queue. This distinction is central because the later asymmetric study explicitly separates exhaustive structure from the particular “serve-longest” reassignment logic (Merati et al., 4 Apr 2026).

If robot $r$ is at location $s_r(t)$ with $x_{s_r(t)}(t)>0$, ESL enforces

$u_r(t)=\mathrm{serve}.$

If instead $x_{s_r(t)}(t)=0$, the robot selects a destination from the feasible set of unoccupied nonempty queues. In the asymmetric paper this feasible set is written

$$\mathcal F_r(z)=\{\, i : x_i>0,\ \text{queue } i \text{ is unoccupied in the current slot} \,\},$$

and ESL chooses

$i^* \in \arg\max_{i\in \mathcal F_r(z)} x_i,$

with ties broken first by higher arrival rate and then by queue index. In the symmetric optimality paper, the theorem allows arbitrary tie-breaking among equal-length longest queues, while the simulations break ties by lower index (Merati et al., 4 Apr 2026, Merati et al., 29 Sep 2025).

The same literature distinguishes three related but nonidentical objects.

Layer	Meaning	Decision content
Exhaustive service	Structural property	Busy robots do not abandon nonempty queues
ESL	Specific exhaustive policy	Idle robots go to a longest unoccupied nonempty queue
EA-AC	Exhaustive-assignment actor-critic	Exhaustive service is hard-coded; only idle-robot reassignment is learned

This decomposition is important because many claims that appear to be about ESL are in fact claims about exhaustive service more generally, or about optimality under symmetry rather than under arbitrary heterogeneity.

2. Discrete-time MDP formulation

The ESL papers formulate the control problem as a discounted-cost Markov decision process on the full robot-location and queue-length state

$$z(t)=\bigl(s(t);x(t)\bigr), \qquad s(t)=(s_1(t),\dots,s_M(t)),\quad x(t)=(x_1(t),\dots,x_N(t)).$$

Queue $N$0 has backlog $N$1 at the beginning of slot $N$2, and robot $N$3 is at $N$4. Arrivals are independent Bernoulli processes,

$N$5

with late arrivals: tasks arriving during slot $N$6 join at the end of the slot and cannot be served until $N$7. A robot serving its current location completes exactly one task in that slot; a robot that switches spends the entire slot traveling and provides no service (Merati et al., 4 Apr 2026).

Before imposing exhaustive structure, the admissible action for robot $N$8 is

$N$9

subject to the one-robot-per-location constraint. With

$r$0

the dynamics are

$r$1

and

$r$2

The stage cost is total holding cost,

$r$3

and for a stationary deterministic policy $r$4,

$r$5

ESL is therefore a structured stationary control law for minimizing discounted backlog in a switching-delay MDP (Merati et al., 4 Apr 2026).

3. Structural optimality in the symmetric model

The 2025 ESL paper gives a full optimality theorem for the symmetric model with i.i.d. Bernoulli$r$6 arrivals at each location, unit service time, one-slot switching delay, no co-location, unit holding cost, and discount factor $r$7. The theorem states that there is an optimal ESL policy with the following action structure: if a robot is at a nonempty location, it serves exhaustively; if it is at an empty location, it switches to a longest unoccupied nonempty location (Merati et al., 29 Sep 2025).

The proof is structural rather than merely empirical. It proceeds through three sample-path domination statements. First, if a robot is at a nonempty queue, idling or switching away is strictly suboptimal. Second, if a robot is at an empty queue and some unoccupied nonempty queue exists, idling is strictly suboptimal. Third, when a switch is required, choosing a shorter queue over a longer queue is strictly suboptimal. The argument repeatedly uses the departure-backlog identity

$r$8

under common-arrival coupling, so earlier cumulative departures imply lower cumulative backlog. Symmetry enters through queue-swapping and mirroring constructions that preserve law only when arrivals are identically distributed across queues (Merati et al., 29 Sep 2025).

The multi-robot extension is also explicit. If $r$9 robots simultaneously require reassignment and $s_r(t)$0 is the set of unoccupied nonempty locations, any collision-free assignment that does not cover the $s_r(t)$1 longest locations in $s_r(t)$2 is strictly dominated by one that does. Thus ESL is not simply a per-robot greedy rule; in the multi-robot case it is an injective assignment policy onto distinct longest queues (Merati et al., 29 Sep 2025).

A common misconception is to treat this theorem as a universal result about longest-queue routing. The available arXiv record does not support that reading. The theorem is proved for the symmetric i.i.d. Bernoulli model with deterministic one-slot travel and one-slot service, not for heterogeneous arrival rates or heterogeneous travel times.

4. Asymmetry and the move from ESL to learned exhaustive assignment

The 2026 paper reformulates the same class of systems under heterogeneous Bernoulli arrival rates $s_r(t)$3 and emphasizes a sharper separation between exhaustive structure and ESL as a particular reassignment rule. Its starting point is the prior structural result that optimal policies are of exhaustive type; what is no longer assumed is that sending idle robots to the longest currently available queue remains optimal when queues have different long-run arrival intensities (Merati et al., 4 Apr 2026).

The paper’s qualitative explanation is direct: in heterogeneous scenarios, “assigning idle robots using only the longest currently available queue is less effective than using a learned reassignment rule that can account for both backlog and long-run arrival asymmetry.” In this formulation, ESL uses current queue length as the primary criterion and uses arrival rate only as a tie-break. That means “longest” refers to instantaneous backlog, not expected workload, discounted cost-to-go, or a weighted index (Merati et al., 4 Apr 2026).

To exploit asymmetry, the paper proposes an exhaustive-assignment actor-critic policy class. Busy and idle robots are split as

$s_r(t)$4

and the policy factorizes as

$s_r(t)$5

so exhaustive service is enforced by construction and only idle-robot assignment is learned. Queue features are

$s_r(t)$6

robot features are

$s_r(t)$7

and the compatibility score for idle robot $s_r(t)$8 and candidate queue $s_r(t)$9 is

$x_{s_r(t)}(t)>0$0

Occupancy masks remove infeasible destinations, and reservation masks prevent multiple idle robots from selecting the same queue (Merati et al., 4 Apr 2026).

The learning objective uses reward

$x_{s_r(t)}(t)>0$1

a critic approximating discounted return, and PPO with clipped surrogate objective

$x_{s_r(t)}(t)>0$2

The reported training setup uses $x_{s_r(t)}(t)>0$3, $x_{s_r(t)}(t)>0$4, $x_{s_r(t)}(t)>0$5, Adam with learning rate $x_{s_r(t)}(t)>0$6, PPO clipping $x_{s_r(t)}(t)>0$7, value-loss coefficient $x_{s_r(t)}(t)>0$8, entropy coefficient $x_{s_r(t)}(t)>0$9, and gradient clipping at $u_r(t)=\mathrm{serve}.$0 (Merati et al., 4 Apr 2026).

5. Empirical behavior across symmetric and asymmetric regimes

The empirical record for ESL in this literature has two distinct parts. In the symmetric model of the 2025 paper, ESL is compared against FCFS and a tuned fixed-dwell cyclic policy. Across server-to-location ratios and loads, ESL yields the lowest discounted holding cost and the smallest mean queue lengths, while action-time fractions show more serving and less switching. In the heavy-load $u_r(t)=\mathrm{serve}.$1 case, ESL reports discounted cost $u_r(t)=\mathrm{serve}.$2 and mean queue length $u_r(t)=\mathrm{serve}.$3, versus FCFS $u_r(t)=\mathrm{serve}.$4 and $u_r(t)=\mathrm{serve}.$5, and cyclic $u_r(t)=\mathrm{serve}.$6 and $u_r(t)=\mathrm{serve}.$7. In the heavy-load $u_r(t)=\mathrm{serve}.$8 case, ESL reports $u_r(t)=\mathrm{serve}.$9 and $x_{s_r(t)}(t)=0$0, versus FCFS $x_{s_r(t)}(t)=0$1 and $x_{s_r(t)}(t)=0$2, and cyclic $x_{s_r(t)}(t)=0$3 and $x_{s_r(t)}(t)=0$4 (Merati et al., 29 Sep 2025).

The 2026 paper uses ESL as the main benchmark for asymmetric arrivals and obtains a more differentiated picture.

Setting	Benchmark relation	Reported outcome
Small asymmetric S1–S3	EA-AC vs exact DP	EA-AC is nearly optimal
Large symmetric S4–S5	EA-AC vs ESL	Essentially indistinguishable
Large asymmetric S1–S7	EA-AC vs ESL	Cost reduction $x_{s_r(t)}(t)=0$5–$x_{s_r(t)}(t)=0$6; mean queue reduction $x_{s_r(t)}(t)=0$7–$x_{s_r(t)}(t)=0$8

In the symmetric cases where ESL is known to be optimal, the learned policy does not outperform it in any meaningful way: for S4, ESL discounted cost is $x_{s_r(t)}(t)=0$9 and EA-AC is $$\mathcal F_r(z)=\{\, i : x_i>0,\ \text{queue } i \text{ is unoccupied in the current slot} \,\},$$0, with mean queue lengths both $$\mathcal F_r(z)=\{\, i : x_i>0,\ \text{queue } i \text{ is unoccupied in the current slot} \,\},$$1; for S5, ESL is $$\mathcal F_r(z)=\{\, i : x_i>0,\ \text{queue } i \text{ is unoccupied in the current slot} \,\},$$2 and EA-AC is $$\mathcal F_r(z)=\{\, i : x_i>0,\ \text{queue } i \text{ is unoccupied in the current slot} \,\},$$3, with mean queue $$\mathcal F_r(z)=\{\, i : x_i>0,\ \text{queue } i \text{ is unoccupied in the current slot} \,\},$$4 versus $$\mathcal F_r(z)=\{\, i : x_i>0,\ \text{queue } i \text{ is unoccupied in the current slot} \,\},$$5. In the asymmetric cases, however, the gap can be substantial: the largest reported reduction is $$\mathcal F_r(z)=\{\, i : x_i>0,\ \text{queue } i \text{ is unoccupied in the current slot} \,\},$$6 in discounted cost and $$\mathcal F_r(z)=\{\, i : x_i>0,\ \text{queue } i \text{ is unoccupied in the current slot} \,\},$$7 in mean queue length, occurring in scenario S6 with $$\mathcal F_r(z)=\{\, i : x_i>0,\ \text{queue } i \text{ is unoccupied in the current slot} \,\},$$8 (Merati et al., 4 Apr 2026).

Operationally, these results support a precise interpretation. ESL is a strong default when queue statistics are approximately homogeneous and switching delays match the one-slot model. Under pronounced heterogeneity, the exhaustive principle remains effective, but the particular “serve-longest” reassignment logic becomes the weak link.

6. Relation to adjacent disciplines and terminological ambiguity

ESL should not be conflated with preemptive Serve-the-Longest Queue disciplines. The paper “A 3-Queue Polling System with Join the Shortest -- Serve the Longest Policy” studies a single-server, three-queue, preemptive JSQ-SLQ model in which the server always attends one of the longest queues and immediately switches when a non-attended queue becomes strictly longer. Interrupted service resumes anew later. The server therefore stays at a queue only while it remains longest, not until the queue empties. That model is relevant background for longest-queue-first polling, but it is not an analysis of exhaustive-serve-longest control (Perel et al., 2022).

There is also an acronym collision outside queueing and robotics. In “Surrogate models for the magnitude of convection in droplets levitated through EML, ADL, and ESL methods,” ESL denotes electro-static levitation in molten-droplet thermofluidics, not exhaustive-serve-longest scheduling. That literature studies Marangoni-driven internal convection in levitated droplets and is unrelated to multi-queue control despite the shared acronym (Usui et al., 2023).

Within the scheduling literature itself, the most persistent misconception is universal optimality. The relevant arXiv papers support a narrower conclusion: ESL is proved optimal in the symmetric i.i.d. Bernoulli model with one-slot service and one-slot switching delay, and it remains a computationally trivial, highly interpretable exhaustive policy in broader settings; but the heterogeneous-arrival study explicitly treats it as a simple baseline whose optimality does not survive asymmetry in general (Merati et al., 29 Sep 2025, Merati et al., 4 Apr 2026).