Augmented Queues: Enhancing Queueing Mechanisms
- Augmented queues are queueing systems enhanced with additional state, measurements, or predictions to mitigate head-of-line blocking, improve fairness, and boost efficiency.
- They are applied in diverse contexts—from elastic, load-adaptive queues in Peacock and per-flow buffer adjustments in Cocoa to augmented Markovian models and learning-augmented priority queues.
- The enhancements integrate dynamic control mechanisms such as adaptive buffering and predictive reordering, delivering robust performance under varied workload and network conditions.
Augmented queues are queueing mechanisms or queue models in which a baseline queue is supplemented with additional state, coordination, or prediction. In Peacock, a fixed-length or unbounded FIFO queue is replaced with an elastic queue whose effective length adapts to the current cluster load, workers form a ring overlay network, and probes are reordered with starvation-free thresholds (Khelghatdoust et al., 2018). In Cocoa, fair-queuing is augmented with per-flow measurements and adaptive buffer limits that are updated at enqueue events without requiring active participation from the endpoint (Bachl et al., 2019). In multiclass GI/M/n+M queues in an alternating renewal environment, augmentation means embedding the non-Markovian queueing dynamics in an augmented Markovian model that incorporates interarrival ages and down-time age (Arapostathis et al., 2019). In learning-augmented priority queues, queue operations are enhanced by oracle predictions while still remaining robust when the prediction is wrong (Benomar et al., 2024).
1. Range of meanings
The term covers several distinct but related research usages. One usage modifies an operational queue directly. Peacock turns a simple FIFO queue into an “augmented queue” by making it elastic, ring-aware and probe-reordering-aware, while Cocoa augments per-flow fair-queuing with a measurements-driven buffer-sizing control loop. A second usage enlarges the state of a queueing model so that the process becomes Markovian, as in the augmented state
for multiclass GI/M/n+M queues in an alternating renewal environment. A third usage appears in algorithm design, where a priority queue is augmented by predictions such as dirty comparisons, pointer predictions, or rank predictions (Khelghatdoust et al., 2018, Bachl et al., 2019, Arapostathis et al., 2019, Benomar et al., 2024).
This suggests that augmentation is not a single construction. In the cited works, it can mean adding workload summaries, adding per-flow control variables, adding latent age variables to recover Markov structure, or adding predictor outputs that change the complexity of queue operations. What remains common is that the queue no longer acts only on raw arrival order or raw key order.
2. Elastic queues, ring overlays, and probe reordering
In Peacock, the baseline object is a standard FIFO queue , in which tasks or “probes” simply line up in arrival order and wait until the head of the queue. The paper identifies two pathologies under heterogeneous workloads: Under-utilization when queues are too short—workers sit idle even though work is pending elsewhere—and Imbalance & Head-of-Line Blocking when queues are too long—short tasks get stuck behind much longer ones. Peacock replaces each fixed-length or unbounded FIFO queue with an elastic queue whose effective length adapts to the current cluster load.
Each worker maintains a local queue of waiting probes and an aggregated remaining-runtime
where is ’s estimated runtime. In addition, workers use a shared state , broadcast by schedulers and rotations, where
Here 0 is the total number of workers. Intuitively, 1 is the average queue length and 2 is the average pending workload per worker.
Dynamic bounding is the first augmentation mechanism. After enqueuing or dequeuing, worker 3 tests 4 or 5. If either bound is violated, the “last” probe in 6 is marked for rotation to its neighbor. The stated impact is twofold: queues cannot grow “too long,” and queues never collapse to zero. Under light load 7, so rotations are rare; under heavy load they kick in to redistribute work.
The second augmentation mechanism is the ring overlay. Workers 8 are connected in a unidirectional ring with
9
During periodic rounds of interval 0, for example 1 s, every worker bundles all probes marked for rotation and sends them in one message to its successor, piggy-backing the latest 2. The reasons given for the ring are symmetry and balance, simplicity, and scalability. Over time a probe will “walk” around the ring until it finds a queue with capacity and non-starving reordering position.
The third augmentation mechanism is probe reordering. Each worker maintains 3 as a sorted list of waiting probes. When a new probe 4 arrives, the worker annotates it with 5 as job-arrival timestamp, 6 as estimated runtime, and 7 as a soft waiting-time threshold. The worker then scans 8 from the tail toward the head. If 9, then 0 may only overtake 1 if 2 and 3’s own threshold would not be violated by letting 4 pass. If 5, then 6 may only overtake 7 if 8 and 9 itself would not violate its threshold by passing, where 0 is the estimated wait before running. If neither party should starve but no safe bypass is possible, 1 is inserted at the current scan point; otherwise 2 is marked for rotation if inserting it would break the starvation-free threshold. Any probe 3 that reaches time 4 will no longer be allowed to be bypassed by any subsequent arrivals.
Peacock also gives an explicit HoL blocking metric: 5 with the goal of driving the expected sum 6 toward zero by reordering short probes ahead of long ones. In large-scale simulation on 7k–8k workers using a Google trace, Peacock is reported as 9–0 faster than Sparrow and 1–2 faster than Eagle in average job completion time across the stated light and heavy loads. At 3 load and 4k workers, 5 of jobs complete under 6 s with Sparrow versus 7 with Peacock. Peacock finishes 8–9 of jobs sooner than Sparrow and 0–1 sooner than Eagle, depending on load. In a real Spark deployment with 2 workers and 3 schedulers, Peacock is 4–5 faster than Sparrow at 6–7 loads and 8–9 faster than Eagle at 0–1 loads (Khelghatdoust et al., 2018).
3. Congestion-control-aware per-flow augmentation
Cocoa is built on top of a fair-queuing discipline, specifically Linux’s fq qdisc. It hashes packets into separate per-flow queues and serves them in round-robin order using deficit-round-robin. Its stated goals are to maximize link utilization for each flow’s congestion control algorithm and to minimize standing queues and hence queuing delay by never holding more packets than absolutely necessary for that flow’s window or rate control.
The augmentation consists in extending each per-flow state with a current buffer-limit 2, a guard-interval timer 3, and counters for packets sent, time active versus idle, and the minimum queue observed. Time is chopped into repeated Guard Intervals. At the end of each Guard Interval, Cocoa picks out the Longest Loss-Free Interval 4 inside that Guard Interval. If the queue never drained to zero during 5, then the minimum queue depth 6 is treated as a standing queue that can safely be subtracted from 7. If the queue did drain to zero, so the link was idle for some 8, Cocoa can add to 9 exactly the number of packets the flow could have sent during that idle gap at its measured rate.
The update rules are
0
when the flow experienced an idle gap, and
1
otherwise. Guard Interval sizing is
2
with default multiplier 3 and 4 s. The paper states that Cocoa passively measures each flow’s bandwidth as “bytes sent” over “time spent sending” in each interval and that no packet-marking or endpoint cooperation is needed.
The interaction with endpoint congestion control algorithms is central. For loss-based CCAs such as Reno and CUBIC, Cocoa increases 5 when the flow idles so that the next window recovery can use the full link, and decreases 6 when it observes a persistent standing queue 7. For model-based CCAs such as BBR, Cocoa lets BBR’s tiny bursts drain and then detects “no queue 8 shrink” or “queue 9 no shrink” at Guard Interval boundaries. The paper characterizes the net effect as zero throughput penalty for BBR and near-zero standing queues.
Cocoa is implemented as a drop-in extension of the existing fq qdisc. The enqueue() callback is extended to run Algorithm 1, which updates intervals, possibly adjusts buffer_size, and decides whether to accept or drop the new packet. The new qdisc knobs are multiplier 0, max_increase 1, and max_GI 2.
The reported empirical results are per-flow and cross-flow. On a 3 Mb/s, 4 ms RTT link with a single CUBIC flow, static fq with a 5-packet buffer yields a 6 ms standing queue, fq_codel underutilizes the link after loss, and Cocoa achieves full 7 Mb/s while minRTT returns to approximately 8 ms after each event. On 9 Mb/s and 00 ms RTT, fq_codel yields average RTT approximately 01 ms and total 02 GB in 03 s, whereas Cocoa yields average RTT approximately 04 ms and total 05 GB in 06 s, about a 07 throughput gain. On Reno with a 08 Mb/s/09 ms link, Cocoa gives more than 10 link use versus fq_codel approximately 11, with throughput gain approximately 12, at average RTT approximately 13 ms. In multi-flow settings, Cocoa preserves fq’s equal-weight round-robin service so fairness indices remain near 14 across flows (Bachl et al., 2019).
4. Augmented state spaces in stochastic queueing control
In multiclass GI/M/n+M queues in an alternating renewal environment, augmentation enters through stochastic-process representation rather than through a deployed queue discipline. The model has class-15 arrivals
16
17 identical servers, i.i.d. 18 service times, i.i.d. 19 patience times, and an up/down indicator 20 under which service is suspended in the down-state but abandonment continues. The queue state satisfies
21
The motivation for augmentation is that 22 alone is not Markov. General renewal arrivals carry residual interarrival age information, and the alternating renewal environment carries residual down-time age. The paper therefore introduces interarrival ages
23
and down-time age
24
and defines the augmented state
25
Under a non-anticipative, preemptive, work-conserving policy 26, the process 27 is a strong Markov process.
The extended generator is written as
28
The arrival part 29 uses the hazard 30 of the renewal interarrival law. The service and abandonment part 31 includes service-departure jumps at rate 32 and abandonment jumps at rate 33 when 34, and only abandonment in the down-state. The environment part 35 encodes up 36 down jumps at rate 37 and down 38 up jumps through the conditioned down-time law.
Under Halfin–Whitt scaling,
39
and, jointly with tight admissible controls, the scaled process converges to a controlled jump diffusion: 40 Here 41, the drift is
42
and the jump term comes from a compound Poisson description of the down-time process. The generator of the scaled pure-state component converges on compact sets to the generator 43 of this limit SDE.
The augmentation is also indispensable for optimal control. For the discounted problem, the value function 44 solves
45
and for the ergodic problem one solves
46
Because the process counting the number of customers in each class is not Markov, the paper states that the usual martingale arguments for convergence of mean empirical measures cannot be applied; the obstacle is addressed by demonstrating the convergence of the generators of the augmented Markovian model and by proving long-run average moment bounds via Foster–Lyapunov equations for that model (Arapostathis et al., 2019).
5. Learning-augmented priority queues
Learning-augmented priority queues study a different queue abstraction: the priority queue as a data structure. The objective is to use potentially inaccurate predictions to enhance worst-case performance while preserving robustness when the prediction is arbitrarily bad. The paper examines three prediction models.
In the dirty comparisons model, a cheap dirty comparator may disagree with the true order. On inserting key 47 into queue 48, the error is
49
The goal is to minimize the number of clean comparisons as a function of 50. In the pointer predictions model, the oracle returns a pointer to 51, the predicted predecessor in a skip-list-based priority queue, and error is the rank distance
52
In the rank predictions model, each inserted key 53 receives a predicted global rank 54, with error
55
The implementation emphasis is on skip lists. Extract-Min is common to all three models: since the first level of the skip list is the full sorted list, extracting the minimum removes head→next at level 56, costing 57 time in expectation and using no predictions. For pointer prediction, insertion starts at the predicted node, performs a bottom-up walk followed by top-down descent, and has expected time
58
For dirty comparisons, the search uses dirty comparisons to stop at a predicted predecessor 59, then a clean exponential search spends 60 clean comparisons; total time remains 61. For rank predictions, the structure maintains both a skip list and a dynamic van Emde Boas tree on predicted ranks; insertion costs
62
amortized, with 63 clean comparisons.
Theoretical bounds are stated explicitly. Dirty comparisons support FindMin/ExtractMin in 64 expectation and Insert in 65 time with 66 dirty and 67 clean comparisons. Pointer predictions support FindMin/ExtractMin in 68 expectation and Insert in 69 expectation. Rank predictions support FindMin/ExtractMin in 70 amortized time and Insert in 71 amortized time. The paper also proves matching lower bounds, including that one needs 72 comparisons in the dirty-comparison and pointer-prediction settings if ExtractMin remains 73.
The applications section shows how the priority queue yields sorting bounds and a Dijkstra bound
74
with key-rank predictions. On real city-road graphs via OSMnx and Poisson–Voronoi random-planar graphs, the experiments report a 75–76 reduction in the number of expensive comparisons versus binary heaps once the predictions are of moderate quality. In this usage, augmentation refers to predictor access and error-sensitive complexity, rather than to explicit queue-length or buffer-length control (Benomar et al., 2024).
6. Comparative interpretation and recurring issues
Several recurrent themes appear across these works. First, augmentation is not merely “adding more buffer.” In Peacock, elasticity imposes automatic size-and-load bounds and rotates excess probes; in Cocoa, 77 can either increase or decrease; in the GI/M/n+M setting, augmentation enlarges the state space; in learning-augmented priority queues, augmentation changes the information available to queue operations rather than the storage budget.
Second, the added information is always tied to a robustness mechanism. Peacock couples runtime-aware reordering with the rule that any probe that reaches time 78 can no longer be bypassed by subsequent arrivals, guaranteeing starvation freedom. Cocoa combines bounded max-increase per Guard Interval, instant shrink by measured standing queue, and per-flow isolation. Learning-augmented priority queues explicitly retain classical worst-case guarantees when the hint is arbitrarily bad. The augmented GI/M/n+M model uses Foster–Lyapunov arguments and generator convergence to obtain asymptotic optimality and moment bounds.
Third, the target of augmentation differs by domain. In distributed data analytics, the target is low latency under heterogeneous workloads and mitigation of Head-of-Line blocking. In active queue management, the target is the utilization-delay trade-off for Reno, CUBIC, BBR, and related CCAs. In many-server queueing control, the target is a Markovian embedding that supports diffusion limits and HJB analysis. In data structures, the target is error-sensitive operation complexity.
A common misconception is to treat all of these as instances of the same algorithmic pattern. The data do not support that stronger claim. What they do support is a narrower and more precise one: each work starts from a conventional queue or queueing model and introduces extra state, measurements, or predictions so that the resulting system can exploit information not present in the baseline formulation. A plausible implication is that “augmented queue” is best understood as a family resemblance across scheduling, networking, stochastic control, and learning-augmented data structures, rather than as a single canonical definition.