MultiQueue: Relaxed Queues and System Designs

• MultiQueue is a design pattern that uses an array of internal sequential queues with randomized operations to achieve relaxed concurrent priority scheduling.
• The approach trades strict ordering for improved throughput and controlled rank error, with tunable parameters like queue factor and candidate count.
• Engineering variants extend MultiQueue to systems, networking, and GPU processing, demonstrating enhanced performance and versatile applications.

“MultiQueue” most commonly denotes a relaxed concurrent priority queue built from many internal sequential priority queues, but the term also appears in networking, storage, and combinatorics to describe multiqueue telemetry, NIC and traffic-control schedulers, external-memory queue families, and multiline-queue constructions. Across these settings, the unifying idea is not a single canonical API but the replacement of one globally contended queue with several coordinated queues or queue-like layers, usually to trade exact ordering for scalability, locality, or domain-specific observability (Rihani et al., 2014, Williams et al., 15 Apr 2025, Bragatto et al., 16 May 2025).

1. Principal meanings of MultiQueue

In the concurrent-data-structure literature, MultiQueue is a relaxed concurrent priority queue for $p$ threads, implemented as an array of $cp$ internal sequential priority queues for a constant $c>1$. Insertions are randomized over the internal queues, while deletions inspect a small random sample and remove from the queue with the best visible minimum (Rihani et al., 2014). In later work, this design is treated as a flexible family rather than a single fixed structure, with queue factor, candidate count, buffering, batching, and locality policies all exposed as tuning parameters (Williams et al., 2021, Williams et al., 15 Apr 2025).

In networking and systems papers, “multiqueue” usually refers instead to multiple logical queues associated with a port, a NIC, or a GPU execution level. MM-INT uses multiqueue state in programmable switches to export queue telemetry from all logical queues on all ports (Bragatto et al., 16 May 2025). A priority-aware NIC design uses multiple receive queues as priority-separated interrupt domains for real-time embedded systems (Behnke et al., 2022). Linux mqprio is the multiqueue strict-priority qdisc used to map traffic classes to hardware transmit queues (Kirdan et al., 2023). Beyond systems engineering, external-memory multimaps use a multiqueue as a secondary structure for variable-length value sets (Angelino et al., 2011), and twisted multiline queues define projection algorithms for TASEP and TAZRP on a ring (Mandelshtam et al., 2024).

Usage	Object	Defining mechanism
Concurrent priority queue	Array of $cp$ sequential PQs	Random insert; best-of-two-or-more delete (Rihani et al., 2014)
Relaxed FIFO / GPU queueing	MultiFIFO, MLMQ	Timestamped sampled FIFO or multi-level GPU queues (Koch et al., 30 Jul 2025, Hu et al., 10 Feb 2026)
Network telemetry and scheduling	MM-INT, priority-aware NIC, mqprio	Per-port or per-queue visibility and differentiated queue policies (Bragatto et al., 16 May 2025, Behnke et al., 2022, Kirdan et al., 2023)
Storage and combinatorics	External-memory multiqueue, twisted multiline queues	Shared-block queue families or row-based queueing projections (Angelino et al., 2011, Mandelshtam et al., 2024)

2. Canonical meaning: relaxed concurrent priority queues

The canonical MultiQueue consists of an array

$Q[1],\dots,Q[cp]$

of internal sequential priority queues, each protected by a lock. An insertion repeatedly chooses a queue uniformly at random, tries to lock it, inserts the element, and unlocks. A deletion repeatedly samples two random queues, compares their stored minima without locking, chooses the queue with smaller observed minimum, locks it, performs deleteMin, and unlocks it. The purpose of the overprovisioning factor $c>1$ is to keep contention low: at most $p$ queues can be locked at once, so a random queue is unlocked with constant probability, yielding constant expected lock-acquisition overhead (Rihani et al., 2014).

The original design was motivated by the observation that exact concurrent priority queues concentrate contention near the global minimum. MultiQueue deliberately abandons exact deleteMin semantics: a deletion need only return an element close to the minimum. In this literature, closeness is measured by rank error, the rank of the deleted item among all current items, equivalently the number of smaller elements still present when it is removed. The 2014 paper analyzes a simplified model in which the $m$ current elements are independently and uniformly distributed across the $cp$ queues and no queue is locked. Under those assumptions,

$$\Pr[\mathrm{rank}=i] = \left(1-\frac{2}{cp}\right)^{i-1}\cdot\frac{2}{cp},$$

so the deleted rank is geometrically distributed, the expected rank is at most

$cp$0

and

$cp$1

This yields $cp$2 expected rank error and $cp$3 rank error with high probability in the simplified model (Rihani et al., 2014).

A second quality metric, emphasized in the later engineering paper, is delay: for an element $cp$4, the number of lower-priority elements deleted before $cp$5. Delay is distinct from rank error because an RPQ can return near-minimum elements while indefinitely postponing a particularly important one. Under the same simplified assumptions, delay is also geometric, with expected value $cp$6, and rank and delay are of the same order (Williams et al., 2021).

The design’s practical importance comes from the fact that the outer algorithm can use ordinary fast sequential queues internally. Both the original paper and the later engineering work use heap-based subqueues, especially 8-ary heaps, because they reduce cache misses relative to binary heaps. The 2021 and 2025 engineering papers add three orthogonal techniques: buffering with insertion and deletion buffers, bulk interaction with the internal sequential queues, and stickiness, where a thread reuses the same local candidate queues for multiple operations to improve locality (Williams et al., 2021, Williams et al., 15 Apr 2025).

3. Analysis, semantics, and exact rank laws

The theoretical status of MultiQueue evolved from simplified sequential approximations to exact stationary analysis and concurrent probabilistic correctness. One line of work introduced distributional linearizability as a correctness condition for randomized relaxed concurrent data structures. For a MultiQueue-style concurrent relaxed queue implemented as an array of $cp$7 linearizable sub-priority-queues, with enqueue to one random queue and dequeue from the better of two sampled queue minima, the paper proves distributional linearizability to a sequential randomized relaxed queue under an oblivious adversary. The resulting guarantee is that at each step the maximum expected rank of the dequeued element is $cp$8, and the average expected rank is $cp$9 (Alistarh et al., 2018).

A separate line gives an exact long-run analysis of the deletion-only model. In the 2024 analysis, a MultiQueue with $c>1$0 internal priority queues is studied after a random initialization of the infinite totally ordered set $c>1$1 into the queues, followed by deletions only. The key insight is to analyze the gaps between the ranks of current queue minima as a Markov chain on $c>1$2. Under the stability condition

$c>1$3

where $c>1$4 is the probability of deleting from the queue whose current minimum is the $c>1$5-th smallest among the queue minima, the stationary gap variables are independent geometric random variables. For the usual best-of-two MultiQueue,

$c>1$6

and the exact long-run expected rank error is

$c>1$7

This upgrades the earlier $c>1$8 result to an exact finite-$c>1$9 stationary formula (Walzer et al., 2024).

The same paper generalizes the analysis to deleting from the best of $cp$0 sampled queues. For non-integer $cp$1, the paper defines the sampling scheme by choosing $cp$2 queues with probability $cp$3 and $cp$4 otherwise, yielding

$cp$5

The expected stationary rank error remains linear in $cp$6, with explicit bounds and an exact expectation formula in terms of $cp$7 (Walzer et al., 2024).

The engineering literature adopts these analytical results as justification for a broader design point: deleting from one random queue is unstable, while deleting from the best of two random queues gives stable quality at low overhead. The 2025 engineering paper therefore treats $cp$8 as the default practical choice and makes $cp$9, queue factor $Q[1],\dots,Q[cp]$0, buffer sizes, and stickiness period $Q[1],\dots,Q[cp]$1 explicit throughput-quality control knobs (Williams et al., 15 Apr 2025).

4. Engineering variants and descendants

The most direct descendant is the Stealing Multi-Queue (SMQ). SMQ departs from the classical “global randomization on every operation” pattern by assigning each thread a local queue and making insertions local. Deletions are also usually local, but with probability $Q[1],\dots,Q[cp]$2 a thread samples another queue and steals a batch if that queue offers a better-priority task. The design combines queue affinity, stealing, and batching. Under the paper’s analytical model, with $Q[1],\dots,Q[cp]$3 threads/queues, batch size $Q[1],\dots,Q[cp]$4, scheduling imbalance parameter $Q[1],\dots,Q[cp]$5, and stealing probability $Q[1],\dots,Q[cp]$6, SMQ preserves nontrivial rank guarantees while reducing scheduler overhead. Empirically it surpasses schedulers such as Galois/OBIM and PMOD on priority-sensitive graph benchmarks, with improvements up to about $Q[1],\dots,Q[cp]$7 reported in the paper (Postnikova et al., 2021).

A second descendant is MultiFIFO, which adapts the MultiQueue idea from relaxed priority queues to relaxed FIFO queues. MultiFIFO replaces the internal sequential priority queues with in-place ring buffers, tags each element with its insertion timestamp, and makes pop sample two ring buffers and remove the older of the two head elements. The paper states that MultiFIFO inherits a linear-in-$Q[1],\dots,Q[cp]$8 expected rank error from MultiQueue,

$Q[1],\dots,Q[cp]$9

and rank errors in $c>1$0 with high probability, while retaining expected $c>1$1 operations. In the same paper, BlockFIFO is introduced as a separate ring-buffer-based relaxed FIFO. Both designs outperform strict and relaxed FIFO baselines on microbenchmarks and BFS, but MultiFIFO is the direct FIFO analogue of the MultiQueue pattern (Koch et al., 30 Jul 2025).

A third extension appears on GPUs as MultiLevelMultiQueue (MLMQ) for SSSP. MLMQ is not merely a set of multiple global queues; it is a three-level hierarchy aligned with GPU execution and memory hierarchies: L0 private queues in registers, L1 partially shared queues in shared memory, and L2 globally shared queues in global memory. Work moves between levels through a cache-like collaboration mechanism driven by Read and Write primitives: empty upper levels refill from lower levels, and full upper levels flush to lower levels. Among the L2 options is an L2 Multi-queue made of $c>1$2 global priority queues, but the paper’s main conclusion is that plain multiple global queues do not map well to GPU bottlenecks unless combined with L0/L1 buffering. Across three GPUs, MLMQ reports average speedups of $c>1$3 to $c>1$4 over prior SSSP implementations (Hu et al., 10 Feb 2026).

The 2025 engineering paper consolidates many of these ideas for the original concurrent priority-queue setting. It describes MultiQueue as a flexible approach using $c>1$5 internal sequential PQs, delete from the best of $c>1$6 sampled queues, and “wait-free locking,” then adds buffering, batch interaction with internal PQs, and locality-aware stickiness. On 128 threads, the paper identifies configurations ranging from “Strict” to “Fast,” and reports that, at similar rank error, MultiQueue can achieve $c>1$7 to $c>1$8 higher throughput than competitors on synthetic benchmarks, with strong application results on SSSP and branch-and-bound knapsack (Williams et al., 15 Apr 2025).

5. Networking and systems interpretations

In programmable-switch telemetry, “multiqueue” refers to the fact that each physical egress port may have several logical queues. MM-INT addresses the resulting observability problem: a conventional INT probe only sees the queue it traverses, so monitoring all queues by sending one probe per queue becomes prohibitive. MM-INT decouples telemetry collection from the traversed queue by storing telemetry metadata from all queues of all ports into registers in the egress pipeline; later probes read those registers and export a per-switch multiqueue snapshot. The paper compares three schemes and shows, in its evaluated topology, that $c>1$9 and $p$0 each require 12 probes while MM-INT requires only 3, with duplicate-probe counts of 12, 8, and 0 respectively. Total telemetry bytes are 2300 for $p$1, 2174 for $p$2, and 814 for MM-INT, summarized in the paper as approximately $p$3 less data transferred than traditional literature solutions (Bragatto et al., 16 May 2025).

In embedded networking, a priority-aware MultiQueue NIC design uses multiple receive queues as priority-separated interrupt domains rather than as throughput-oriented RSS channels. Packets are classified to per-process queues via a destination-port mapping installed by the OS and driver; each queue is then configured with priority-dependent interrupt moderation. Critical queues may generate immediate interrupts, while lower-priority queues use absolute or packet timers such as $p$4, $p$5, $p$6, or $p$7. In the paper’s simulated FreeRTOS/ESP32 setup, this yields a reduction of 93% of interrupts and an 80% decrease of processing delay of the critical task under the tested packet-flood conditions (Behnke et al., 2022).

In Linux TSN, “MultiQueue” denotes the mqprio qdisc. Here the mechanism is strict-priority forwarding via traffic classes mapped to NIC transmit queues, not a relaxed data structure. The paper on OPC UA over TSN treats mqprio as the Linux realization of IEEE 802.1Q/802.1Qav strict priority forwarding. In point-to-point experiments, MQPRIO achieves RTTs of 251 $p$8 on one setup and 250 $p$9 on another, with jitters of 0.999 $m$0 and 8.92 $m$1 respectively. In the bridged topology, the best overall result is obtained with ETF on the end hosts and MQPRIO on the bridge, with $m$2, RTT $m$3, and jitter $m$4 (Kirdan et al., 2023).

These networking meanings differ materially from the concurrent-priority-queue usage. In MM-INT, multiqueue is a visibility problem caused by multiple logical egress queues (Bragatto et al., 16 May 2025). In the NIC paper, multiqueue is a mechanism for priority-aware interrupt moderation (Behnke et al., 2022). In mqprio, it is a traffic-control and hardware-queue mapping abstraction (Kirdan et al., 2023). The commonality is architectural: multiple queues become first-class scheduling or measurement objects.

6. Other technical meanings

A distinct storage-oriented use appears in external-memory multimaps. There, a multiqueue is the secondary structure that stores the values associated with each key. Small queues share blocks, while large queues receive dedicated linked blocks, and an auxiliary dictionary $m$5 makes the structure location-aware. The paper’s theorem states that remove(x) and isMember(x) each use $m$6 I/Os, while enqueue(x,H) uses $m$7 expected I/Os, where $m$8 is the expected insertion cost in an external-memory cuckoo table of size $m$9. The full multimap uses $cp$0 blocks and supports findAll(k) in $cp$1 I/Os (Angelino et al., 2011).

A mathematically separate usage appears in twisted multiline queues for multispecies TASEP and 0-TAZRP on a ring. Here a fermionic multiline queue is a tuple of subsets $cp$2 and a bosonic multiline queue is a tuple of multisets $cp$3. “Twisted” means the shape is a composition rather than a partition. The paper defines projection algorithms with a pairing phase and a collapsing phase; in the fermionic case, the projection is

$cp$4

while the bosonic projection is

$cp$5

These maps are invariant under adjacent row swaps: $cp$6 which is the key twisted-queue theorem. On the bosonic side, the paper also defines a Markov chain on twisted multiline queues with stationary weight

$cp$7

and proves that it projects to the 0-TAZRP (Mandelshtam et al., 2024).

Taken together, these usages show that “MultiQueue” is best treated as a family resemblance rather than a single artifact. The dominant meaning remains the relaxed concurrent priority queue based on multiple internal sequential PQs and randomized best-of-two deletion (Rihani et al., 2014, Williams et al., 15 Apr 2025). Yet the same design intuition—replace one globally contended queue with multiple coordinated queues, or reinterpret queues as structured layers—recurs in telemetry, NIC scheduling, FIFO relaxation, GPU graph processing, external-memory indexing, and integrable probability (Bragatto et al., 16 May 2025, Behnke et al., 2022, Koch et al., 30 Jul 2025, Hu et al., 10 Feb 2026, Angelino et al., 2011, Mandelshtam et al., 2024).