MultiLevelMultiQueue (MLMQ) in GPU SSSP

• MultiLevelMultiQueue (MLMQ) is a hierarchical queue design that employs three levels—thread-private, warp-shared, and device-wide—to optimize GPU single-source shortest path (SSSP) computations.
• It integrates unified Read/Write primitives and a cache-like collaboration mechanism to efficiently manage local buffering and reduce synchronization overhead.
• An adaptive configuration using a RandomForest regressor tailors queue composition to graph characteristics, balancing parallelism with ordering quality for improved performance.

MultiLevelMultiQueue (MLMQ) denotes, in its most specific current usage, a hierarchical queue data structure for GPU single-source shortest path (SSSP) that “distributes multiple queues across the GPU’s multi-level parallelism and memory hierarchy” (Hu et al., 10 Feb 2026). In this formulation, MLMQ is neither a single global worklist nor a purely flat relaxed multi-queue. Rather, it is a three-level composition of thread-private, warp-shared, and device-wide queues coordinated by unified Read and Write primitives and a cache-like collaboration mechanism. The term also sits at the intersection of older research traditions: flat relaxed MultiQueues for concurrent priority scheduling, multi-level priority queues with shared servers in queueing theory, and queue-length-dependent multi-level single-server models (Rihani et al., 2014, Zuk et al., 2023, Kobayashi et al., 6 May 2025).

1. Terminological scope and antecedents

The name MultiLevelMultiQueue is explicitly introduced for GPU SSSP in “Beyond a Single Queue: Multi-Level-Multi-Queue as an Effective Design for SSSP problems on GPUs” (Hu et al., 10 Feb 2026). Earlier work on MultiQueues did not use the MLMQ term and did not introduce a hierarchical queue-of-queues structure. Instead, those papers studied a flat relaxed concurrent priority queue built from many ordinary sequential priority queues, with randomized insertion and sampled deletion; the key idea was to relax global ordering to reduce contention (Rihani et al., 2014). This flat design is therefore a conceptual precursor to MLMQ, but not a multilevel instance of it.

A separate queueing-theoretic usage of “multi-level” refers to analytically tractable priority or regime-switching systems rather than GPU worklists. One line studies a non-preemptive $M/M/c$ queue with an arbitrary number of priority levels, where each priority level behaves as a queue class sharing a common server pool (Zuk et al., 2023). Another studies a single FCFS queue whose arrival and service speeds change with queue-length-defined levels, explicitly describing it as a multi-level queue rather than a network of multiple physical queues (Kobayashi et al., 6 May 2025). These models are relevant for interpretation, but they are structurally different from the GPU MLMQ hierarchy.

This terminological separation matters because the 2026 MLMQ design is fundamentally architectural. Its “levels” correspond to GPU execution scopes and memory tiers, not only to logical priority classes. By contrast, the flat MultiQueue literature emphasizes relaxed delete-min quality under concurrency, while the queueing literature emphasizes stationary distributions, waiting times, and heavy-traffic limits.

2. GPU SSSP motivation and hierarchical organization

The GPU MLMQ paper places queue design at the center of SSSP performance. It formulates SSSP on a weighted graph $G(V,E,W)$ as repeated relaxation over active vertices, with shortest-path distances from source $s$ written as

$L(s) = \{ l_{su} \mid u \in V \},$

and outgoing neighbors

$N(u)=\{v \mid e_{uv}\in E\}.$

The queue determines both available parallelism and the amount of redundant relaxation work (Hu et al., 10 Feb 2026).

The paper frames Dijkstra and Bellman-Ford/FIFO as two extremes. Dijkstra offers strong priority ordering and near-optimal work efficiency but poor parallelism, whereas Bellman-Ford/FIFO provides weak ordering, high parallelism, and much redundant work. Prior GPU systems are criticized for relying on a single, fixed queue in global memory, which incurs severe synchronization overhead, high memory latency, and poor adaptivity to diverse inputs. Profiling in the paper reports atomic operations reaching up to $11.8\times$ the optimal work for FIFO queues and up to $99.1\times$ for bucket queues, while queue-related global-memory accesses can exceed effective work by up to four orders of magnitude (Hu et al., 10 Feb 2026).

MLMQ addresses this by distributing queue state across three levels matched to GPU hardware:

Level	Scope and storage	Representative role
L0	thread-private, registers	fastest local buffer
L1	warp-shared, shared memory	medium-capacity collaborative queue
L2	device-wide, global memory	backing store and global coordination

In the paper’s default organization, L0 is a private vector queue in registers, L1 is a warp-shared queue in shared memory, and L2 is a globally shared queue in global memory. This mapping is the defining architectural move of MLMQ: registers provide the cheapest access but smallest capacity, shared memory enables low-latency warp-level collaboration, and global memory supplies device-wide visibility and capacity (Hu et al., 10 Feb 2026).

3. Read–write semantics, cache-like collaboration, and queue families

MLMQ is exposed to the SSSP kernel through unified Read and Write primitives. Read attempts to obtain work from L0, then L1, then L2; if all three are empty, it returns READ_EMPTY. Write first inserts into L0; if L0 overflows it performs a batched write-back to L1, and if L1 overflows the batch is written onward to L2 (Hu et al., 10 Feb 2026). This creates a software-managed cache hierarchy for worklist elements: upper levels buffer hot items, lower levels provide capacity and global visibility.

The collaboration policy is intentionally limited to adjacent levels. L0 exchanges only with L1, and L1 exchanges only with L2. The paper explicitly argues that CPU-style work stealing is ill-suited to GPUs because inter-warp and inter-block communication is expensive; a cache-style protocol better matches GPU hardware. Refills occur on underflow and flushes occur on overflow, with batched transfers used to reduce warp divergence, improve memory coalescing, and amortize synchronization (Hu et al., 10 Feb 2026).

Within this framework the paper defines several queue families. L0 is fixed as a vector queue. L1 offers multiple shared-memory designs: L1V (FIFO vector), L1NF (near-far queue with threshold $NF$), L1FQ (filter queue with threshold $F$), and L1SLF (shortest-length-first queue). L2 offers global-memory designs: L2V (block-based FIFO vector), L2DQ ($\Delta$-stepping-style bucket queue), L2PQ (priority queue), and L2MPQ (multi-queue priority structure). The bucket queue’s $G(V,E,W)$0-th bucket holds priorities in

$G(V,E,W)$1

When the first bucket becomes empty after a Read, $G(V,E,W)$2 is increased by $G(V,E,W)$3 (Hu et al., 10 Feb 2026).

A central empirical point is that queue strength and cost trade off against each other. The paper characterizes queue families by priority guarantee versus access efficiency: vector queues are cheap but weakly ordered, priority queues are strongly ordered but expensive, and mixed designs often achieve the best balance. It also notes that naïvely adding local queues can worsen performance, because priority inversion in local storage may generate “redundant wavefronts.” L1V therefore uses periodic flushing after every $G(V,E,W)$4 invocations of Write, and stronger L1 variants such as L1NF, L1FQ, and L1SLF attempt to restore enough order to suppress redundant work (Hu et al., 10 Feb 2026).

4. Integration with the SSSP kernel and adaptive configuration

The implementation couples an MLMQ manager with an SSSP worker under persistent-thread execution. A work kernel repeatedly performs q->Read(u, valid), discards stale items via the check u.dist > dist[u.id], relaxes outgoing edges, and reinserts improved vertices through q->Write(new_v, update) (Hu et al., 10 Feb 2026). Load-balanced edge traversal is controlled by a degree threshold

$G(V,E,W)$5

on Ampere GPUs: vertices above the threshold are processed cooperatively by a warp, whereas smaller-degree work is distributed across threads.

Termination detection is nontrivial because L0 and L1 may contain work not yet visible in L2. The paper therefore maintains global_reserve and global_done, updating them only through L2 interactions. When L2 writes occur, global_reserve increases; when L2 reads succeed, the number of consumed items is accumulated in warp-local state and later committed to global_done when an L2 read returns READ_EMPTY. The queue hierarchy is considered empty when

$G(V,E,W)$6

This delayed accounting is the paper’s mechanism for reconciling local buffering with device-wide termination (Hu et al., 10 Feb 2026).

Because no fixed queue composition dominates on all graphs, the paper adds an input-adaptive configuration scheme based on a RandomForest regressor. At preprocessing time it extracts graph features including the number of vertices $G(V,E,W)$7, number of edges $G(V,E,W)$8, average and maximum degree, degree standard deviation, and average, standard deviation, and maximum edge weight. These features are used to predict both the relative performance of candidate MLMQ configurations and configuration parameters such as $G(V,E,W)$9 and $s$0 (Hu et al., 10 Feb 2026).

The resulting choices are graph-dependent. On the reported workloads, small road networks favor L1V-L2V, medium roads favor L1NF-L2V, larger roads favor L1V-L2DQ, meshes and circuits favor L1FQ-L2V, and power-law graphs favor L1SLF-L2DQ. The paper attributes these differences to sensitivity to priority inversion, degree skew, and graph size. Power-law graphs are said to be highly sensitive to priority inversion, whereas meshes are less sensitive; larger graphs benefit more from work-efficient queues, and smaller graphs benefit more from parallel-friendly ones (Hu et al., 10 Feb 2026).

5. Relation to flat MultiQueues and analytical queueing models

Although MLMQ is specialized to GPU SSSP, its design can be situated relative to two adjacent bodies of work. The first is the relaxed concurrent MultiQueue literature. The original MultiQueue maintains an array of $s$1 sequential priority queues for $s$2 threads, inserts by random placement, and deletes by sampling two queues and removing from the one with the smaller minimum (Rihani et al., 2014). Its quality metric is rank error, and under a simplified model the expected deleted rank is approximately $s$3. A later exact stationary analysis for the flat deletion-only model derives the long-term expected rank error

$s$4

for the ordinary 2-choice MultiQueue, and generalizes the analysis to best-of-$s$5 deletion schemes (Walzer et al., 2024). These results establish the multi-queue principle—distributed local queues plus limited-choice selection—but they do so in a flat, exchangeable setting.

The second adjacent body of work is classical multi-level queueing theory. The non-preemptive multi-server multi-level Markovian priority queue models $s$6 priority levels sharing $s$7 servers and derives explicit stationary balance equations, multivariate PGFs, and PMFs for the joint queue-length vector (Zuk et al., 2023). The multi-level single-server queue with queue-length-dependent arrival and service clocks studies a different notion of “multi-level,” namely regime switching over state-space intervals, and derives a heavy-traffic reflected diffusion together with a closed-form limiting stationary density (Kobayashi et al., 6 May 2025). These models are mathematically rich but are not hierarchical GPU worklists.

MLMQ can therefore be read as combining ideas that were previously separate. From flat MultiQueues it inherits the intuition that multiple queues mitigate contention and permit controlled relaxation. From queueing theory it inherits the broader idea that levels alter service behavior and waiting-time structure. The 2026 GPU paper’s distinctive step is to realize these principles across hardware levels—thread, warp, and device—while preserving a modular interface and an adaptive configuration space.

6. Performance claims, observed trade-offs, and limitations

The paper reports that MLMQ achieves average speedups of 1.87x to 17.13x over state-of-the-art GPU SSSP implementations (Hu et al., 10 Feb 2026). On RTX 3080 Ti, the reported averages are 1.87x over ADDS, 15.08x over H-BF, and 17.13x over Gunrock. On Tesla A100, they are 2.67x, 12.54x, and 11.31x respectively; on RTX 4090, 1.94x, 14.92x, and 16.81x. Against CPU baselines, the reported speedups are above 100x on the tested platforms. On 680 SuiteSparse matrices, MLMQ is said to speed up more than 91.5% of matrices over ADDS, with average speedup exceeding 2.3x (Hu et al., 10 Feb 2026).

The performance explanation is explicitly two-dimensional. On some inputs MLMQ increases active warps and tolerates more redundant work because lower queue overhead dominates. On others it reduces active warps but lowers total workload enough to produce a net gain. For the small road network NY, the paper reports a 1.53x improvement in parallelism, a 1.7x increase in redundant work, and a net 2.06x speedup over ADDS. For the large road network USA, it reports active warps reduced to 0.27x of ADDS, total workload reduced by 3.5x, and a net 1.56x speedup. This is presented as evidence that MLMQ succeeds not by maximizing either raw parallelism or strict work efficiency, but by balancing the two (Hu et al., 10 Feb 2026).

The adaptive selector is also quantitatively important. On SuiteSparse, it improves over the best fixed configuration by over $s$8 on average, and under a relative-performance threshold of 0.9 it reaches 93.9% near-optimal coverage, versus 64.6% for the best fixed queue configuration, L1V-L2V (Hu et al., 10 Feb 2026).

The paper is nevertheless careful about trade-offs. Local queues can hurt performance if they exacerbate priority inversion; private local queues on a bucket queue can degrade performance by up to 31% due to up to 45% additional work. CPU-effective GPU-global priority queues and multi-queues are also not universally successful: even when L0 and L1 buffering improve L2PQ or L2MPQ by up to 6.40x in one case study, those designs still tend to be less competitive than GPU-friendly L2V and L2DQ because of global-memory and root-node costs (Hu et al., 10 Feb 2026). A plausible implication is that MLMQ’s contribution lies less in any single queue algorithm than in the composition rule itself: fast local buffering, limited cross-level coordination, and graph-adaptive mixing of weak and strong ordering.

In that sense, MLMQ marks a shift from queue design as an isolated data-structure problem to queue design as a hierarchical systems problem. The paper’s central claim is not simply that “more queues” are better, but that GPU SSSP requires multiple queue types distributed across multiple hardware levels, with the hierarchy treated as an active control surface for parallelism, latency, and work efficiency (Hu et al., 10 Feb 2026).