---
title: MultiLevelMultiQueue (MLMQ) in GPU SSSP
url: https://www.emergentmind.com/topics/multilevelmultiqueue-mlmq
type: topic
---

# MultiLevelMultiQueue (MLMQ) in GPU SSSP

MultiLevelMultiQueue (MLMQ) denotes, in its most specific current usage, a hierarchical queue data structure for GPU single-source shortest path (SSSP) that “distributes multiple queues across the GPU’s multi-level parallelism and memory hierarchy” [2602.10080]. In this formulation, MLMQ is neither a single global worklist nor a purely flat relaxed multi-queue. Rather, it is a three-level composition of thread-private, warp-shared, and device-wide queues coordinated by unified `Read` and `Write` primitives and a cache-like collaboration mechanism. The term also sits at the intersection of older research traditions: flat relaxed MultiQueues for concurrent priority scheduling, multi-level priority queues with shared servers in queueing theory, and queue-length-dependent multi-level single-server models [1411.1209] [2311.01641] [2505.03504].

## 1. Terminological scope and antecedents

The name **MultiLevelMultiQueue** is explicitly introduced for GPU SSSP in “Beyond a Single Queue: Multi-Level-Multi-Queue as an Effective Design for SSSP problems on GPUs” [2602.10080]. Earlier work on **MultiQueues** did not use the MLMQ term and did not introduce a hierarchical queue-of-queues structure. Instead, those papers studied a flat relaxed concurrent priority queue built from many ordinary sequential priority queues, with randomized insertion and sampled deletion; the key idea was to relax global ordering to reduce contention [1411.1209]. This flat design is therefore a conceptual precursor to MLMQ, but not a multilevel instance of it.

A separate queueing-theoretic usage of “multi-level” refers to analytically tractable priority or regime-switching systems rather than GPU worklists. One line studies a non-preemptive \(M/M/c\) queue with an arbitrary number of priority levels, where each priority level behaves as a queue class sharing a common server pool [2311.01641]. Another studies a single FCFS queue whose arrival and service speeds change with queue-length-defined levels, explicitly describing it as a multi-level queue rather than a network of multiple physical queues [2505.03504]. These models are relevant for interpretation, but they are structurally different from the GPU MLMQ hierarchy.

This terminological separation matters because the 2026 MLMQ design is fundamentally architectural. Its “levels” correspond to GPU execution scopes and memory tiers, not only to logical priority classes. By contrast, the flat MultiQueue literature emphasizes relaxed delete-min quality under concurrency, while the queueing literature emphasizes stationary distributions, waiting times, and heavy-traffic limits.

## 2. GPU SSSP motivation and hierarchical organization

The GPU MLMQ paper places queue design at the center of SSSP performance. It formulates SSSP on a weighted graph \(G(V,E,W)\) as repeated relaxation over active vertices, with shortest-path distances from source \(s\) written as
\[
L(s) = \{ l_{su} \mid u \in V \},
\]
and outgoing neighbors
\[
N(u)=\{v \mid e_{uv}\in E\}.
\]
The queue determines both available parallelism and the amount of redundant relaxation work [2602.10080].

The paper frames **Dijkstra** and **Bellman-Ford/FIFO** as two extremes. Dijkstra offers strong priority ordering and near-optimal work efficiency but poor parallelism, whereas Bellman-Ford/FIFO provides weak ordering, high parallelism, and much redundant work. Prior GPU systems are criticized for relying on a single, fixed queue in global memory, which incurs severe synchronization overhead, high memory latency, and poor adaptivity to diverse inputs. Profiling in the paper reports atomic operations reaching up to **\(11.8\times\)** the optimal work for FIFO queues and up to **\(99.1\times\)** for bucket queues, while queue-related global-memory accesses can exceed effective work by **up to four orders of magnitude** [2602.10080].

MLMQ addresses this by distributing queue state across three levels matched to GPU hardware:

| Level | Scope and storage | Representative role |
|---|---|---|
| **L0** | thread-private, registers | fastest local buffer |
| **L1** | warp-shared, shared memory | medium-capacity collaborative queue |
| **L2** | device-wide, global memory | backing store and global coordination |

In the paper’s default organization, L0 is a private vector queue in registers, L1 is a warp-shared queue in shared memory, and L2 is a globally shared queue in global memory. This mapping is the defining architectural move of MLMQ: registers provide the cheapest access but smallest capacity, shared memory enables low-latency warp-level collaboration, and global memory supplies device-wide visibility and capacity [2602.10080].

## 3. Read–write semantics, cache-like collaboration, and queue families

MLMQ is exposed to the SSSP kernel through unified `Read` and `Write` primitives. `Read` attempts to obtain work from L0, then L1, then L2; if all three are empty, it returns `READ_EMPTY`. `Write` first inserts into L0; if L0 overflows it performs a batched write-back to L1, and if L1 overflows the batch is written onward to L2 [2602.10080]. This creates a software-managed cache hierarchy for worklist elements: upper levels buffer hot items, lower levels provide capacity and global visibility.

The collaboration policy is intentionally limited to adjacent levels. L0 exchanges only with L1, and L1 exchanges only with L2. The paper explicitly argues that CPU-style work stealing is ill-suited to GPUs because inter-warp and inter-block communication is expensive; a cache-style protocol better matches GPU hardware. Refills occur on underflow and flushes occur on overflow, with batched transfers used to reduce warp divergence, improve memory coalescing, and amortize synchronization [2602.10080].

Within this framework the paper defines several queue families. L0 is fixed as a vector queue. L1 offers multiple shared-memory designs: **L1V** (FIFO vector), **L1NF** (near-far queue with threshold \(NF\)), **L1FQ** (filter queue with threshold \(F\)), and **L1SLF** (shortest-length-first queue). L2 offers global-memory designs: **L2V** (block-based FIFO vector), **L2DQ** (\(\Delta\)-stepping-style bucket queue), **L2PQ** (priority queue), and **L2MPQ** (multi-queue priority structure). The bucket queue’s \(i\)-th bucket holds priorities in
\[
[base + \Delta * (i-1),\; base + \Delta * i).
\]
When the first bucket becomes empty after a `Read`, \(base\) is increased by \(\Delta\) [2602.10080].

A central empirical point is that queue strength and cost trade off against each other. The paper characterizes queue families by **priority guarantee** versus **access efficiency**: vector queues are cheap but weakly ordered, priority queues are strongly ordered but expensive, and mixed designs often achieve the best balance. It also notes that naïvely adding local queues can worsen performance, because priority inversion in local storage may generate “redundant wavefronts.” L1V therefore uses periodic flushing after every \(wb\) invocations of `Write`, and stronger L1 variants such as L1NF, L1FQ, and L1SLF attempt to restore enough order to suppress redundant work [2602.10080].

## 4. Integration with the SSSP kernel and adaptive configuration

The implementation couples an **MLMQ manager** with an **SSSP worker** under persistent-thread execution. A work kernel repeatedly performs `q->Read(u, valid)`, discards stale items via the check `u.dist > dist[u.id]`, relaxes outgoing edges, and reinserts improved vertices through `q->Write(new_v, update)` [2602.10080]. Load-balanced edge traversal is controlled by a degree threshold
\[
TH\_V = 16
\]
on Ampere GPUs: vertices above the threshold are processed cooperatively by a warp, whereas smaller-degree work is distributed across threads.

Termination detection is nontrivial because L0 and L1 may contain work not yet visible in L2. The paper therefore maintains `global_reserve` and `global_done`, updating them only through L2 interactions. When L2 writes occur, `global_reserve` increases; when L2 reads succeed, the number of consumed items is accumulated in warp-local state and later committed to `global_done` when an L2 read returns `READ_EMPTY`. The queue hierarchy is considered empty when
\[
global\_reserve = global\_done.
\]
This delayed accounting is the paper’s mechanism for reconciling local buffering with device-wide termination [2602.10080].

Because no fixed queue composition dominates on all graphs, the paper adds an input-adaptive configuration scheme based on a **RandomForest regressor**. At preprocessing time it extracts graph features including the number of vertices \(\text{m}\), number of edges \(\text{nnz}\), average and maximum degree, degree standard deviation, and average, standard deviation, and maximum edge weight. These features are used to predict both the relative performance of candidate MLMQ configurations and configuration parameters such as \(NF\) and \(F\) [2602.10080].

The resulting choices are graph-dependent. On the reported workloads, small road networks favor **L1V-L2V**, medium roads favor **L1NF-L2V**, larger roads favor **L1V-L2DQ**, meshes and circuits favor **L1FQ-L2V**, and power-law graphs favor **L1SLF-L2DQ**. The paper attributes these differences to sensitivity to priority inversion, degree skew, and graph size. Power-law graphs are said to be highly sensitive to priority inversion, whereas meshes are less sensitive; larger graphs benefit more from work-efficient queues, and smaller graphs benefit more from parallel-friendly ones [2602.10080].

## 5. Relation to flat MultiQueues and analytical queueing models

Although MLMQ is specialized to GPU SSSP, its design can be situated relative to two adjacent bodies of work. The first is the relaxed concurrent **MultiQueue** literature. The original MultiQueue maintains an array of \(cp\) sequential priority queues for \(p\) threads, inserts by random placement, and deletes by sampling two queues and removing from the one with the smaller minimum [1411.1209]. Its quality metric is rank error, and under a simplified model the expected deleted rank is approximately \(\frac{c}{2}p\). A later exact stationary analysis for the flat deletion-only model derives the long-term expected rank error
\[
\tfrac{5}{6}n - 1 + \tfrac{1}{6n}
\]
for the ordinary 2-choice MultiQueue, and generalizes the analysis to best-of-\(c\) deletion schemes [2410.08714]. These results establish the multi-queue principle—distributed local queues plus limited-choice selection—but they do so in a flat, exchangeable setting.

The second adjacent body of work is classical multi-level queueing theory. The non-preemptive multi-server multi-level Markovian priority queue models \(K\) priority levels sharing \(c\) servers and derives explicit stationary balance equations, multivariate PGFs, and PMFs for the joint queue-length vector [2311.01641]. The multi-level single-server queue with queue-length-dependent arrival and service clocks studies a different notion of “multi-level,” namely regime switching over state-space intervals, and derives a heavy-traffic reflected diffusion together with a closed-form limiting stationary density [2505.03504]. These models are mathematically rich but are not hierarchical GPU worklists.

MLMQ can therefore be read as combining ideas that were previously separate. From flat MultiQueues it inherits the intuition that multiple queues mitigate contention and permit controlled relaxation. From queueing theory it inherits the broader idea that levels alter service behavior and waiting-time structure. The 2026 GPU paper’s distinctive step is to realize these principles across hardware levels—thread, warp, and device—while preserving a modular interface and an adaptive configuration space.

## 6. Performance claims, observed trade-offs, and limitations

The paper reports that MLMQ achieves average speedups of **1.87x to 17.13x** over state-of-the-art GPU SSSP implementations [2602.10080]. On **RTX 3080 Ti**, the reported averages are **1.87x** over ADDS, **15.08x** over H-BF, and **17.13x** over Gunrock. On **Tesla A100**, they are **2.67x**, **12.54x**, and **11.31x** respectively; on **RTX 4090**, **1.94x**, **14.92x**, and **16.81x**. Against CPU baselines, the reported speedups are above **100x** on the tested platforms. On **680 SuiteSparse matrices**, MLMQ is said to speed up **more than 91.5%** of matrices over ADDS, with average speedup exceeding **2.3x** [2602.10080].

The performance explanation is explicitly two-dimensional. On some inputs MLMQ increases active warps and tolerates more redundant work because lower queue overhead dominates. On others it reduces active warps but lowers total workload enough to produce a net gain. For the small road network NY, the paper reports a **1.53x** improvement in parallelism, a **1.7x** increase in redundant work, and a net **2.06x** speedup over ADDS. For the large road network USA, it reports active warps reduced to **0.27x** of ADDS, total workload reduced by **3.5x**, and a net **1.56x** speedup. This is presented as evidence that MLMQ succeeds not by maximizing either raw parallelism or strict work efficiency, but by balancing the two [2602.10080].

The adaptive selector is also quantitatively important. On SuiteSparse, it improves over the best fixed configuration by **over \(1.1\times\)** on average, and under a relative-performance threshold of 0.9 it reaches **93.9%** near-optimal coverage, versus **64.6%** for the best fixed queue configuration, L1V-L2V [2602.10080].

The paper is nevertheless careful about trade-offs. Local queues can hurt performance if they exacerbate priority inversion; private local queues on a bucket queue can degrade performance by **up to 31%** due to **up to 45% additional work**. CPU-effective GPU-global priority queues and multi-queues are also not universally successful: even when L0 and L1 buffering improve L2PQ or L2MPQ by up to **6.40x** in one case study, those designs still tend to be less competitive than GPU-friendly L2V and L2DQ because of global-memory and root-node costs [2602.10080]. A plausible implication is that MLMQ’s contribution lies less in any single queue algorithm than in the composition rule itself: fast local buffering, limited cross-level coordination, and graph-adaptive mixing of weak and strong ordering.

In that sense, MLMQ marks a shift from queue design as an isolated data-structure problem to queue design as a hierarchical systems problem. The paper’s central claim is not simply that “more queues” are better, but that GPU SSSP requires multiple queue types distributed across multiple hardware levels, with the hierarchy treated as an active control surface for parallelism, latency, and work efficiency [2602.10080].

Source: https://www.emergentmind.com/topics/multilevelmultiqueue-mlmq