Beyond a Single Queue: Multi-Level-Multi-Queue as an Effective Design for SSSP problems on GPUs

Abstract: As one of the most fundamental problems in graph processing, the Single-Source Shortest Path (SSSP) problem plays a critical role in numerous application scenarios. However, existing GPU-based solutions remain inefficient, as they typically rely on a single, fixed queue design that incurs severe synchronization overhead, high memory latency, and poor adaptivity to diverse inputs. To address these inefficiencies, we propose MultiLevelMultiQueue (MLMQ), a novel data structure that distributes multiple queues across the GPU's multi-level parallelism and memory hierarchy. To realize MLMQ, we introduce a cache-like collaboration mechanism for efficient inter-queue coordination, and develop a modular queue design based on unified Read and Write primitives. Within this framework, we expand the optimization space by designing a set of GPU-friendly queues, composing them across multiple levels, and further providing an input-adaptive MLMQ configuration scheme. Our MLMQ design achieves average speedups of 1.87x to 17.13x over state-of-the-art implementations. Our code is open-sourced at https://github.com/Leo9660/MLMQ.git.

• The paper introduces MLMQ, a hierarchical, adaptive queue system that reduces synchronization overhead and optimizes memory access for GPU-based SSSP.
• It employs multi-level queues across registers, shared, and global memory to balance parallelism with work efficiency.
• MLMQ achieves up to 17.13x speedups over baseline GPU algorithms and demonstrates broad applicability to other graph workloads like BFS.

Multi-Level-Multi-Queue (MLMQ): An Adaptive Queue Design for GPU SSSP

Motivation: SSSP Bottlenecks in GPU Queue Designs

The Single-Source Shortest Path (SSSP) problem is a cornerstone in graph algorithms, underpinning numerous applications across transportation, bioinformatics, circuit simulation, social networks, and recommender systems. While GPUs offer massive parallelism, traditional GPU-based SSSP implementations rely predominantly on a single, global concurrent queue. This fixed-type queue induces three major performance bottlenecks: excessive synchronization from high concurrency, severe bandwidth limitations owing to global memory access, and limited adaptability to heterogeneous graph inputs. Profiling indicates that atomic operation counts and global memory accesses per relaxation far exceed optimal work, especially for bucket queues (Figure 1).

Figure 1: Normalized performance profiling reveals synchronization and memory access overhead in single-queue designs far surpassing optimal work in Dijkstra’s algorithm.

Single-queue approaches are further challenged by their inability to balance parallelism and work efficiency: Dijkstra’s priority queue is optimal for work efficiency but serial, while Bellman-Ford’s FIFO maximizes parallelism at the expense of redundant relaxations. Intermediate designs like MultiQueue and $\Delta$-stepping exploit parallelism via bucketing but are not universally optimal for diverse graph types.

Additionally, naively partitioning workloads via multiple thread-private queues induces priority inversion, resulting in redundant wavefronts and degraded efficiency (Figure 2).

Figure 2: Redundant wavefronts from priority inversion when using local queues, generating excessive work.

MLMQ Framework: Multi-Level Hierarchical Queues

The paper introduces the Multi-Level-Multi-Queue (MLMQ), which orchestrates queue management across registers (L0), shared memory (L1), and global memory (L2), mapped to thread-private, warp-shared, and grid-shared usage, respectively (Figure 3, Figure 4). MLMQ leverages hierarchical placement to reduce contention, optimize memory access locality, and enable fine-grained control over queue policy at each hierarchy.

Figure 3: Conceptual comparison between a single concurrent queue and MLMQ’s distributed queues mapped to GPU thread hierarchy.

Figure 4: MLMQ architecture overview with modular queue instances across L0 (register), L1 (shared), and L2 (global) levels.

The interplay between queue levels employs cache-like collaboration, using unified Read and Write primitives across levels. This modularity allows for distinct queue implementations (FIFO, Near-far, Filter, Shortest-Length-First at L1; FIFO, Bucket, Priority, MultiQueue at L2), tailored to optimize either parallelism or priority guarantee (Figure 5).

Figure 5: Structural illustration of the L1 queue variants, demonstrating varying behaviors and trade-offs in priority guarantee.

Input-Adaptive Configuration and Learning-Based Selector

MLMQ recognizes the sensitivity of queue policy effectiveness to input graph structure and scale. The framework facilitates adaptive queue configuration using both heuristic rules and a trained RandomForest regressor, leveraging readily available graph features to select optimal queue type and parameters. This input-adaptive selection is validated on broad datasets, including 680 SuiteSparse graphs, achieving consistently near-optimal performance (Figure 6).

Figure 6: CDF of relative performance versus the best achievable per graph; adaptive configuration maintains high coverage near optimality.

Numerical Results and Performance Analysis

Comprehensive evaluation on three NVIDIA GPUs (RTX 3080Ti, Tesla A100, RTX 4090) and a spectrum of graph types demonstrates MLMQ’s average speedups of 1.87x–17.13x versus baseline GPU algorithms (ADDS, H-BF, Gunrock) and up to ~200x over CPU implementations. MLMQ consistently outperforms ADDS on all individual test graphs and achieves >2.3x average speedup on SuiteSparse matrices (Figure 7, Figure 8).

Figure 7: Performance comparison across SSSP implementations, showing MLMQ’s superiority across all three GPUs and graph types.

Figure 8: MLMQ’s speedup distribution against ADDS on SuiteSparse matrices, confirming broad generalizability.

Kernel runtime analysis shows that MLMQ either increases parallelism at small graph sizes (at minor additional work cost) or prioritizes work efficiency for larger graphs, adapting its configuration dynamically (Figure 9).

Figure 9: Runtime dynamics of ADDS vs. MLMQ on road networks; MLMQ flexibly balances parallelism and workload.

Normalized work and performance trends across graph categories further support the necessity of mixed queue designs and adaptive policies, with mesh/circuit and power-law graphs favoring distinct queue combinations (Figure 10).

Figure 10: Normalized performance and work for MLMQ configurations, revealing graph-type dependent trade-offs.

Specialized Queue Analysis and BFS Generalizability

CPU-effective priority queues (L2PQ, L2MPQ) exhibit subpar performance in high-concurrency GPU environments due to root contention; MLMQ’s hierarchical design alleviates this with distributed L0/L1 queues, resulting in significant speedups (Figure 11). Similarly, MLMQ generalizes efficiently to BFS workloads, outperforming Gunrock and ADDS by wide margins (Figure 12).

Figure 11: Concurrency-level sensitivity and mitigation of bottlenecks with L1 queue introduction in priority and multi-queue designs.

Figure 12: MLMQ achieves substantial speedups over state-of-the-art frameworks on BFS workloads.

Practical and Theoretical Implications

MLMQ shifts the paradigm from single global queues to modular, hierarchical, and adaptive queue composition, exploiting the GPU’s unique memory and parallelism structure. The design expands the optimization space for high-throughput, work-efficient graph processing, and demonstrates algorithm independence by efficiently generalizing beyond SSSP to BFS and other traversal algorithms.

Theoretical implications pertain to optimal scheduling strategies with hardware-induced constraints, and highlight the non-universality of traditional CPU-centric data structures such as priority or multi-queue schedulers for GPU workloads. Practically, MLMQ offers a foundation for robust, input-adaptive, high-performance graph processing on modern heterogeneous computing platforms.

Conclusion

MLMQ delivers a significant advancement in effective GPU queue management for SSSP and related graph traversal tasks, consistently outperforming prior state-of-the-art. Its cache-like collaboration, modularity, and input-adaptive configuration enable robust scalability, strong numerical results, and broader applicability in graph algorithm acceleration. The unified design paves the way for future work: extending MLMQ to multi-GPU distributed environments and other irregular graph workloads, promising further performance improvements and generality.