ACGraph: SSD-Optimized Graph Processing

• ACGraph is a high-throughput, asynchronous graph processing system that executes large-scale analytics on commodity hardware using SSD storage and limited DRAM.
• It employs a block-centric, event-driven architecture that fuses computation with dynamic scheduling and I/O to overcome read and work inflation.
• The system achieves significant speedups and reduced memory footprint compared to traditional synchronous, out-of-core frameworks, optimizing throughput and efficiency.

ACGraph is a high-throughput, asynchronous graph processing system tailored to execute large-scale graph analytics on a single machine equipped with SSD storage and limited DRAM. It replaces iteration-barrier, synchronous paradigms with a block-centric, event-driven architecture that tightly fuses computation, dynamic scheduling, and low-latency storage access. ACGraph addresses the read and work inflation characteristic of prior out-of-core systems and optimizes for both throughput and memory efficiency, enabling the processing of massive real-world graphs on commodity hardware (Chen et al., 11 Nov 2025).

1. Motivation and Systemic Challenges

The motivation for ACGraph lies in the limitations of existing out-of-core graph processing frameworks. As graph sizes routinely exceed available DRAM, traditional distributed solutions introduce significant network and coordination overheads, while early single-machine SSD-backed approaches (GraphChi, X-Stream, GridGraph) employ synchronous execution with entire-iteration barriers designed for HDDs. These legacy choices result in two central inefficiencies:

• Read inflation: Frequent reloading of 4 KB SSD blocks for vertex-centric updates, leading to destruction of temporal locality and repeated I/O even for edge-centric algorithms.
• Work inflation: Unnecessary computation from bulk activation patterns (e.g., label propagation), yielding superfluous edge traversals.
• Synchronization stalls: Under synchronous barriers, compute threads block on the completion of entire frontiers, leading to idle SSD bandwidth between iterations.

ACGraph proposes an asynchronous design to leverage SSD random-access bandwidth, collapse redundant block accesses, and fuse computation with continuous I/O, maximizing single-node graph analytic performance (Chen et al., 11 Nov 2025).

2. Core Architectural Principles

2.1 Block-Centric Asynchronous Execution

• Block as Scheduling Primitive: ACGraph partitions the graph into 4 KB blocks, each containing a disjoint set of vertices with all incident edge lists. Blocks are annotated with a 64-byte metadata structure, containing state flags, priority, and an Adaptive Frontier Set (AFS).
• Adaptive Frontier Set (AFS): Vertices active within a block are tracked either as a sparse array (≤11) or as a dense 360-bit bitmap, encoding activity efficiently per block.
• Block State Machine: Blocks transit between Inactive, Uncached, Cached, Processing, and Reactivated states—progressing through memory residency, SSD I/O, execution, and potential immediate reuse if further activations occur in-memory.
• Block Prioritization: A block’s processing priority is the aggregate (typically max or min) of its active vertices’ priorities, enabling dynamic, application-defined scheduling (e.g., BFS distance minimization, WCC label propagation) (Chen et al., 11 Nov 2025).

2.2 Online Asynchronous Worklist

• Dual-Queue Worklist: Maintains separate queues for cached (in-memory) and uncached (on SSD) active blocks, always prioritizing cache reuse and deferring I/O until strictly necessary.
• Atomic Updates: Block’s priority and AFS are updated atomically upon vertex activation, enabling real-time scheduling adjustments and in-memory state transition to the Reactivated state without SSD interaction for hot blocks.

2.3 Unified Pipelined I/O and Computation

• Thread Pool with io_uring Integration: Executors in a common thread pool issue non-blocking I/O (via io_uring), trigger background prefetch of high-priority blocks, and overlap computation with disk access, achieving sustained 4–5 GB/s throughput on hardware rated at 6 GB/s.
• Pipelined Event Loop: Each poll of the worklist can trigger I/O submission, handle completion events, process newly cached blocks, activate vertices, and immediately propagate further work (Chen et al., 11 Nov 2025).

2.4 Hybrid Storage Format for Low-Degree Vertices

• Degree-Aware Partitioning: After localized, window-based graph partitioning, vertices with degree ≤2 (“mini-vertices”) are separated for embedding in a compact mini_data array, indexed via auxiliary id arrays and eliminating explicit per-vertex degree fields.
• Storage Optimization: Large-degree vertices retain traditional CSR offset structures, while virtual boundary vertices restore the shifted offset invariant without direct per-vertex degree storage, reducing both RAM and SSD I/O overhead, especially effective in the presence of skewed degree distributions (Chen et al., 11 Nov 2025).

3. Algorithmic Framework and Supported Primitives

ACGraph exports a general-purpose apply/propagate API, suitable for implementing a wide array of block-centric graph algorithms. Five specific instantiations are highlighted:

Algorithm	State/Frontier Encoding	Priority Function
Breadth-First Search (BFS)	disv; initial s=0	max (distance layer)
Weakly Connected Components (WCC)	label[v]=v	min (label)
k-Core Decomposition	deg[v], activated when <k	constant or event
Personalized PageRank (PPR)	rv, pv	max (residual)
PageRank (PR)	as in PPR; initial r[v]=1/	V

Algorithms are encoded as follows:

• apply(u): Generates messages from a vertex’s state (e.g., dis[u], label[u], residual).
• propagate(msg,v): Attempts updates to neighbors, possibly using atomic compare-and-swap, returning a new priority if activation succeeds. The block scheduler leverages priority and locality, immediately exploiting reusable in-memory blocks and avoiding iteration-wide barriers (Chen et al., 11 Nov 2025).

4. Performance Characteristics and Evaluation

Extensive benchmarks on real graphs (Twitter, Friendster, UK-Union, ClueWeb12) and comparison against Blaze (2022) and CAVE (2024) under 256 MB RAM demonstrate the following:

• Runtime Acceleration: ACGraph achieves 3.8×–15.2× speedups over Blaze and CAVE across BFS, WCC, k-Core, PPR, and PR. k-Core shows the largest gain (15.2× Blaze, 8.8× CAVE).
• I/O Efficiency: Reduces bytes read per edge in BFS to <7 B/edge (down to 4.8 B/edge on Friendster), outperforming Blaze (≥8.1 B) and CAVE (≥8.6 B). In WCC, priority scheduling lowers work inflation by up to 60%.
• Throughput and Scalability: Maintains 4–5 GB/s SSD throughput (vs. Blaze’s ~2 GB/s; CAVE <1 GB/s), scaling nearly linearly to 64 threads, limited only by disk bandwidth.
• Memory Footprint: Uses 20–30% less RAM than Blaze and CAVE on Twitter, Friendster, and UK-Union; even on ClueWeb12 (978M vertices), metadata growth does not offset the aggregate speedup (15×).
• Parameter Robustness: Performance is relatively insensitive to buffer pool size (1–16% of graph) and mini-vertex degree threshold (default d=2 optimal) (Chen et al., 11 Nov 2025).

Speedups arise from three design choices: elimination of iteration barriers, aggressive merging/reuse of loaded blocks, and dynamic priority-based execution.

5. Systemic Impact and Positioning

ACGraph redefines the efficiency frontier for single-machine, SSD-optimized, out-of-core graph analytics. Its event-driven, block-centric asynchronous model decouples I/O and computation, exploiting SSD hardware characteristics. By fusing fine-grained priority scheduling at the block level with cache-aware work coalescence and storage-aware data layouts, ACGraph dramatically reduces both I/O and unnecessary computation compared to synchronous iteration-barrier frameworks. This results in a system that approaches the SSD hardware's theoretical bandwidth, even on billion-edge graphs (Chen et al., 11 Nov 2025).

A plausible implication is that, as graph datasets continue to outpace DRAM growth while SSDs offer ever-higher parallelism, the ACGraph design paradigm will be essential for scalable, low-latency, high-throughput graph computation on commodity hardware.

6. Limitations and Prospective Extensions

While ACGraph’s architecture excels on SSD-backed, single-node analytics, metadata overhead increases with graph scale (notably on the ClueWeb12 dataset). Further research may focus on minimizing such overhead, adaptive degree thresholds, and workload-specific block partitioning. As ACGraph does not address distributed or NUMA architectures directly, future work may extend core principles—block-centric asynchrony, prioritized worklists, I/O-compute pipelining—to networked or hierarchical storage environments. Integration of advanced SSD features (e.g., native computation, zoned namespaces) and automatic block partitioning could further optimize cost and throughput (Chen et al., 11 Nov 2025).

• "ACGraph: An Efficient Asynchronous Out-of-Core Graph Processing Framework" (Chen et al., 11 Nov 2025)