2000 character limit reached
GVEL: Fast Graph Loading in Edgelist and Compressed Sparse Row (CSR) formats
Published 24 Nov 2023 in cs.PF | (2311.14650v5)
Abstract: Efficient IO techniques are crucial in high-performance graph processing frameworks like Gunrock and Hornet, as fast graph loading can help minimize processing time and reduce system/cloud usage charges. This research study presents approaches for efficiently reading an Edgelist from a text file and converting it to a Compressed Sparse Row (CSR) representation. On a server with dual 16-core Intel Xeon Gold 6226R processors and Seagate Exos 10e2400 HDDs, our approach, which we term as GVEL, outperforms Hornet, Gunrock, and PIGO by significant margins in CSR reading, exhibiting an average speedup of 78x, 112x, and 1.8x, respectively. For Edgelist reading, GVEL is 2.6x faster than PIGO on average, and achieves a Edgelist read rate of 1.9 billion edges/s. For every doubling of threads, GVEL improves performance at an average rate of 1.9x and 1.7x for reading Edgelist and reading CSR respectively.
- DaxVM: Stressing the Limits of Memory as a File Interface. In 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 369–387.
- The GAP benchmark suite. arXiv preprint arXiv:1508.03619 (2015).
- Hornet: An efficient data structure for dynamic sparse graphs and matrices on gpus. In 2018 IEEE High Performance extreme Computing Conference (HPEC). IEEE, 1–7.
- Transcending POSIX: The End of an Era? ; login: (2022).
- TriCache: A User-Transparent Block Cache Enabling High-Performance Out-of-Core Processing with In-Memory Programs. ACM Transactions on Storage 19, 2 (2023), 1–30.
- Kasimir Gabert and Ümit V Çatalyürek. 2021. PIGO: A parallel graph input/output library. In 2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 276–279.
- TurboGraph: a fast parallel graph engine handling billion-scale graphs in a single PC. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. 77–85.
- Satoshi Imamura and Eiji Yoshida. 2019. POSTER: AR-MMAP: Write Performance Improvement of Memory-Mapped File. In 2019 28th International Conference on Parallel Architectures and Compilation Techniques (PACT). IEEE, 493–494.
- The suitesparse matrix collection website interface. Journal of Open Source Software 4, 35 (2019), 1244.
- {{\{{GraphChi}}\}}:{{\{{Large-Scale}}\}} graph computation on just a {{\{{PC}}\}}. In 10th USENIX symposium on operating systems design and implementation (OSDI 12). 31–46.
- Virtual-Memory Assisted Buffer Management. Proceedings of the ACM on Management of Data 1, 1 (2023), 1–25.
- Userland CO-PAGER: boosting data-intensive applications with non-volatile memory, userspace paging. In Proceedings of the 3rd International Conference on High Performance Compilation, Computing and Communications. 78–83.
- Mmap: Fast billion-scale graph computation on a pc via memory mapping. In 2014 IEEE International Conference on Big Data (Big Data). IEEE, 159–164.
- HugeMap: Optimizing Memory-Mapped I/O with Huge Pages for Fast Storage. In Euro-Par 2020: Parallel Processing Workshops: Euro-Par 2020 International Workshops, Warsaw, Poland, August 24–25, 2020, Revised Selected Papers 26. Springer, 344–355.
- Bishard parallel processor: A disk-based processing engine for billion-scale graphs. International Journal of Multimedia and Ubiquitous Engineering 9, 2 (2014), 199–212.
- A lightweight infrastructure for graph analytics. In Proceedings of the twenty-fourth ACM symposium on operating systems principles. 456–471.
- Memory-mapped I/O on steroids. In Proceedings of the Sixteenth European Conference on Computer Systems. 277–293.
- Optimizing memory-mapped {{\{{I/O}}\}} for fast storage devices. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). 813–827.
- X-stream: Edge-centric graph processing using streaming partitions. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles. 472–488.
- Julian Shun and Guy E Blelloch. 2013. Ligra: a lightweight graph processing framework for shared memory. In Proceedings of the 18th ACM SIGPLAN symposium on Principles and practice of parallel programming. 135–146.
- Efficient memory-mapped I/O on fast storage device. ACM Transactions on Storage (TOS) 12, 4 (2016), 1–27.
- Low-latency memory-mapped i/o for data-intensive applications on fast storage devices. In 2012 SC Companion: High Performance Computing, Networking Storage and Analysis. IEEE, 766–770.
- NetworKit: A tool suite for large-scale complex network analysis. Network Science 4, 4 (2016), 508–530.
- DI-MMAP—a scalable memory-map runtime for out-of-core data-intensive applications. Cluster Computing 18 (2015), 15–28.
- Scaleg: A distributed disk-based system for vertex-centric graph processing. IEEE Transactions on Knowledge and Data Engineering 35, 2 (2021), 2019–2033.
- Gunrock: A high-performance graph processing library on the GPU. In Proceedings of the 21st ACM SIGPLAN symposium on principles and practice of parallel programming. 1–12.
- GraphBLAST: A high-performance linear algebra-based graph framework on the GPU. ACM Transactions on Mathematical Software (TOMS) 48, 1 (2022), 1–51.
- {{\{{EvFS}}\}}: User-level,{{\{{Event-Driven}}\}} File System for {{\{{Non-Volatile}}\}} Memory. In 11th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 19).
- {{\{{FlashGraph}}\}}: Processing {{\{{Billion-Node}}\}} graphs on an array of commodity {{\{{SSDs}}\}}. In 13th USENIX Conference on File and Storage Technologies (FAST 15). 45–58.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
