CSR5: An Efficient Storage Format for Cross-Platform Sparse Matrix-Vector Multiplication (1503.05032v2)

Abstract: Sparse matrix-vector multiplication (SpMV) is a fundamental building block for numerous applications. In this paper, we propose CSR5 (Compressed Sparse Row 5), a new storage format, which offers high-throughput SpMV on various platforms including CPUs, GPUs and Xeon Phi. First, the CSR5 format is insensitive to the sparsity structure of the input matrix. Thus the single format can support an SpMV algorithm that is efficient both for regular matrices and for irregular matrices. Furthermore, we show that the overhead of the format conversion from the CSR to the CSR5 can be as low as the cost of a few SpMV operations. We compare the CSR5-based SpMV algorithm with 11 state-of-the-art formats and algorithms on four mainstream processors using 14 regular and 10 irregular matrices as a benchmark suite. For the 14 regular matrices in the suite, we achieve comparable or better performance over the previous work. For the 10 irregular matrices, the CSR5 obtains average performance improvement of 17.6\%, 28.5\%, 173.0\% and 293.3\% (up to 213.3\%, 153.6\%, 405.1\% and 943.3\%) over the best existing work on dual-socket Intel CPUs, an nVidia GPU, an AMD GPU and an Intel Xeon Phi, respectively. For real-world applications such as a solver with only tens of iterations, the CSR5 format can be more practical because of its low-overhead for format conversion. The source code of this work is downloadable at https://github.com/bhSPARSE/Benchmark_SpMV_using_CSR5

Overview of CSR5: An Efficient Storage Format for Cross-Platform Sparse Matrix-Vector Multiplication

The paper presents CSR5, a storage format designed to optimize sparse matrix-vector multiplication (SpMV) across various platforms, including CPUs, GPUs, and Xeon Phi. By extending the traditional CSR format, CSR5 aims to achieve high throughput for both regular and irregular matrices without requiring significant preprocessing costs associated with format conversion. The authors propose that CSR5 achieves excellent performance on a collection of diverse hardware architectures.

Key Contributions and Methodology

1. Introduction of CSR5: The CSR5 format retains the core structure of the traditional CSR format while introducing an efficient mechanism for handling non-zero matrix entries. CSR5's design is inherently insensitive to matrix sparsity structures, eliminating the need for detailed format tuning, which is a core requirement for many other formats.
2. Automated Tuning of Parameters: CSR5 utilizes two main parameters—tile width and tile height—that are auto-tuned based on the hardware architecture to ensure optimal SIMD utilization. The format is refined to avoid the costly structure-dependent tuning that plagues many traditional formats.
3. Cross-Platform Applicability: The authors implement CSR5 on diverse platforms: dual-socket Intel CPUs, an Nvidia GPU, an AMD GPU, and an Intel Xeon Phi. They demonstrate that CSR5 consistently delivers high-performance SpMV across these architectures.
4. Improved Segmented Sum Algorithm: A redesigned segmented sum algorithm supports the CSR5 format, allowing efficient parallel processing of the matrix. By using a fast segmented sum algorithm, CSR5 promises better load balance and performance scalability.
5. Performance Evaluation: CSR5 is compared against 11 state-of-the-art formats, revealing that CSR5 is competitive or superior regarding performance across a suite of both regular and irregular matrices. Notably, for irregular matrices, CSR5 achieves an average performance improvement of up to 293.3% on some platforms compared to the second-best method.

Implications and Future Directions

The introduction of CSR5 has several significant implications for the landscape of sparse matrix computations:

• Application in Diverse Scenarios: The format's insensitivity to matrix irregularity makes it versatile, potentially beneficial for applications involving large and complex datasets typical in scientific computing, machine learning, and data analytics.
• Cross-Platform Efficiency: The consistent performance across different hardware platforms indicates potential for standardized sparse matrix operations in heterogeneous computing environments.
• Reduced Preprocessing Costs: By minimizing the conversion overhead from CSR, CSR5 offers practical improvements in iterative method scenarios where preprocessing costs can outweigh SpMV performance gains.

Speculation on Future Developments

The future work could explore extensions of the CSR5 format to other sparse operations beyond SpMV, such as sparse solvers or preconditioners in iterative methods. Further investigation into alignment with emerging hardware architectures, especially new SIMD extensions and GPU models, could solidify CSR5's position as a format of choice in high-performance computing. Additionally, the potential integration of machine learning techniques for auto-tuning might further enhance CSR5's adaptability and efficiency.

CSR5 represents a significant step in developing efficient storage formats that deliver high performance on diverse computing platforms, addressing limitations of existing methods, and paving the way for more unified sparse numerical computation approaches.