Mapping Sparse Triangular Solves to GPUs via Fine-grained Domain Decomposition

Abstract: Sparse linear systems are typically solved using preconditioned iterative methods, but applying preconditioners via sparse triangular solves introduces bottlenecks due to irregular memory accesses and data dependencies. This work leverages fine-grained domain decomposition to adapt triangular solves to the GPU architecture. We develop a fine-grained domain decomposition strategy that generates non-overlapping subdomains, increasing parallelism in the application of preconditioner at the expense of a modest increase in the iteration count for convergence. Each subdomain is assigned to a thread block and is sized such that the subdomain vector fits in the GPU shared memory, eliminating the need for inter-block synchronization and reducing irregular global memory accesses. Compared to other state-of-the-art implementations using the ROCm$^{\text{TM}}$ software stack, we achieve a 10.7$\times$ speedup for triangular solves and a 3.2$\times$ speedup for the ILU0-preconditioned biconjugate gradient stabilized (BiCGSTAB) solver on the AMD Instinct$^{\text{TM}}$ MI210 GPU.