A Hardware-aware and Stable Orthogonalization Framework (2204.13393v1)

Abstract: The orthogonalization process is an essential building block in Krylov space methods, which takes up a large portion of the computational time. Commonly used methods, like the Gram-Schmidt method, consider the projection and normalization separately and store the orthogonal base explicitly. We consider the problem of orthogonalization and normalization as a QR decomposition problem on which we apply known algorithms, namely CholeskyQR and TSQR. This leads to methods that solve the orthogonlization problem with reduced communication costs, while maintaining stability and stores the orthogonal base in a locally orthogonal representation. Furthermore, we discuss the novel method as a framework which allows us to combine different orthogonalization algorithms and use the best algorithm for each part of the hardware. After the formulation of the methods, we show their advantageous performance properties based on a performance model that takes data transfers within compute nodes as well as message passing between compute nodes into account. The theoretic results are validated by numerical experiments.