Optimal normalization frequency and placement for quasi multi-word arithmetic in CG

Determine the optimal frequency and algorithmic placement of normalization operations when using quasi double-word (QDW) and quasi triple-word (QTW) arithmetic in the Conjugate Gradient (CG) method, balancing numerical accuracy and execution time. In particular, ascertain whether normalizing only the residual vector once per iteration after the AXPY update (using QuickTwoSum for QDW and VecSum3 for QTW) is optimal, or whether more frequent and/or differently placed normalization yields better overall efficiency and accuracy.

Background

Quasi multi-word algorithms (QDW and QTW) omit per-operation normalization to reduce cost, which can lead to overlap of word components and accuracy degradation over repeated operations. For iterative methods like Conjugate Gradient (CG), such degradation can accumulate and impair convergence.

The authors found that CG without any normalization fails to converge, but that normalizing the residual vector after the AXPY update once per iteration restores convergence with iteration counts comparable to non-quasi algorithms, incurring minimal overhead since SpMV dominates CG cost.

They highlight a trade-off: more frequent normalization may improve accuracy but increases runtime. Identifying the best frequency and location within the CG algorithm to apply normalization (e.g., residual update only versus additional points) remains unresolved.