Carryover of CG differentiation theory to nonsymmetric Krylov solvers (GMRES, BiCGStab)

Determine the extent to which established theoretical results on differentiating through conjugate gradient (CG) linear solvers carry over to nonsymmetric Krylov solvers, specifically GMRES and BiCGStab.

Background

The paper contrasts low-level (algorithmic) differentiation through iterative linear solver implementations with high-level differentiation that treats the linear solve as an elementary operation and derives derivatives via matrix calculus. For conjugate gradient (CG), prior work by Gratton et al. (2014) and Christianson (2018) provides theoretical foundations for differentiating through CG, including conditions and performance guarantees in certain settings.

In contrast, for nonsymmetric Krylov solvers such as GMRES and BiCGStab, the authors note a lack of clarity regarding whether the CG-based differentiation theory applies. This uncertainty motivates their empirical paper and highlights a theoretical gap concerning low-level differentiation for nonsymmetric systems.