Critical momentum threshold and non-convergence region for KGSM

Determine the threshold momentum parameter values M and the corresponding region in the (M, β) hyperparameter space of the Kaczmarz with geometrically smoothed momentum (KGSM) iteration, defined by x_{k+1} = x_k + ((b_{i_k} − ⟨a_{i_k}, x_k⟩)/||a_{i_k}||_2^2) a_{i_k} + M y_k and y_{k+1} = β y_k + (1 − β)(x_{k+1} − x_k) with i_k sampled proportionally to ||a_i||_2^2, under which the method fails to converge for consistent linear systems Ax = b. Ascertain how this critical boundary depends on properties of A (e.g., its singular values) and the choice of sampling distribution.

Background

The paper introduces the Kaczmarz with geometrically smoothed momentum (KGSM) algorithm and analyzes expected signed error decay along singular vector directions. Numerical experiments reveal that for some matrices and parameter choices, KGSM exhibits accelerated convergence, while for others it becomes unstable and diverges.

In the Discussion section, the authors note empirical evidence of a critical momentum beyond which KGSM fails, and explicitly ask for a characterization of the parameter region where convergence breaks down. Identifying this boundary would clarify safe hyperparameter settings and deepen understanding of the method’s stability.