Convergence in absolute error and ℓ2-norm for KGSM

Establish convergence guarantees for the KGSM algorithm in terms of the expected absolute error along singular vector directions, E[|⟨x − x_k, v_l⟩|], and/or the ℓ2-norm error, E[||x − x_k||_2], possibly under assumptions on the singular value spectrum of A and the hyperparameters (M, β).

Background

The paper provides formulas for expected signed error along singular directions, which can model performance in some cases, but do not directly yield absolute error or ℓ2-norm convergence guarantees.

The authors explicitly pose the question of establishing convergence for absolute directional error and the ℓ2-norm, indicating a gap between current theory and desired practical guarantees.

References

Another related question is establishing convergence for the expected absolute error in the direction of a singular vector $|\langle x-x_k,v_l \rangle|$ or for the $\ell2$-norm $|x-x_k|_2$.

Randomized Kaczmarz with geometrically smoothed momentum (2401.09415 - Alderman et al., 17 Jan 2024) in Discussion, Limitations and questions — Direction of convergence and convergence in ℓ2-norm