Directional stability: generalize Steinerberger’s result to KGSM

Establish an analog of Steinerberger’s directional stability theorem (Theorem 1.3 for randomized Kaczmarz) for the KGSM iteration, proving bounds on the expected change in direction of the error vector between successive iterates and characterizing how this directional change depends on the momentum parameter M and the smoothing parameter β.

Background

Steinerberger’s Theorem 1.3 shows directional stability properties for randomized Kaczmarz without momentum. Extending such results to KGSM would clarify whether and how momentum affects the iterate direction, which is central to understanding acceleration versus instability.

The authors explicitly note they did not establish such an analog and ask whether this result can be generalized, especially with dependence on M.

References

We did not establish an analog of Theorem 1.3, which would help to explain any directional change in KGSM. Can this result be generalized to the setting with geometrically smoothed momentum? In particular, can the change in direction be controlled as a function of $M$?

— Randomized Kaczmarz with geometrically smoothed momentum (2401.09415 - Alderman et al., 17 Jan 2024) in Discussion, Limitations and questions — Direction of convergence and convergence in ℓ2-norm