Adaptive estimation of KGSM hyperparameters from data

Investigate methods to estimate suitable values of the momentum parameter M and the geometric smoothing parameter β adaptively during the execution of the KGSM iteration for solving Ax = b, using information from the observed iterates and residuals, without prior knowledge of optimal settings.

Background

While KGSM depends on two hyperparameters (M and β), the paper does not provide a mechanism for estimating them directly from data. Effective deployment would benefit from adaptive procedures that infer these parameters on the fly.

The authors explicitly point out that they did not investigate such estimation methods and ask whether adaptive estimation is possible, highlighting a practical challenge for applying KGSM to real problems.

References

We did not investigate methods for estimating effective parameters $M$ and $\beta$ from data. Even if formulas for $M$ and $\beta$ are known, these would need to be estimated from data to implement the method for practical problems. Is it possible to estimate suitable parameters as the algorithm runs adaptively?

— Randomized Kaczmarz with geometrically smoothed momentum (2401.09415 - Alderman et al., 17 Jan 2024) in Discussion, Limitations and questions — Estimating M and β from data