Alternative step-size choices for RBCD

Determine whether step-size choices for randomized block coordinate descent other than polynomially decaying schedules can improve the established optimization and generalization guarantees, including schedules chosen inversely proportional to the Lipschitz constants of the partial gradients.

Background

The paper establishes its main guarantees using polynomially decaying step sizes, including schedules of the form eta_t proportional to t^{-theta} for general nonconvex problems and eta_t proportional to t^{-1} for gradient-dominated problems. The iteration count is tuned to balance optimization and generalization errors under these schedules.

The authors explicitly leave unresolved whether other commonly used RBCD step-size rules can yield stronger results. They specifically mention choosing step sizes inversely proportional to the Lipschitz constants of the partial gradients as an example of an alternative schedule.