Develop quantized learning theory for momentum-based algorithms

Establish rigorous excess risk bounds for momentum-based optimization algorithms (such as SGD with momentum) under the quantization framework introduced in this paper, where data features, labels, parameters, activations, and output gradients are quantized (via operators Q_d, Q_l, Q_p, Q_a, Q_o) for high-dimensional linear regression. The objective is to characterize the population risk of iterate-averaged training with momentum under practical quantization constraints.

Background

The current work analyzes the learning performance of quantized one-pass SGD without momentum, deriving excess risk bounds across multiple quantization types. Algorithms with momentum can exhibit different stability and variance properties, which may interact nontrivially with quantization noise and data spectrum distortion.

The authors explicitly note that extending their analysis to momentum-based algorithms remains open, indicating that a dedicated learning-theoretic treatment under quantization is needed.