Extension of the realized edge-of-stability framework to stochastic optimization

Extend the realized edge-of-stability framework, including the ratio of curvature load to alignment score, from deterministic full-batch optimization to stochastic mini-batch optimization, where the relevant quantities become random because the instantaneous loss landscape changes along the trajectory.

Background

The paper develops the realized edge of stability for full-batch optimization, where the alignment score, curvature load, and their ratio are analyzed deterministically. In stochastic mini-batch optimization, these quantities fluctuate because each update uses a changing instantaneous loss landscape, making the interpretation of the ratio substantially more difficult.

The authors explicitly distinguish this unresolved extension from the related literature on the edge of stochastic stability (EoSS), which studies stochastic scenarios for edge-of-stability behavior. A stochastic version of the proposed framework would therefore be needed to determine whether its diagnostic quantities and stability interpretation remain valid under mini-batch noise.