Analytical explanation of the left stability boundary

Derive a complete analytical explanation of the left stability boundary for Adam applied to superquadratic one-dimensional losses, including the complex oscillatory dynamics and finite-step effects that determine the boundary coefficient.

Background

For one-dimensional superquadratic losses of the form L(x)=k|x|n, the authors identify a wedge-shaped region of recurrent macroscopic spikes in the (β₁,β₂) plane. The right boundary is derived analytically from a local stability condition, whereas the left boundary is characterized only empirically by C_L≈12(n−2)/(3n−4).

The unresolved aspect concerns the oscillatory dynamics and finite-step effects governing the left boundary. A complete analysis would explain why the empirically observed coefficient takes this form and would complement the theorem establishing the right boundary for trajectories attracted to rescaled fixed points.

References

These regimes show that the left boundary involves more complex oscillatory dynamics and finite-step effects, and a complete analytical explanation remains open.

— Beyond Quadratic Loss: The Stability Phase Diagram of Adam  (2609.18314 - Tang et al., 16 Sep 2026) in Section 3, “One-dimensional toy model of Adam stability,” immediately after Figure 3

More generally, non-quadratic geometry with an exponent $n_{\rm eff}$ that changes across direction, scale, and training time is likely to be the norm. Can an optimizer estimate $n_{\rm eff}$ at its update scale and adapt its learning rate or moment timescales to the evolving phase boundary?

— Beyond Quadratic Loss: The Stability Phase Diagram of Adam  (2609.18314 - Tang et al., 16 Sep 2026) in Section 6, “Conclusion and outlook,” item 2