Limiting distribution for nonlinear SGD with infinite-variance noise

Determine the limiting distribution of averaged iterates generated by a general nonlinear stochastic gradient descent algorithm when the gradient noise has infinite variance.

Background

The paper establishes quenched α-stable convergence for stochastic gradient descent when the recursion has an affine iterated-random-function structure or when the process is an additive functional of a linear process. These assumptions do not cover general nonlinear stochastic gradient descent with infinite-variance gradient noise.

The authors identify the unresolved issue that, outside those tractable settings, the limiting distribution is not characterized. They then introduce projected stochastic gradient descent as a robust alternative: projection yields geometric moment contraction and permits Gaussian asymptotic results under suitable conditions, but this does not resolve the limiting-distribution problem for the unrestricted nonlinear algorithm.

References

However, for a general nonlinear SGD algorithm with infinite variance noises, the limiting distribution remains unknown.

— Stochastic Gradient Descent: A Nonlinear Time Series Perspective  (2610.20894 - Li et al., 14 Oct 2026) in Supplemental Material, Section “Convergence of SGD with Infinite Variances,” subsection “Quenched α-Stable Convergence”