Matching the Lower Bounds: Stochastic Contracting Cubic Newton and Its Optimal Acceleration

Abstract: We study second-order methods for convex stochastic optimization, where gradients and Hessians are available only through stochastic estimates with variances $σ_1<sup>2$ and $σ_2<sup>2$, respectively. First, we propose the Stochastic Contracting Cubic Newton method. At each iteration, it minimizes a cubic model with additional quadratic regularization and then contracts the step toward the current point. After $T$ iterations, the method achieves the expected convergence rate $\mathcal{O}(σ_1/\sqrt{T}+σ_2/T+1/T<sup>2)$. Building on this construction, we develop an accelerated variant achieving $$\mathcal{O}(σ_1/\sqrt{T}+σ_2/T<sup>2+1/T<sup>{7/2})$$, matching the known lower bounds of Agafonov et al. (2024) in all three terms.