Convergence rate of USNA with 1/n step size

Derive a theoretical convergence rate for the Universal Stochastic Newton Algorithm when the learning rate is set to ν_n = 1/n, quantifying how fast the iterates theta_n approach the minimizer theta of G(h) = E[g(X,h)] under the paper’s regularity assumptions on G and the specified online inverse-Hessian estimator A_n.

Background

In experiments, the authors use the harmonic stepsize ν_n = 1/n for USNA to enable direct comparison with classical Stochastic Newton methods. While empirical results show good behavior, a matching theory is missing.

They explicitly note the absence of a theoretical proof for the convergence rate of USNA under ν_n = 1/n, identifying a gap between practice and available guarantees.