Centered Permutation Prefixes for SGD with Random Reshuffling: Sharp Rates, Hölder Geometry, and Composite Proximal Extensions

Abstract: We study stochastic gradient descent with random reshuffling for finite sums [ F(x)=\frac1n\sum_{i=1}^n f_i(x). ] For fresh reshuffling with a constant component stepsize, if each $f_i$ has an $L$-Lipschitz gradient and the average $F$ is $μ$-strongly convex with a Lipschitz-continuous Hessian, we prove the last-epoch rate [ \mathbb E[F(y_K)-F(x_\star)] =\widetilde O!\left(T^{-2}+n^2T^{-3}\right), \qquad T=nK, ] matching the known quadratic lower bound in its $(n,K)$-dependence. The components may be nonconvex, and no componentwise Hessian continuity or separate bounded-iterate assumption is required. More generally, a $ν$-Hölder-continuous average Hessian adds only $\widetilde O(n<sup>{1+ν}T<sup>{-2-2ν})$, so every $ν\ge 1/2$ preserves the quadratic rate. Under convex components, a decreasing-stepsize result removes the large-epoch requirement and recovers the same two-term scale once $nK$ exceeds the condition-number scale. We also analyze epoch-wise ProxRR for $\mathcal P=F+ψ$. Writing $x<sup>\dagger$ for the composite minimizer and $β<em>\star=|\nabla F(x<sup>\dagger)|$, we prove [ \mathbb E|y_K-x^\dagger|^2 =\widetilde O!\left( \frac{β\star^2}{K^2} +T^{-2}+n^2T^{-3} +n^{1+ν}T^{-2-2ν} \right). ] For $ν\ge 1/2$, we show that the $β_\star<sup>2/K<sup>2$ splitting term is unavoidable and obtain a matching lower bound up to logarithms in the stated constant-stepsize regime.