Finite-Step SGD: Convergence & Variance

• Finite-step SGD is a stochastic optimization regime that focuses on finite-iteration dynamics, emphasizing non-asymptotic convergence, adaptivity, and practical performance.
• It employs adaptive step-size strategies—such as Barzilai–Borwein, non-monotonic schedules, and warm restart methods—to achieve robust convergence and enhanced generalization.
• The topic also highlights variance reduction techniques and noise structure analysis, which together improve stability and convergence rates in practical ML applications.

Finite-step stochastic gradient descent (SGD) encompasses a spectrum of algorithms and analytical frameworks that explicitly address the non-asymptotic, stepwise dynamics of SGD in practical optimization scenarios, extending beyond the classical infinite-iteration or diffusion-limit paradigms. This research area focuses on understanding, characterizing, and improving the convergence, stability, and robustness of SGD methods over finite (and often small) numbers of iterations—a regime of paramount practical relevance in modern machine learning and scientific computing. Recent advances have yielded rigorous finite-step convergence rates under stochastic and dependent sampling, new adaptivity strategies for step-size selection, non-asymptotic variance reduction tools, and sharp insights into the role of problem geometry, gradient noise structure, and discrete-time effects.

1. Step-Size Strategies and Self-Adaptive Learning Rates

Finite-step analyses highlight the pivotal influence of step-size regimes and adaptivity on both theoretical convergence and practical performance:

• Barzilai–Borwein (BB) Step Size for SGD: The BB approach adaptively selects the learning rate using a quasi-Newton rationale, leveraging the observed displacements of iterates and gradient differences. For SVRG, the step size per epoch is

$$\eta_k = \frac{1}{m} \frac{\|\tilde{x}_k - \tilde{x}_{k-1}\|^2}{ (\tilde{x}_k - \tilde{x}_{k-1})^\top (g_k - g_{k-1}) },$$

with $g_k$ the full gradient at the epoch's tail point. For standard SGD, a similar formula with averaged stochastic gradients is used, augmented by a smoothing procedure to stabilize noisy denominators. The linear convergence rate for strongly convex cases is established, and experiments confirm rapid, robust convergence without manual tuning (Tan et al., 2016).

• Bandwidth-Based and Non-Monotonic Step-Sizes: Rather than imposing a monotone decay, the learning rate is allowed to oscillate within a prescribed band; for example, $m/t \leq \eta(t) \leq M/t$, enabling up-down or periodic policies. Theoretical error bounds remain $O(1/T)$ under mild conditions, and non-monotonic schedules demonstrably improve training loss and test accuracy in convex and nonconvex deep learning tasks (Wang et al., 2021).
• Logarithmically Modified and Warm Restart Step Sizes: Incorporating slowly varying decay, such as $\eta_t = \eta_0/(\sqrt{t}+\ln t)$ (Shamaee et al., 2023) or $\eta_t = \eta_0 (1-(\ln t)/(\ln T))$ with warm restarts (Shamaee et al., 1 Apr 2024), yields $O(\ln T / \sqrt{T})$ or $O(1/\sqrt{T})$ convergence for nonconvex objectives, improving late-stage convergence and final test accuracy over classical $\sim 1/\sqrt{t}$ or cosine schedules.
• Preconditioned Polyak Step-Size: Combining Polyak's goal-seeking step-size (which scales by $[f(w_t)-f^*]/\|\nabla f(w_t)\|^2$) with coordinate preconditioning (via Hessian diagonal approximation, AdaGrad, or Adam metrics) yields robust, tuning-free updates that adapt to ill-conditioned or non-isotropic loss surfaces (Abdukhakimov et al., 2023). The update is

$$w_{t+1} = w_t - \frac{f(w_t)}{\|\nabla f(w_t)\|^2_{B_t^{-1}}} B_t^{-1} \nabla f(w_t)$$

with $B_t$ an adaptive preconditioner.

2. Convergence, Criticality, and Markov Chain Dynamics

Modern analyses provide rigorous finite-step results under weak assumptions—bypassing classical limitations such as global Lipschitz continuity or square-summable step sizes:

• Relaxed Step-Size and Stopping-Time Methods: For nonconvex objectives, almost sure and $L_2$ convergence of $\{\theta_t\}$ to critical points can be ensured under step-sizes $\{\epsilon_t\}$ with $\sum_t \epsilon_t = +\infty$ and $\sum_t \epsilon_t^p < +\infty$ (for some $p>2$), using a stopping-time framework that slices the trajectory into random intervals based on loss cross-levels (Jin et al., 17 Apr 2025). This generalizes beyond Robbins–Monro type conditions.
• GSLLN and Heavy-Tailed Noise: The generalized strong law of large numbers (GSLLN) paradigm decouples assumptions on function regularity from those on stochastic noise, allowing convergence of zero-order or classic SGD under arbitrary noise sequences (not requiring bounded variance or mean) as long as a weighted average converges almost surely with respect to the step-size sequence (Karandikar et al., 16 May 2025). This greatly expands the admissible noise classes, accommodating heavy tails and degenerate cases.
• Markovian and Separable Structure: With constant step-size, SGD iterates form a Markov chain, and for separable (coordinate-wise) nonconvex objectives, the chain decomposes the state space into transient and absorbing sets, each supporting unique invariant measures which serve as global attractors with geometric convergence. Notably, the limiting distribution need not support global minimizers—SGD may (with probability one) escape global minimizers in favor of local modes, and bifurcations in the number of invariant measures can occur as step-size and problem parameters vary (Shirokoff et al., 18 Sep 2024).
• Sampling Structure and Mixing: When gradients are sampled via ergodic Markov chains (as in reinforcement learning), finite-step SGD achieves nearly the same convergence rates as in the i.i.d. sampling regime, incurring only an extra logarithmic penalty proportional to the chain's mixing time:

$$E[\|x_{k+1} - x^*\|^2] \leq O(1/k^2) + O((\log k) / k)$$

under strong convexity, without requiring bounded iterates or gradients (Doan et al., 2020).

3. Finite-Step Variance Reduction and Control Variates

Variance reduction in the finite-step regime is essential for accelerating convergence and improving stability, especially for problems posed as expectation functionals over continuous distributions:

• Least-Squares Control Variate (LSCV) for SGD: By fitting a surrogate gradient model $v \in V_m$ (via weighted least-squares on past data) and using the update

$$u_{k+1} = u_k - T_k [Vg(u_k, Y_k) - v(Y_k) + E_{Y}[v(Y) | \mathcal{S}_k]],$$

the variance of the stochastic gradient estimator is reduced without requiring storage of all gradient information or discretization of the expectation (Nobile et al., 28 Jul 2025). Under standard strong convexity and smoothness assumptions, this yields sublinear convergence guarantees and can outperform classical variance reduction for continuous probability models.

• Efficiency Ordering: Input sequences with higher sampling efficiency (e.g., non-backtracking random walks or shuffling schemes in distributed settings) yield lower asymptotic covariance in SGD iterates, as formalized by Loewner ordering in the covariance matrices arising in the central limit theorem for SGD errors. This efficiency ordering persists empirically across variants such as accelerated SGD and Adam (Hu et al., 2022).

4. Impact of Noise Structure and Diffusion Approximations

Finite-step SGD dynamics are intricately affected by the structure and tails of gradient noise:

• Heavy-Tailed Noise and Metastability: If the gradient noise is modeled as symmetric $\alpha$-stable ($\mathcal{S}_\alpha \mathcal{S}(\sigma)$) rather than Gaussian, the first exit time from basins in the loss landscape scales with the basin's width rather than its depth, supporting the observed bias of SGD toward "wide minima". However, the equivalence between the continuous-time SDE and the discrete SGD is preserved only if the step size is sufficiently small, explicit conditions for which are derived in terms of problem parameters and noise characteristics (Nguyen et al., 2019).
• Effective Temperature and Generalization: Mini-batch sampling induces a nontrivial state-dependent noise, whose magnitude can be quantified via an effective temperature $T_{\rm eff}$ estimated through fluctuation-dissipation theory (FDT) and dynamical mean-field theory (DMFT). Higher effective temperatures lead to wider decision boundaries and better generalization, as seen in both under- and over-parameterized settings (Mignacco et al., 2021).
• Discrete-Time SDE Models: For least-squares regression, the discrete dynamics of SGD can be matched to a continuous-time Itô SDE with drift $-\nabla L(\theta)$ and diffusion coefficient determined by the covariance of the stochastic update. The step size governs both the convergence rate and the emergence of heavy tails in the stationary distribution; large $\gamma$ can induce infinite higher-order moments, and averaging or step-size annealing is required to mitigate this (Schertzer et al., 2 Jul 2024).

5. Structured Objectives and Stochastic vs Deterministic Roles

• Splitting Stochastic and Deterministic Objective Terms: For objectives

$$\psi(x) = F(x) + h(x), \quad F(x) = \frac{1}{n} \sum_{i=1}^n f_i(x)$$

where $F$ is estimated stochastically and $h$ deterministically, finite-step analyses reveal that the contraction rate and convergence radius depend asymmetrically on the Lipschitz and convexity constants of both terms. Optimal step size and final neighborhood depend more sensitively on the stochastic component's smoothness and noise parameters, with deterministic strong convexity (from $h$) enabling sharper rates and smaller stationary errors. As batch size increases, the stochastic error vanishes, and convergence matches classical gradient descent (Li et al., 3 Sep 2025).

• Mini-Batch Size, Steps, and Complexity: For nonconvex optimization using an Armijo line search, the required number of steps $K(b)$ decreases monotonically and convexly with batch size $b$, while SFO complexity $N(b) = K(b) b$ is convex in $b$ and minimized at a specific critical batch size. This provides a concrete, data-driven strategy for optimizing resource allocation during SGD training (Tsukada et al., 2023).

6. Practical Implications, Algorithm Design, and Recommendations

Finite-step analytical results, step-size selection strategies, and noise modeling insights yield several concrete operational guidelines:

• Prioritize self-adaptive or data-driven step-size strategies (such as BB, Polyak, or warm restart logarithmic schedules), reducing reliance on manual hyperparameter tuning and improving robustness across diverse problem settings (Tan et al., 2016, Abdukhakimov et al., 2023, Shamaee et al., 1 Apr 2024).
• For probabilistic and GLM models, when using constant step-size SGD, averaging predictions (moment parameters) rather than natural parameters leads to both theoretically and empirically superior convergence and generalization, especially under model misspecification or in infinite-dimensional function spaces (Babichev et al., 2018).
• Non-monotonic and banded step-size policies should be considered for non-convex and deep learning applications, as these schedules mitigate premature annealing and improve finite-step convergence and generalization (Wang et al., 2021).
• Sampling strategy and noise structure should be explicitly incorporated into algorithm design (favoring efficient input sequences such as shuffling or non-backtracking walks for distributed or decentralized problems), as these substantially affect long-term error via variance ordering (Hu et al., 2022).
• When the gradient noise is heavy-tailed, practitioners must ensure the step-size is small enough to preserve desired metastability properties and wide basin preference; discrete-time effects can otherwise fundamentally alter optimization dynamics (Nguyen et al., 2019).
• For structured objective functions combining stochastic finite-sum and deterministic terms, tuning batch size, step size, and term assignment can exploit asymmetric convergence properties (Li et al., 3 Sep 2025).
• Variance reduction by learning control variates from past gradients via least-squares surrogates is effective in finite-step continuous-distribution settings, outperforming classical finite-sum-based variance reduction methods for such problems (Nobile et al., 28 Jul 2025).

7. Summary Table: Core Techniques and Practical Outcomes

Finite-Step SGD Technique	Core Feature	Empirical/Theoretical Benefit
Barzilai–Borwein/Polyak adaptive step-size	Data-driven, requires no manual tuning	Achieves (provable) fast convergence, robust
Bandwidth-based/up-down step-size policies	Non-monotonic, fluctuates within bounded envelope	Better loss/accuracy, especially nonconvex
Markovian/GSLLN convergence analysis	Weakens classical (strong) noise and Lipschitz requirements	Broader step-size/noise admissibility
Least-squares control variate acceleration	Surrogate gradient model, fits past gradients	Strong variance reduction, faster convergence
Efficiency ordering of sampling schemes	Matrix orderings, shuffling/NBRW vs i.i.d./SRW	Lower asymptotic covariance, extendable to SGD variants
Heavy-tail SDE analysis, effective temperature	Explicit characterization of tails and dynamics	Explains generalization via wide minima/noisiness
Structured objectives (stochastic+deterministic, batch size tuning)	Harnesses term asymmetry, batch size schedule	Provable rates, tight resource allocation

This compendium demonstrates that careful finite-step analysis, adaptive step-size design, explicit variance modeling, and problem-adaptive algorithm construction yield robust, efficient, and theory-backed stochastic optimization procedures. Such advances are indispensable for modern large-scale machine learning and scientific computation, where iteration counts are fixed, noise is structured, and practical considerations dominate.