Shuffling the Data, Stretching the Step-size: Sharper Bias in constant step-size SGD

Abstract: From adversarial robustness to multi-agent learning, many machine learning tasks can be cast as finite-sum min-max optimization or, more generally, as variational inequality problems (VIPs). Owing to their simplicity and scalability, stochastic gradient methods with constant step size are widely used, despite the fact that they converge only up to a constant term. Among the many heuristics adopted in practice, two classical techniques have recently attracted attention to mitigate this issue: \emph{Random Reshuffling} of data and \emph{Richardson--Romberg extrapolation} across iterates. Random Reshuffling sharpens the mean-squared error (MSE) of the estimated solution, while Richardson-Romberg extrapolation acts orthogonally, providing a second-order reduction in its bias. In this work, we show that their composition is strictly better than both, not only maintaining the enhanced MSE guarantees but also yielding an even greater cubic refinement in the bias. To the best of our knowledge, our work provides the first theoretical guarantees for such a synergy in structured non-monotone VIPs. Our analysis proceeds in two steps: (i) we smooth the discrete noise induced by reshuffling and leverage tools from continuous-state Markov chain theory to establish a novel law of large numbers and a central limit theorem for its iterates; and (ii) we employ spectral tensor techniques to prove that extrapolation debiases and sharpens the asymptotic behavior even under the biased gradient oracle induced by reshuffling. Finally, extensive experiments validate our theory, consistently demonstrating substantial speedups in practice.

• The paper introduces a novel synergy between random reshuffling and Richardson-Romberg extrapolation to achieve cubic bias refinement in constant step-size SGD.
• It rigorously analyzes asymptotic bias and MSE bounds using a Markov chain framework, demonstrating an O(γ^3) bias and quadratic MSE improvement.
• Empirical results on synthetic games and GANs validate that the combined approach outperforms standard constant step-size SGD across various optimization settings.

Sharper Bias in Constant Step-Size SGD: Synergizing Random Reshuffling and Richardson-Romberg Extrapolation

Introduction and Motivation

Stochastic optimization with finite-sum formulations underpins a wide spectrum of machine learning applications, from empirical risk minimization to structured domains such as min-max optimization, multi-agent games, and variational inequalities (VIs). Stochastic Gradient Descent (SGD) with a constant step size is a mainstay due to its operational simplicity, rapid initial progress, and ease of hyperparameter tuning. However, it is well-established that, with a constant step size, SGD only converges up to a persistent bias and variance (MSE) neighborhood around the optimum, with both terms typically scaling with the learning rate.

To mitigate this suboptimal asymptotic error, two prevalent heuristics have gained widespread adoption:

• Random Reshuffling (RR1): Processing each data point exactly once per epoch in a random order, which empirically accelerates convergence and improves finite-sum stochastic approximation performance over with-replacement sampling.
• Richardson-Romberg Extrapolation (RR2): Leveraging linear combinations of solutions obtained with multiple step sizes to systematically cancel leading-order bias terms, a classical idea from numerical analysis now applied to stochastic optimization.

While each method is known to independently sharpen either variance (RR1) or bias (RR2), their theoretical interaction, especially in structured non-monotone VIs, has been unresolved. This work establishes that combining RR1 with RR2 achieves a further cubic improvement in the asymptotic bias, decisively outperforming either mechanism alone, and provides rigorous theoretical analysis for variational inequalities under mild regularity assumptions.

Theoretical Contributions

Problem Setting and Assumptions

The analysis is situated within unconstrained finite-sum VIs of the form: find $x^* \in \mathbb{R}^d$ such that

$$\langle F(x^*), x - x^* \rangle \geq 0, \ \forall x \in \mathbb{R}^d$$

where $F(x) = \frac{1}{n} \sum_{i=1}^n F_i(x)$, and each $F_i$ is $L_i$-Lipschitz. The focus is on quasi-strongly monotone operators (a generalization of strong monotonicity), which captures a wide class of practical models, including but not limited to convex minimization, Nash equilibria for convex games, and saddle-point formulations.

Algorithmic Framework

The proposed algorithm maintains two synchronized SGD chains under RR1: one with step size $\gamma$ and the other with $2\gamma$, both traversing identical reshuffled data orderings within each epoch (i.e., sharing the permutation). Epoch-end iterates from both chains are then combined via Richardson-Romberg extrapolation: $\hat{x}_{k+1} = 2x_{k+1,\gamma} - x_{k+1,2\gamma}$ A calibrated Gaussian preprocessing step smooths the discrete reshuffling noise, permitting application of continuous-state Markov chain arguments.

Main Results: Cubic Bias Refinement

Asymptotic Bias and MSE Bounds

Building on Markov chain geometric ergodicity, higher-moment bounding, and spectral analysis of the epoch-resolved dynamics, the paper establishes the following sharp rates:

Method	Asymptotic Bias Order	Asymptotic MSE Order
SGD	$O(\gamma)$	$O(\gamma)$
SGD + RR1	$$\langle F(x^*), x - x^* \rangle \geq 0, \ \forall x \in \mathbb{R}^d$$0	$$\langle F(x^*), x - x^* \rangle \geq 0, \ \forall x \in \mathbb{R}^d$$1
SGD + RR2	$$\langle F(x^*), x - x^* \rangle \geq 0, \ \forall x \in \mathbb{R}^d$$2	$$\langle F(x^*), x - x^* \rangle \geq 0, \ \forall x \in \mathbb{R}^d$$3
SGD + RR2+RR1	$$\langle F(x^*), x - x^* \rangle \geq 0, \ \forall x \in \mathbb{R}^d$$4	$$\langle F(x^*), x - x^* \rangle \geq 0, \ \forall x \in \mathbb{R}^d$$5

The composition strictly cancels both the $$\langle F(x^*), x - x^* \rangle \geq 0, \ \forall x \in \mathbb{R}^d$$6 and $$\langle F(x^*), x - x^* \rangle \geq 0, \ \forall x \in \mathbb{R}^d$$7 bias terms through extrapolation, leaving only a residual $$\langle F(x^*), x - x^* \rangle \geq 0, \ \forall x \in \mathbb{R}^d$$8 bias. This is the first such theoretical guarantee for non-monotone VIs under the finite-sum, constant step-size regime.

Markov Chain Perspective

By demonstrating that the epoch-level algorithm forms a time-homogeneous Markov chain with a unique invariant measure and geometric mixing in total variation, the analysis proves the existence of LLN/CLT for the iterates, further justifying the empirical estimation of performance metrics along the chain trajectory.

Higher-Order Moment Control

A critical technical advance is the development of combinatorial tools to control fourth moments in the context of without-replacement (reshuffled) sampling—essential for demonstrating that RR2-extrapolation does not deteriorate higher moments and that the bias cancellation mechanism persists under nontrivial dependencies induced by reshuffling.

Empirical Validation

Experiments focus on both synthetic two-player strongly monotone quadratic games and Wasserstein GANs. The synergy of RR2+RR1 consistently yields:

• Substantially reduced relative error and bias at convergence compared to all baselines.
• Empirically linear convergence to a tighter optimality neighborhood, validating the theoretical cubic bias refinement and quadratic MSE bounds.
• Robust benefits across multiple stochastic first- and second-order algorithms (SGDA, SEG, OMD, SCO), and step sizes, with minimal wall-clock overhead thanks to easy parallelization.

Implications and Future Directions

The results close a theoretical-practical gap between standard practice in ML optimization pipelines and the theoretical understanding of their convergence behavior. By showing that RR1 and RR2 are not merely additive but synergistic, the work challenges the design of future stochastic optimization algorithms—suggesting that systematic debiasing via multiple step sizes and structured data traversal can yield provable and practical gains, even in challenging non-monotone settings.

Potential extensions include:

• Application to non-quasi-strongly monotone or more adversarially nonconvex/nonconcave regimes, e.g., modern GANs or high-dimensional RL.
• Exploration of other higher-order extrapolation techniques.
• Refined data-dependent analysis exploiting intrinsic problem geometry beyond worst-case Lipschitz and monotonicity constants.

Conclusion

This paper provides the first rigorous theoretical and empirical evidence that the interaction of random reshuffling and Richardson-Romberg extrapolation in constant step-size SGD yields a strictly cubic bias refinement in structured variational inequality problems. This work bridges a longstanding disconnect between proven theory and common large-scale learning practice, and lays a foundation for further advances in theoretically-grounded, high-performance stochastic optimization algorithms for complex, large-scale machine learning systems.

Reference:

"Shuffling the Data, Stretching the Step-size: Sharper Bias in constant step-size SGD" (2604.10373)