---
title: Random Reshuffling in Stochastic Optimization
url: https://www.emergentmind.com/topics/random-reshuffling
type: topic
---

# Random Reshuffling in Stochastic Optimization

Random reshuffling (RR) is a batchwise, without-replacement data permutation scheme that has become the practical standard for stochastic optimization in large-scale machine learning. In RR, data points are processed in randomly permuted order in cycles ("epochs"), with each data point accessed exactly once per epoch before a new random permutation is drawn for the next epoch. RR contrasts with uniform-with-replacement sampling typical in classical stochastic gradient descent (SGD). Over the last decade, RR's empirical and theoretical performance advantages have been increasingly understood across smooth, nonsmooth, convex, nonconvex, distributed, and variance-reduced optimization, as well as in stochastic sampling and reinforcement learning.

## 1. Algorithmic Framework and Variants

RR operates on the finite-sum minimization problem:
$$
\min_{x \in \mathbb{R}^d} f(x) = \frac{1}{n} \sum_{i=1}^n f_i(x),
$$
where each $f_i$ is typically smooth and possibly convex. The basic RR iteration proceeds as follows:
- At epoch $t$, sample a random permutation $\pi^t$ of $\{1, \dots, n\}$.
- Set $x^t_0 = x^t$.
- For $i = 1, \dots, n$:
    $$
    x^t_i = x^t_{i-1} - \eta \nabla f_{\pi^t(i)}(x^t_{i-1}).
    $$
- Set $x^{t+1} = x^t_n$.

Key variants include:
- **Shuffle-Once (SO):** A single permutation drawn at $t=0$ is reused for all epochs.
- **Incremental Gradient (IG):** Fixed deterministic order (possibly adversarial).
- **Proximal RR (ProxRR):** A proximal operator is applied at the end (or, heuristically, each step) for composite problems.
- **Variance-Reduced RR:** Variants embed control variates (e.g., RR-SVRG, SAGA-RR, AVRG) into the reshuffling paradigm [2104.09342], [1708.01383].
- **Distributed RR:** Each agent performs local RR, combined via communication and consensus mechanisms [2306.12037], [2112.15287], [2111.03820].
- **RR with Momentum:** Momentum terms are included in the inner iteration, matching popular practical SGD optimizers [2404.18452].

For non-smooth or composite objectives, RR is combined with proximal steps, and in distributed/federated optimization, RR is applied locally at each node with periodic communication [2102.06704], [2205.03914].

## 2. Convergence Theory and Complexity Results

### Smooth Strongly Convex Regime

For $L$-smooth, $\mu$-strongly convex objectives, RR achieves linear convergence with strictly faster rates and smaller steady-state error than SGD. Specifically:
- **Steady-state error:** For constant stepsize $\eta \lesssim 1/Ln$, RR achieves mean-squared error $O(\eta^2)$, while SGD yields $O(\eta)$ [1803.07964], [2006.05988].
- **Sample Complexity:** To achieve accuracy $\epsilon$,
  $$
  O\left(\kappa n \log \frac{1}{\epsilon}\right)
  $$
  where $\kappa = L/\mu$, matching lower bounds for finite-sum optimization [2006.05988], [1510.08560].
- **Accelerated Asymptotics:** With Polyak–Ruppert-style iterate averaging and stepsize $\eta_k \sim 1/k^s$ for $s \in (1/2, 1)$, RR can achieve objective suboptimality $O(1/k^{2s})$, arbitrarily close to $O(1/k^2)$, outperforming SGD's $O(1/k)$ [1510.08560].

### Smooth Nonconvex and PL Regimes

For smooth, possibly nonconvex objectives, RR achieves improved convergence rates compared to SGD:
- **Nonconvex ($L$-smooth):**
  $$
  O\left(\frac{1}{n^{1/3} T^{2/3}}\right)\ \text{for RR}, \qquad O\left(\frac{1}{\sqrt{nT}}\right)\ \text{for SGD},
  $$
  for $T$ epochs or $nT$ gradients [2006.05988], [2112.15287], [2404.18452].
- **PL condition (Polyak–Łojasiewicz):**
  $$
  O\left(\frac{1}{n T^2}\right)\ \text{for RR}, \qquad O\left(\frac{1}{n T}\right)\ \text{for SGD},
  $$
  with linear convergence to a neighborhood in the presence of persistent variance [2306.12037], [2112.15287], [2304.00459].

### High Probability and Last Iterate Guarantees

RR admits high-probability complexity results for both first- and second-order guarantees. Concentration results for without-replacement sampling have enabled last-iterate stopping rules with finite-sample complexity matching or improving in-expectation bounds [2311.11841].

### Kurdyka–Łojasiewicz (KL) and Asymptotic Convergence

Under the KL property, the full RR sequence converges to a stationary point, with rates depending on the KL exponent. For exponents $\theta \leq 1/2$, convergence rates $O(1/t)$ for iterates and $O(1/t^2)$ for objective gap are attainable with polynomially-decaying stepsizes [2110.04926], [2404.18452], [2312.01047].

### Nonsmooth and Proximal Optimization

Proximal RR algorithms extend the above results to composite objectives with weakly convex, nonsmooth regularizers. The normal map–based norm-PRR achieves $O(n^{-1/3} T^{-2/3})$ complexity for nonsmooth, nonconvex problems—improving existing bounds [2312.01047]. In convex, distributed, or federated settings, ProxRR and FedRR efficiently reduce the cost of expensive proximal operators [2102.06704], [2205.03914].

## 3. Variance Reduction and Data Ordering

Variance-reduced algorithms, when deployed with RR (e.g., RR-SVRG, SAGA-RR, AVRG), achieve linear convergence—and, in the big data regime ($n \gg \kappa$), match optimal $O(\kappa \log(1/\epsilon))$ epochs [2104.09342], [1708.01383]. Critically, RR and single-shuffle (SO) outperform with-replacement counterparts both in theory and practice.

Greedy or optimized orderings (herding, Gradient Balancing/GraB) further accelerate convergence, in some settings eliminating the $n^{1/3}$ dependency to achieve optimal $O(T^{-2/3})$ rates for smooth nonconvex objectives, though with increased memory/computation [2205.10733].

| Method           | Nonconvex Rate           | PL/Strong Convexity Rate      |
|------------------|-------------------------|-------------------------------|
| SGD-replacement  | $O(T^{-1/2})$           | $O(T^{-1})$                   |
| RR               | $O(n^{1/3}T^{-2/3})$    | $O(nT^{-2})$                  |
| Herding/GraB     | $\widetilde{O}(T^{-2/3})$| $\widetilde{O}(T^{-2})$      |
| VR (e.g. RR-SVRG)| $O(\kappa^{3/2}\log(1/\epsilon))$ | $O(\kappa\log(1/\epsilon))$, big data |

## 4. Distributed, Federated, and Proximal RR

In networked/distributed environments, RR variants (D-RR, FedRR, GT-RR, ED-RR) inherit the improved rates from centralized RR. For distributed smooth strongly convex objectives, RR achieves $O(1/(m T^2))$ rates, matching centralized results up to network-dependent constants determined by the spectral gap. For nonconvex objectives, distributed RR matches the centralized $O(1/(m^{1/3} T^{2/3}))$ rates [2306.12037], [2112.15287], [2111.03820].

Federated RR algorithms extend these benefits to practical federated learning, providing resilience to data heterogeneity, reduced communication, and efficient variance reduction and compressed communication, with theoretically guaranteed convergence matching (or surpassing) Local SGD and its extensions [2102.06704], [2205.03914].

In composite, nonsmooth, or constrained problems, ProxRR and its distributed/federated variants leverage RR to reduce the number of expensive proximal operator evaluations by a factor $n$, at no loss of convergence speed compared to per-iteration prox methods [2102.06704], [2312.01047].

## 5. Algorithmic Properties: Bias, Variance, and Practical Implications

RR's key algorithmic advantage is the suppression of stochastic gradient variance within each epoch, leading to:
- **Smaller steady-state error:** $O(\mu^2)$ for RR versus $O(\mu)$ for SGD under constant step-size in the strongly convex regime [1803.07964], [2006.05988].
- **Oscillatory intra-epoch error:** RR exhibits periodicity reflecting bias/variance tied to position within the epoch, a phenomenon not present in with-replacement SGD [1803.07964], [2006.05988].
- **Reduced bias in stochastic sampling:** When used within stochastic gradient Langevin dynamics, RR provably reduces bias in the stationary distribution measured in Wasserstein distance relative to with-replacement schemes [2501.16055].
- **Empirical acceleration:** RR achieves faster epochs-to-solution and improved cache efficiency versus with-replacement methods, crucial for large-scale implementations [2006.05988], [2501.16055].

Concentration results for RR have led to tight high-probability complexity bounds and practical stopping criteria, such as the RR-sc rule for first-order optimality [2311.11841].

## 6. Applications and Extensions

RR is integral to the practical training of deep neural networks, regularized generalized linear models, matrix-factorization, and reinforcement learning. Recent extensions include:
- **Distributed and federated learning:** RR improves performance and communication efficiency in distributed and federated systems, including under lossy communication (compression), data heterogeneity, and variance reduction [2306.12037], [2205.03914], [2112.15287].
- **Experience replay:** Translating RR’s variance reduction to reinforcement learning settings enhances sample efficiency in off-policy RL with both uniform and prioritized experience replay buffers [2503.02269].
- **Stochastic sampling:** RR dramatically suppresses sampling bias in stochastic gradient Langevin dynamics, with improved stationary distribution accuracy and practical hardware efficiency [2501.16055].
- **Nonsmooth and constraint-heavy regimes:** Norm-PRR and related methods extend RR’s theory to nonsmooth/nonconvex composite objectives, with efficient constraints handling and sharpened complexity [2312.01047].

## References

- "A New Random Reshuffling Method for Nonsmooth Nonconvex Finite-sum Optimization" [2312.01047]
- "Distributed Random Reshuffling Methods with Improved Convergence" [2306.12037]
- "Stochastic Learning under Random Reshuffling with Constant Step-sizes" [1803.07964]
- "GraB: Finding Provably Better Data Permutations than Random Reshuffling" [2205.10733]
- "Random Reshuffling with Momentum for Nonconvex Problems: Iteration Complexity and Last Iterate Convergence" [2404.18452]
- "Random Reshuffling Dominates Stochastic Gradient Descent" [2606.32005]
- "Random Reshuffling: Simple Analysis with Vast Improvements" [2006.05988]
- "Experience Replay with Random Reshuffling" [2503.02269]
- "Distributed stochastic proximal algorithm with random reshuffling for non-smooth finite-sum optimization" [2111.03820]
- "Random Reshuffling with Variance Reduction: New Analysis and Better Rates" [2104.09342]
- "Why Random Reshuffling Beats Stochastic Gradient Descent" [1510.08560]
- "A simple linear convergence analysis of the reshuffling Kaczmarz method" [2410.01140]
- "Convergence of Random Reshuffling Under The Kurdyka-Łojasiewicz Inequality" [2110.04926]
- "Fast Convergence of Random Reshuffling under Over-Parameterization and the Polyak-Łojasiewicz Condition" [2304.00459]
- "Proximal and Federated Random Reshuffling" [2102.06704]
- "Federated Random Reshuffling with Compression and Variance Reduction" [2205.03914]
- "Variance-Reduced Stochastic Learning under Random Reshuffling" [1708.01383]
- "High Probability Guarantees for Random Reshuffling" [2311.11841]
- "Distributed Random Reshuffling over Networks" [2112.15287]
- "Random Reshuffling for Stochastic Gradient Langevin Dynamics" [2501.16055]

Source: https://www.emergentmind.com/topics/random-reshuffling