Maximum Sample Reuse (MSR) in ML & Storage

• Maximum Sample Reuse (MSR) is a multifaceted concept with domain-specific meanings, ranging from aggressive data reuse in machine learning to optimal repair in distributed storage.
• In reinforcement learning and black-box optimization, methods like GePPO, SMR, and NSR maximize sample efficiency while carefully managing instability, bias, and overfitting.
• In coding theory, MSR denotes Minimum Storage Regenerating codes that minimize repair bandwidth, illustrating a distinct, rigorous approach to data recovery in storage systems.

Maximum Sample Reuse (MSR) is not a single standardized technical term across the arXiv literature. In reinforcement learning, black-box optimization, and variance-reduction methods, it denotes the attempt to extract as much learning signal as possible from a fixed set of trajectories, transitions, or stochastic queries before collecting new data. In distributed storage, by contrast, “MSR” conventionally means Minimum Storage Regenerating codes, a class of MDS codes defined by optimal repair bandwidth rather than data reuse. In Mining Software Repositories, “MSR” names the research area itself, and recent work studies reuse at the level of annotated datasets. The resulting literature is unified less by a common formal definition than by recurring trade-offs among reuse, stability, variance, and operational constraints (Queeney et al., 2021, Lyu et al., 2023, Guruswami et al., 2019, Ather et al., 29 May 2026).

1. Terminological scope and domain-specific meanings

The phrase “Maximum Sample Reuse” is used most naturally in machine learning and optimization settings where the same observations, simulator calls, or sampled indices can be exploited multiple times. In those literatures, the central question is how far reuse can be pushed before instability, bias, or overfitting dominates. In coding theory, the acronym is already occupied: MSR means Minimum Storage Regenerating, not maximum reuse of observations or data points (Alrabiah et al., 2019, Ramkumar et al., 2024).

Context	Meaning of MSR	Representative mechanism
Reinforcement learning	Maximizing useful reuse of trajectories or transitions	GePPO, SMR, NSR
Black-box optimization and variance reduction	Reusing previously generated randomness or evaluated samples	Importance mixing, IGO reuse, pseudo-independent reuse
Mining Software Repositories	Reuse of existing MSR datasets	MIRAGE metadata enrichment
Distributed storage	Minimum Storage Regenerating	Exact MSR and $\varepsilon$-MSR codes

This terminological split is explicit in several papers. The GePPO paper states that “Maximum Sample Reuse (MSR)” is not a named concept there, but that the work is about how far one can safely push sample reuse in PPO without breaking policy improvement guarantees. The MIRAGE paper uses “MSR” strictly in the Mining Software Repositories sense and discusses maximizing reuse of existing datasets through richer annotation. The coding-theory papers explicitly state that “MSR” means Minimum Storage Regenerating rather than Maximum Sample Reuse (Queeney et al., 2021, Ather et al., 29 May 2026, Guruswami et al., 2019).

2. Reinforcement learning: off-policy reuse, repeated updates, and novelty weighting

In reinforcement learning, MSR-style methods differ mainly in what is being reused and what safety mechanism constrains the reuse. GePPO generalizes PPO by reusing data from the last $M$ policies,

$\pi_k,\pi_{k-1},\dots,\pi_{k-M+1},$

rather than discarding trajectories after a single on-policy update. At iteration $k$, it collects $n$ fresh samples with $\pi_k$, then uses $n$ samples from each of the last $M$ policies to approximate $L^{\text{GePPO}_k(\pi)}$, so each update uses $Mn$ samples and each sample can be used in updates for up to $M$0 iterations. Its safety mechanism is a generalized policy improvement lower bound with an importance-sampled surrogate term and a penalty term based on total variation distance, together with generalized clipping around

$M$1

and an adaptive learning rate that decreases when $M$2 and increases when $M$3. The paper also derives the scaling

$M$4

which ties safe trust-region size directly to the effective age of reused data. On six MuJoCo tasks, with $M$5, GePPO improved average performance over $M$6M steps by $M$7 on Swimmer, $M$8 on Hopper, $M$9 on HalfCheetah, $\pi_k,\pi_{k-1},\dots,\pi_{k-M+1},$0 on Walker2d, $\pi_k,\pi_{k-1},\dots,\pi_{k-M+1},$1 on Ant, and $\pi_k,\pi_{k-1},\dots,\pi_{k-M+1},$2 on Humanoid; the steps required to match PPO’s final performance fell to $\pi_k,\pi_{k-1},\dots,\pi_{k-M+1},$3M, $\pi_k,\pi_{k-1},\dots,\pi_{k-M+1},$4M, $\pi_k,\pi_{k-1},\dots,\pi_{k-M+1},$5M, $\pi_k,\pi_{k-1},\dots,\pi_{k-M+1},$6M, $\pi_k,\pi_{k-1},\dots,\pi_{k-M+1},$7M, and $\pi_k,\pi_{k-1},\dots,\pi_{k-M+1},$8M respectively, versus $\pi_k,\pi_{k-1},\dots,\pi_{k-M+1},$9M for PPO (Queeney et al., 2021).

A second line of work treats reuse as repeated optimization over the same sampled batch. “Sample Multiple Reuse” reuses one replay-buffer mini-batch for $k$0 consecutive actor-critic updates before resampling. In tabular nonreturnable MDPs, this yields an effective learning rate

$k$1

and the paper proves convergence of Q-SMR under standard assumptions for tabular Q-learning. In continuous control, the same mechanism is applied to SAC, TD3, TQC, DARC, REDQ, and DrQ-v2. With SAC-SMR at $k$2, the number of environment steps needed to reach fixed score thresholds dropped from $k$3K to $k$4K on Hopper-v2 at $k$5, from $k$6K to $k$7K on Ant-v2 at $k$8, from $k$9K to $n$0K on HalfCheetah-v2 at $n$1, and from $n$2K to $n$3K on Walker2d-v2 at $n$4. The paper also reports that $n$5 already improves sample efficiency, that performance generally improves up to $n$6, and that $n$7 may help further but increases compute and overfitting risk (Lyu et al., 2023).

A third approach reallocates reuse unevenly across states. NSR, “Novelty-guided Sample Reuse,” measures state novelty with Random Network Distillation and converts the normalized novelty score into a clipped weight $n$8. The critic loss is multiplied by $n$9 inside the temporal-difference residual and the actor loss by $\pi_k$0, so rare states receive more effective updates while frequent states retain at least unit weight. In Fetch Reach, Push, and PickAndPlace, NSR reaches higher success rates than DDPG-HER at the same number of training steps. The paper also shows that literal repeated updates can be slightly better in some cases but at significantly higher wall-clock time, while novelty-based weights outperform both mean-weight and random-weight ablations, indicating that the location of extra reuse matters, not only its magnitude (Duan et al., 2024).

3. Black-box optimization and variance reduction: importance weighting, rejection sampling, and pseudo-independence

In evolutionary policy search, the reuse problem is distributional rather than temporal-difference based. Importance mixing constructs a new generation that remains i.i.d. from the new sampling distribution $\pi_k$1 while reusing as many old evaluated policies from $\pi_k$2 as possible. Old samples are accepted with probability

$\pi_k$3

and newly drawn samples from the new distribution are accepted with probability

$\pi_k$4

Extended Importance Mixing applies the same logic to several past generations. Empirically, however, most effective reuse comes from the immediately preceding generation: total reuse was about $\pi_k$5 for ES, $\pi_k$6 for CEM, and $\pi_k$7–$\pi_k$8 for CMA-ES and SNES, but the fraction attributable specifically to the extended archive was $\pi_k$9 for ES, about $n$0 for CEM, and only about $n$1–$n$2 for CMA-ES and SNES (Pourchot et al., 2018).

In Information Geometric Optimization, reuse is expressed through importance sampling over a mixture of current and past search distributions,

$n$3

so that samples from the last $n$4 generations contribute to both quantile estimation and natural-gradient estimation with weights proportional to $n$5. The paper proves that this mixture-based estimator has lower variance than the corresponding sum of per-distribution importance-sampling estimators under the stated condition $n$6 constant, and shows empirically that $n$7 already yields strong gains, that $n$8–$n$9 captures most of the remaining benefit, and that large $M$0 can hurt on multimodal functions such as Rastrigin (Shirakawa et al., 2018).

The 2025 variance-reduction framework generalizes reuse further by separating randomness into an oblivious seed $M$1, which generates the sample queries, and an adaptive seed $M$2, which governs the remaining randomness of the sub-solver. Its meta-algorithm $M$3 has a Reuse mode in which the same $M$4 is used in all outer iterations. The key technical notion is $M$5-pseudo-independence: a randomized algorithm $M$6 is pseudo-independent of $M$7 if, for most realizations of $M$8, its output distribution is close in total variation to that of a randomized algorithm $M$9 that does not depend on $L^{\text{GePPO}_k(\pi)}$0 at all. The paper proves that adding uniform noise converts sufficiently accurate sub-solvers into pseudo-independent ones, and then uses this to improve full-batch versus sample-query trade-offs. For finite-sum minimization, the trade-off becomes

$L^{\text{GePPO}_k(\pi)}$1

improving on the prior

$L^{\text{GePPO}_k(\pi)}$2

Analogous improvements are given for top-eigenvector computation, discounted MDPs, average-reward MDPs, and matrix games (Jin et al., 2 Sep 2025).

4. Dataset-level reuse in Mining Software Repositories

In Mining Software Repositories, reuse refers not to replaying trajectories or resampling gradients but to making existing datasets easier to discover, evaluate, and repurpose. MIRAGE extends an earlier “Directory of MSR Datasets” by collecting paper-level metadata for MSR papers from $L^{\text{GePPO}_k(\pi)}$3 to $L^{\text{GePPO}_k(\pi)}$4 through the Semantic Scholar API, applying LDA topic modeling with $L^{\text{GePPO}_k(\pi)}$5 topics to preprocessed abstracts, and adding dataset-level attributes such as hosting platform, dataset format, accessibility, dataset type, research method, reusability score, and quality flag. The number of attributes per dataset increases from about $L^{\text{GePPO}_k(\pi)}$6 to more than $L^{\text{GePPO}_k(\pi)}$7, enabling multi-dimensional filtering and a more explicit FAIR-oriented view of reusability (Ather et al., 29 May 2026).

The empirical analysis is descriptive rather than causal, but it isolates several patterns associated with higher reuse and citation impact. Software Issues datasets are the most frequent type at about $L^{\text{GePPO}_k(\pi)}$8, while Version Control datasets have the highest citation averages. Among hosting platforms, “Other platforms” show about $L^{\text{GePPO}_k(\pi)}$9 average citations, GitHub about $Mn$0, Zenodo about $Mn$1, and Bitbucket or Figshare about $Mn$2–$Mn$3. Among dataset formats, ZIP archives average about $Mn$4 citations whereas “Unknown” format averages about $Mn$5. The paper also notes one topic with about $Mn$6 average citations. It argues that GitHub’s visibility and ease of cloning support reuse, that ZIP packaging simplifies download and setup, and that richer annotation improves discoverability, but it simultaneously records major limitations: citation bias by age, heuristic URL and metadata extraction, the absence of a formal FAIR score, no user study of discoverability, and no regression or causal analysis (Ather et al., 29 May 2026).

In this setting, a plausible implication is that “maximum sample reuse” becomes an infrastructural problem: reuse is bounded less by estimator variance than by metadata quality, accessibility, and packaging. MIRAGE treats those factors as operational proxies for reuse potential rather than as a single numerical objective.

5. Minimum Storage Regenerating codes: the unrelated but standard meaning of MSR

In coding theory, MSR has a precise and unrelated meaning. For an $Mn$7 MDS code over a field $Mn$8 with sub-packetization $Mn$9 over a base field $M$00, the repair bandwidth for a node $M$01 and helper set $M$02 is bounded below by the cut-set bound

$M$03

and equality is attained when each helper returns exactly

$M$04

subsymbols. A code achieving this bound is a Minimum Storage Regenerating code (Guruswami et al., 2019).

The core obstruction in exact MSR coding is sub-packetization. For general linear exact-repair MSR codes with redundancy $M$05, the 2019 lower-bound paper proves that one must have

$M$06

which the abstract describes as settling a central open question. The result shows that exponentially large sub-packetization is inherent for linear exact-repair MSR codes, not merely an artifact of existing constructions (Alrabiah et al., 2019).

This motivates near-MSR relaxations. For Reed–Solomon codes, the 2019 $M$07-MSR paper defines $M$08-MSR by allowing repair bandwidth at most a factor $M$09 above the cut-set bound while keeping per-helper download within the same factor, and constructs constant-rate $M$10-MSR Reed–Solomon codes with polynomial sub-packetization

$M$11

instead of the $M$12 behavior associated with exact MSR Reed–Solomon codes. A later 2024 paper constructs $M$13-MSR codes for any set of helper nodes, for any $M$14, and shows existence of codes with $M$15 for fixed $M$16 and $M$17, linear field size, and a multi-erasure extension that supports all $M$18 and all $M$19 simultaneously (Guruswami et al., 2019, Ramkumar et al., 2024).

The explicit high-rate coupled-layer construction of 2016 gives a concrete exact-MSR family with

$M$20

so that with $M$21,

$M$22

It provides all-node exact repair, field size $M$23, and a coupled-layer architecture built from a scalar $M$24 MDS code of block length $M$25. The same architecture is further generalized to vector binary MDS codes, allowing computations over fields as small as $M$26 in an appropriate vector-symbol model (Sasidharan et al., 2016).

6. Recurring constraints, common misconceptions, and comparative themes

Across the literatures that actually use MSR in the sample-reuse sense, reuse is never unconstrained. GePPO limits reuse through a finite horizon $M$27, policy-age weights $M$28, generalized clipping, and adaptive control of empirical total variation. SMR exposes reuse through an explicit ratio $M$29, but the paper reports that very large $M$30 can increase compute and overfitting risk, and that DDPG-SMR on Ant-v2 underperforms vanilla DDPG, apparently because repeated updates amplify overestimation bias. NSR bounds reuse asymmetrically by clipping novelty weights to $M$31, thereby preventing any single state from receiving unbounded effective update strength (Queeney et al., 2021, Lyu et al., 2023, Duan et al., 2024).

In the importance-sampling and variance-reduction literatures, the limiting factor is distribution drift or dependence on reused randomness. Importance mixing shows that archive depth is formally possible but empirically weak because older generations rapidly lose overlap with the current search distribution. The IGO reuse paper shows similar saturation: moderate $M$32 helps, but very large $M$33 offers little benefit and can hurt on multimodal objectives. The 2025 framework makes the same point in a more abstract form: only the oblivious randomness $M$34 is reused, while adaptive randomness $M$35 remains fresh, and reuse becomes valid only after the sub-solver is converted into a pseudo-independent one (Pourchot et al., 2018, Shirakawa et al., 2018, Jin et al., 2 Sep 2025).

A recurrent misconception is that “MSR” has a uniform meaning across technical fields. The evidence is the opposite. In reinforcement learning and optimization, the phrase denotes aggressive reuse of data, batches, or randomness; in Mining Software Repositories, reuse concerns annotated dataset artifacts; in coding theory, MSR denotes Minimum Storage Regenerating codes and has no relation to sample reuse at all. Another misconception is that stronger reuse automatically implies better performance. The available results are more conditional: MIRAGE’s citation patterns are explicitly exploratory and not supported by regression or causal analysis, NSR is empirical and does not provide formal convergence guarantees, and SMR’s improvements depend on the stability properties of the base algorithm (Ather et al., 29 May 2026, Duan et al., 2024, Guruswami et al., 2019).

This suggests that, in practice, “maximum” reuse is usually a constrained optimum rather than a literal maximum. The operative question is not whether every sample can be reused indefinitely, but under which structural conditions—trust regions, bounded novelty weights, rejection-sampling corrections, importance weights, pseudo-independence, or richer metadata—the marginal value of further reuse remains positive rather than destabilizing.