ReLoop: Iterative Self-Correction Framework

• ReLoop is defined as a cross-domain iterative self-correction mechanism that reintroduces intermediate outputs to refine model predictions.
• The framework employs closed-loop feedback, using previous errors or residuals to drive targeted adjustments in recommender systems, multimodal hallucinatory mitigation, and optimization tasks.
• Its adaptable design spans multiple domains with tailored implementations, demonstrating measurable improvements in metrics such as AUC, CTR uplift, and semantic accuracy.

Searching arXiv for the specified ReLoop-related papers to ground the article in the cited literature. ReLoop is a recurrent label for methods that close a feedback loop between a model’s current behavior and a subsequent correction, consolidation, or verification stage. In the literature, the name denotes a self-correction continual learning loop for recommender systems, a responsive error-compensation loop for non-stationary recommendation, a ring-shaped closed-loop training framework for multimodal hallucination mitigation, and a structured modeling and behavioral verification framework for reliable LLM-based optimization; in additional technical usages, it also appears in repetitive-control-based iterative learning control and in depth-recurrent state-space modeling (Cai et al., 2022, Zhu et al., 2023, Yang et al., 7 Jul 2025, Lian et al., 17 Feb 2026, Krishnamoorthy et al., 2014, Farsang et al., 15 May 2026). The repeated reuse of the name suggests a common emphasis on iterative self-correction, although the objectives, mathematical structures, and empirical claims are domain-specific.

1. Terminological scope and recurring loop structure

Across its major usages, ReLoop designates a closed-loop procedure in which an intermediate artifact from one stage is explicitly reintroduced into a later stage. In recommender systems, that artifact is the previous model’s pointwise error; in ReLoop2 it is a serve-time memory of recent residuals; in multimodal training it is a backward consistency signal derived from the model’s own answer; in optimization it is solver-grounded behavioral evidence about whether a formulation responds correctly to perturbation. A plausible implication is that the name functions less as a single canonical algorithm than as a cross-domain descriptor for feedback-based refinement.

Usage	Core loop	Reported result
ReLoop for recommender systems	New model versions are trained to reduce prediction errors over the previous model version	AUC gains of +1.4%…+4.7% across four public sets; overall average uplift ≈ +1.46% CTR
ReLoop2	A fast error memory compensates base-model bias during serving and is refreshed with new observations	ReLoop2 consistently outperforms a minute-level incremental learning baseline on the production dataset
ReLoop for MLLMs	A ring-shaped closed loop combines semantic reconstruction, visual description, and attention supervision	Object hallucination 24.5% → 10.3%; CHAIRₛ 49.0 → 38.8 on MiniGPT-4
ReLoop for optimization	Structured generation is followed by execution recovery and solver-based behavioral verification	Correctness 22.6% → 31.1% and execution 72.1% → 100.0% on the strongest model

2. Self-correction continual learning in recommender systems

In recommender systems, ReLoop is introduced as a self-correction learning loop that augments the conventional train–serve–log–train cycle with a loss term that explicitly remembers where the previous model made mistakes. Model $M_{t-1}$ scores candidate user–item pairs and records $\hat y_i^{(t-1)}$; the next training round assembles $\{(x_i,y_i,\hat y_i^{(t-1)})\}$ over a sliding window and minimizes

$$L=(1-\alpha)L_{\mathrm{CE}}+\alpha L_{\mathrm{sc}},\quad \alpha\in[0,1],$$

where

$$L_{\mathrm{CE}}(y,\hat y)= -\bigl[y\log\hat y+(1-y)\log(1-\hat y)\bigr]$$

and

$$L_{\mathrm{sc}}(y,\hat y;\hat y_{\mathrm{last}}) = y\max(\hat y_{\mathrm{last}}-\hat y,0) + (1-y)\max(\hat y-\hat y_{\mathrm{last}},0).$$

The associated error terms are $$e_i^{\text{last}}=\lvert y_i-\hat y_i^{(t-1)}\rvert$$ and $e_i^{\text{curr}}=\lvert y_i-\hat y_i\rvert$, and the extra penalty is imposed only if the new model does worse on a sample that was already mis-predicted. The paper contrasts this mechanism with plain incremental retraining, knowledge distillation, and continual-learning approaches for catastrophic forgetting; knowledge distillation, in particular, is described as encouraging replication of the teacher’s outputs everywhere rather than correction of mispredictions (Cai et al., 2022).

The empirical study uses Criteo, Avazu, MovieLens tag logs, Frappe, and a private industrial “Production” dataset. The reported offline metrics are AUC and Logloss. Applying ReLoop to DCN and DeepFM yields consistent AUC gains of +1.4%…+4.7% across the four public sets. One explicit example is Criteo, where DeepFM alone obtains AUC $=0.8025$, Logloss $=0.4501$, and ReLoop+DeepFM obtains AUC $\hat y_i^{(t-1)}$0 and Logloss $\hat y_i^{(t-1)}$1. On the industrial dataset, the baseline has AUC $\hat y_i^{(t-1)}$2, Logloss $\hat y_i^{(t-1)}$3; the KD variant has AUC $\hat y_i^{(t-1)}$4, Logloss $\hat y_i^{(t-1)}$5; and the ReLoop variant has AUC $\hat y_i^{(t-1)}$6, Logloss $\hat y_i^{(t-1)}$7. The online A/B test uses two traffic buckets, each $\hat y_i^{(t-1)}$8K active users, over 1 week, and reports day-by-day CTR uplifts of $\hat y_i^{(t-1)}$9, $\{(x_i,y_i,\hat y_i^{(t-1)})\}$0, $\{(x_i,y_i,\hat y_i^{(t-1)})\}$1, $\{(x_i,y_i,\hat y_i^{(t-1)})\}$2, $\{(x_i,y_i,\hat y_i^{(t-1)})\}$3, $\{(x_i,y_i,\hat y_i^{(t-1)})\}$4, and $\{(x_i,y_i,\hat y_i^{(t-1)})\}$5, for an overall average uplift $\{(x_i,y_i,\hat y_i^{(t-1)})\}$6. The method is described as incurring no extra inference cost and only a small $\{(x_i,y_i,\hat y_i^{(t-1)})\}$7–$\{(x_i,y_i,\hat y_i^{(t-1)})\}$8 overhead in training time. The ablations vary $\{(x_i,y_i,\hat y_i^{(t-1)})\}$9 from $$L=(1-\alpha)L_{\mathrm{CE}}+\alpha L_{\mathrm{sc}},\quad \alpha\in[0,1],$$0 to $$L=(1-\alpha)L_{\mathrm{CE}}+\alpha L_{\mathrm{sc}},\quad \alpha\in[0,1],$$1, with a sweet spot typically around $$L=(1-\alpha)L_{\mathrm{CE}}+\alpha L_{\mathrm{sc}},\quad \alpha\in[0,1],$$2–$$L=(1-\alpha)L_{\mathrm{CE}}+\alpha L_{\mathrm{sc}},\quad \alpha\in[0,1],$$3. The stated limitations are that the current error measure is pointwise, the implementation is purely loss-based, and a formal analysis of convergence or generalization remains open.

3. ReLoop2 and responsive error compensation at test time

ReLoop2 extends the original ReLoop framework from training-time self-correction to test-time adaptation in non-stationary recommendation. Its architecture is explicitly cast as a slow–fast pair inspired by complementary learning systems: a slow module $$L=(1-\alpha)L_{\mathrm{CE}}+\alpha L_{\mathrm{sc}},\quad \alpha\in[0,1],$$4 produces the base CTR prediction, a fast module $$L=(1-\alpha)L_{\mathrm{CE}}+\alpha L_{\mathrm{sc}},\quad \alpha\in[0,1],$$5 stores recent prediction errors in an error memory, and an error estimator $$L=(1-\alpha)L_{\mathrm{CE}}+\alpha L_{\mathrm{sc}},\quad \alpha\in[0,1],$$6 retrieves similar records to estimate the current model bias. The serve-time correction is

$$L=(1-\alpha)L_{\mathrm{CE}}+\alpha L_{\mathrm{sc}},\quad \alpha\in[0,1],$$7

with $$L=(1-\alpha)L_{\mathrm{CE}}+\alpha L_{\mathrm{sc}},\quad \alpha\in[0,1],$$8. The memory stores

$$L=(1-\alpha)L_{\mathrm{CE}}+\alpha L_{\mathrm{sc}},\quad \alpha\in[0,1],$$9

where $$L_{\mathrm{CE}}(y,\hat y)= -\bigl[y\log\hat y+(1-y)\log(1-\hat y)\bigr]$$0 is a hidden-layer vector. Given a query key $$L_{\mathrm{CE}}(y,\hat y)= -\bigl[y\log\hat y+(1-y)\log(1-\hat y)\bigr]$$1, ReLoop2 retrieves a top-$$L_{\mathrm{CE}}(y,\hat y)= -\bigl[y\log\hat y+(1-y)\log(1-\hat y)\bigr]$$2 neighbor set $$L_{\mathrm{CE}}(y,\hat y)= -\bigl[y\log\hat y+(1-y)\log(1-\hat y)\bigr]$$3, forms attention weights

$$L_{\mathrm{CE}}(y,\hat y)= -\bigl[y\log\hat y+(1-y)\log(1-\hat y)\bigr]$$4

computes $$L_{\mathrm{CE}}(y,\hat y)= -\bigl[y\log\hat y+(1-y)\log(1-\hat y)\bigr]$$5 and $$L_{\mathrm{CE}}(y,\hat y)= -\bigl[y\log\hat y+(1-y)\log(1-\hat y)\bigr]$$6, and estimates

$$L_{\mathrm{CE}}(y,\hat y)= -\bigl[y\log\hat y+(1-y)\log(1-\hat y)\bigr]$$7

In practice the paper sets $$L_{\mathrm{CE}}(y,\hat y)= -\bigl[y\log\hat y+(1-y)\log(1-\hat y)\bigr]$$8, so $$L_{\mathrm{CE}}(y,\hat y)= -\bigl[y\log\hat y+(1-y)\log(1-\hat y)\bigr]$$9, and therefore

$$L_{\mathrm{sc}}(y,\hat y;\hat y_{\mathrm{last}}) = y\max(\hat y_{\mathrm{last}}-\hat y,0) + (1-y)\max(\hat y-\hat y_{\mathrm{last}},0).$$0

To obtain constant-time read/write, the memory is realized as a count-sketch based on LSH signed random projections, with fixed memory $$L_{\mathrm{sc}}(y,\hat y;\hat y_{\mathrm{last}}) = y\max(\hat y_{\mathrm{last}}-\hat y,0) + (1-y)\max(\hat y-\hat y_{\mathrm{last}},0).$$1 and $$L_{\mathrm{sc}}(y,\hat y;\hat y_{\mathrm{last}}) = y\max(\hat y_{\mathrm{last}}-\hat y,0) + (1-y)\max(\hat y-\hat y_{\mathrm{last}},0).$$2 access (Zhu et al., 2023).

The experiments use AmazonElectronics, MicroVideo, KuaiVideo, and a production dataset of $$L_{\mathrm{sc}}(y,\hat y;\hat y_{\mathrm{last}}) = y\max(\hat y_{\mathrm{last}}-\hat y,0) + (1-y)\max(\hat y-\hat y_{\mathrm{last}},0).$$3M records from $$L_{\mathrm{sc}}(y,\hat y;\hat y_{\mathrm{last}}) = y\max(\hat y_{\mathrm{last}}-\hat y,0) + (1-y)\max(\hat y-\hat y_{\mathrm{last}},0).$$4 days of Huawei news-feed logs. The reported metrics are AUC and user-grouped gAUC, with chronological evaluation over ten or twelve time slots. Appending ReLoop2 to the strongest base models yields, for example, AmazonElectronics with DIEN at gAUC $$L_{\mathrm{sc}}(y,\hat y;\hat y_{\mathrm{last}}) = y\max(\hat y_{\mathrm{last}}-\hat y,0) + (1-y)\max(\hat y-\hat y_{\mathrm{last}},0).$$5 and AUC $$L_{\mathrm{sc}}(y,\hat y;\hat y_{\mathrm{last}}) = y\max(\hat y_{\mathrm{last}}-\hat y,0) + (1-y)\max(\hat y-\hat y_{\mathrm{last}},0).$$6, MicroVideo with DCN-V2 at gAUC $$L_{\mathrm{sc}}(y,\hat y;\hat y_{\mathrm{last}}) = y\max(\hat y_{\mathrm{last}}-\hat y,0) + (1-y)\max(\hat y-\hat y_{\mathrm{last}},0).$$7 and AUC $$L_{\mathrm{sc}}(y,\hat y;\hat y_{\mathrm{last}}) = y\max(\hat y_{\mathrm{last}}-\hat y,0) + (1-y)\max(\hat y-\hat y_{\mathrm{last}},0).$$8, and KuaiVideo with DIEN at gAUC $$L_{\mathrm{sc}}(y,\hat y;\hat y_{\mathrm{last}}) = y\max(\hat y_{\mathrm{last}}-\hat y,0) + (1-y)\max(\hat y-\hat y_{\mathrm{last}},0).$$9 and AUC $$e_i^{\text{last}}=\lvert y_i-\hat y_i^{(t-1)}\rvert$$0. Even strong sequential models such as DIN, DIEN, and BST each gain an extra $$e_i^{\text{last}}=\lvert y_i-\hat y_i^{(t-1)}\rvert$$1–$$e_i^{\text{last}}=\lvert y_i-\hat y_i^{(t-1)}\rvert$$2 AUC when augmented with ReLoop2. On the production dataset, ReLoop2 consistently outperforms a minute-level incremental learning baseline, and the gap grows as drift intensifies. A comparison on MicroVideo shows that ReLoop2 alone matches or exceeds incremental retraining on most slots, while combining both yields the best curve, which the paper interprets as orthogonality. The ablations find best gAUC around $$e_i^{\text{last}}=\lvert y_i-\hat y_i^{(t-1)}\rvert$$3 on MicroVideo and $$e_i^{\text{last}}=\lvert y_i-\hat y_i^{(t-1)}\rvert$$4 on AmazonElectronics, and optimal $$e_i^{\text{last}}=\lvert y_i-\hat y_i^{(t-1)}\rvert$$5–$$e_i^{\text{last}}=\lvert y_i-\hat y_i^{(t-1)}\rvert$$6 depending on dataset. The listed limitations are memory staleness after base-model retraining, noisy error filtering, open questions about hybrid training, and the absence of formal convergence guarantees under streaming drift.

4. Ring-shaped closed-loop training for multimodal hallucination mitigation

In multimodal LLMs, ReLoop is a ring-shaped closed-loop training framework designed to mitigate hallucinations by making the model “seeing twice and thinking backwards.” The base model $$e_i^{\text{last}}=\lvert y_i-\hat y_i^{(t-1)}\rvert$$7 first processes an image $$e_i^{\text{last}}=\lvert y_i-\hat y_i^{(t-1)}\rvert$$8 and question $$e_i^{\text{last}}=\lvert y_i-\hat y_i^{(t-1)}\rvert$$9 to generate an answer $e_i^{\text{curr}}=\lvert y_i-\hat y_i\rvert$0 under ordinary cross-entropy training. A frozen Consistency Feedback Plugin then produces three backward signals. CFP-Lang performs semantic reconstruction: from $e_i^{\text{curr}}=\lvert y_i-\hat y_i\rvert$1, it proposes $e_i^{\text{curr}}=\lvert y_i-\hat y_i\rvert$2 candidate questions $e_i^{\text{curr}}=\lvert y_i-\hat y_i\rvert$3, and a lightweight semantic aggregator $e_i^{\text{curr}}=\lvert y_i-\hat y_i\rvert$4 based on BERT+MLP ranks them against the original $e_i^{\text{curr}}=\lvert y_i-\hat y_i\rvert$5 via BERTScore and selects $e_i^{\text{curr}}=\lvert y_i-\hat y_i\rvert$6. CFP-Vis generates a descriptive caption $e_i^{\text{curr}}=\lvert y_i-\hat y_i\rvert$7 from $e_i^{\text{curr}}=\lvert y_i-\hat y_i\rvert$8. Attention supervision extracts token-to-image cross-attention maps $e_i^{\text{curr}}=\lvert y_i-\hat y_i\rvert$9 from the decoder and compares them to an entropy-based soft pseudo-ground truth $=0.8025$0. Only $=0.8025$1 and $=0.8025$2 are updated; CFP modules and CLIP/BERT remain frozen. The losses are

$=0.8025$3

$=0.8025$4

$=0.8025$5

and

$=0.8025$6

with $=0.8025$7, $=0.8025$8, $=0.8025$9, and an adaptive consistency weight $=0.4501$0 determined by BERTScore: $=0.4501$1 if $=0.4501$2, $=0.4501$3 if $=0.4501$4 BERTScore $=0.4501$5, and $=0.4501$6 if $=0.4501$7 (Yang et al., 7 Jul 2025).

The training data comprise $=0.4501$8K high-quality $=0.4501$9 triplets from LLaVA-Instruct-150K plus contrastive hallucinated examples synthesized by perturbing object, attribute, relation, and event terms and then human-verified. Training uses $\hat y_i^{(t-1)}$00A100 GPUs, fp16, $\hat y_i^{(t-1)}$01 epochs, AdamW with $\hat y_i^{(t-1)}$02, $\hat y_i^{(t-1)}$03, weight decay $\hat y_i^{(t-1)}$04, effective batch size $\hat y_i^{(t-1)}$05, learning rate $\hat y_i^{(t-1)}$06, $\hat y_i^{(t-1)}$07 warm-up steps, and cosine decay. Quantitatively, ReLoop reduces hallucination rates across four types: object $\hat y_i^{(t-1)}$08, attribute $\hat y_i^{(t-1)}$09, relation $\hat y_i^{(t-1)}$10, and event $\hat y_i^{(t-1)}$11. On MiniGPT-4, the paper reports POPE $\hat y_i^{(t-1)}$12, CHAIRₛ $\hat y_i^{(t-1)}$13, CHAIRᵢ $\hat y_i^{(t-1)}$14, F1 $\hat y_i^{(t-1)}$15, Faith $\hat y_i^{(t-1)}$16, and FaithS $\hat y_i^{(t-1)}$17. The method also generalizes to InstructBLIP, LLaVA-1.5, mPLUG-owl, and ShareGPT4V, with benchmark-level improvements including AMBER $\hat y_i^{(t-1)}$18 versus $\hat y_i^{(t-1)}$19, MMHal-B $\hat y_i^{(t-1)}$20 versus $\hat y_i^{(t-1)}$21, and HallusionBench $\hat y_i^{(t-1)}$22 versus $\hat y_i^{(t-1)}$23. A slight decrease on MME, $\hat y_i^{(t-1)}$24 versus $\hat y_i^{(t-1)}$25, is explicitly noted as a common alignment/perception trade-off. The limitations emphasize that relation and event hallucinations remain harder because they require higher-order spatial or temporal reasoning, and that the framework depends on clean triplets and pretrained CFP modules such as CLIP and BLIP-2.

5. Structured modeling and behavioral verification for reliable LLM-based optimization

In LLM-based optimization, ReLoop addresses silent failures: cases in which generated code executes, the solver returns a feasible solution, yet the mathematical formulation is semantically incorrect. Formally, for a natural-language problem description $\hat y_i^{(t-1)}$26, the generated code is $\hat y_i^{(t-1)}$27 and the solver output is $\hat y_i^{(t-1)}$28. The paper defines semantic correctness as exact agreement between the encoded feasible region and objective and those intended in $\hat y_i^{(t-1)}$29, and defines a silent failure as code that executes without syntax or runtime errors, yields a feasible solution, but is not semantically correct. On RetailOpt-190, state-of-the-art models can achieve up to $\hat y_i^{(t-1)}$30 solver-feasible execution yet only $\hat y_i^{(t-1)}$31 semantic correctness, producing a $\hat y_i^{(t-1)}$32pp feasibility–correctness gap. ReLoop combines two mechanisms. The first is structured generation, a four-stage pipeline

$\hat y_i^{(t-1)}$33

where the formalization stage writes $\hat y_i^{(t-1)}$34 and requires explicit variable-type reasoning, and synthesis enforces that all parameters be read from a pre-loaded data[...] dictionary rather than hardcoded literals. The second is behavioral verification, which perturbs parameters and checks whether the optimum changes substantially: $\hat y_i^{(t-1)}$35 Constraint Presence Testing perturbs capacity to $\hat y_i^{(t-1)}$36original, demand to $\hat y_i^{(t-1)}$37original, and other parameters to $\hat y_i^{(t-1)}$38original. Objective Presence Testing perturbs cost or revenue coefficients analogously. The thresholds are $\hat y_i^{(t-1)}$39 and $\hat y_i^{(t-1)}$40: $\hat y_i^{(t-1)}$41 is a warning and triggers repair, $\hat y_i^{(t-1)}$42 is logged as uncertain, and $\hat y_i^{(t-1)}$43 or perturbation-induced infeasibility is a pass (Lian et al., 17 Feb 2026).

Execution recovery forms an additional verification layer. Before behavioral testing, ReLoop performs syntax parsing, runtime execution with a $\hat y_i^{(t-1)}$44 s timeout, solver-status checks, and a duality-gap check. If the model is infeasible, it computes the Irreducible Inconsistent Subsystem; if unbounded, it identifies unbounded ray variables. Any fatal error triggers up to $\hat y_i^{(t-1)}$45 regenerations, each supplied with IIS or unbounded-ray diagnostics. The benchmarks are RetailOpt-190, MAMO-ComplexLP, and IndustryOR; the models span foundation, supervised-finetuned, and solver-informed RL paradigms. On RetailOpt-190, for Claude Opus 4.6 under pass@1 greedy decoding, the reported numbers are Exec\% $\hat y_i^{(t-1)}$46 for Base, $\hat y_i^{(t-1)}$47CoT, and $\hat y_i^{(t-1)}$48ReLoop, and Acc\% $\hat y_i^{(t-1)}$49 $\hat y_i^{(t-1)}$50. The abstract summarizes the strongest-model result as correctness $\hat y_i^{(t-1)}$51 and execution $\hat y_i^{(t-1)}$52, with consistent gains across five models and three benchmarks. The paper’s interpretation is complementary rather than monolithic: structured generation dominates on highly compositional problems, behavioral verification is the largest single contributor on localized formulation defects, and execution recovery is especially useful for models that crash under CoT. The stated limitations are prompt-format mismatch for some SFT models, the runtime overhead of up to $\hat y_i^{(t-1)}$53 perturbation tests, and residual undetectable errors such as wrong decompositions or coefficient-scale errors within the $\hat y_i^{(t-1)}$54 buffer.

6. Control-theoretic and sequence-model usages

In a control-theoretic usage, ReLoop denotes a zero-phase repetitive-control-based iterative learning control design. The key object is a noncausal zero-phase filter

$\hat y_i^{(t-1)}$55

whose lifted representation is a symmetric banded Toeplitz matrix. For a plant factorized as $\hat y_i^{(t-1)}$56, the modified ILC law chooses zero-phase learning filters $\hat y_i^{(t-1)}$57 and $\hat y_i^{(t-1)}$58, defines

$\hat y_i^{(t-1)}$59

updates

$\hat y_i^{(t-1)}$60

and obtains the state-transition matrix

$\hat y_i^{(t-1)}$61

The paper shows that $\hat y_i^{(t-1)}$62 has symmetric banded Toeplitz structure and that the sufficient frequency-domain convergence condition is

$\hat y_i^{(t-1)}$63

For the prototype case $\hat y_i^{(t-1)}$64, this reduces to $\hat y_i^{(t-1)}$65. As the data length approaches infinity, the $\hat y_i^{(t-1)}$66-norm condition becomes not only sufficient but also necessary, and the design can be translated directly into repetitive-control loop shaping (Krishnamoorthy et al., 2014).

In time-series classification with state-space models, the technical summary for “Looped SSMs” explicitly describes a ReLoop depth-recurrent architecture in which the same block $\hat y_i^{(t-1)}$67 is reused across $\hat y_i^{(t-1)}$68 layers: $\hat y_i^{(t-1)}$69 This is contrasted with an independent $\hat y_i^{(t-1)}$70-layer model $\hat y_i^{(t-1)}$71, and the expressivity relation

$\hat y_i^{(t-1)}$72

is stated explicitly. Partial sharing patterns AAAAAA, ABABAB, and ABCABC are studied for $\hat y_i^{(t-1)}$73. The companion design axis is input reshaping through a concentration hyperparameter $\hat y_i^{(t-1)}$74: low-dimensional inputs use timestep concatenation, while high-dimensional inputs are flattened and rechunked. Across four architectures and six benchmarks, the paper reports that a looped SSM with $\hat y_i^{(t-1)}$75 parameters iterated $\hat y_i^{(t-1)}$76 times consistently closely matches or outperforms a standard SSM with $\hat y_i^{(t-1)}$77 independent parameters, and that input reshaping yields accuracy gains of $\hat y_i^{(t-1)}$78–$\hat y_i^{(t-1)}$79 across all models, confirmed over $\hat y_i^{(t-1)}$80 random seeds. The reported explanation is not extra expressivity but a beneficial inductive bias from parameter sharing that simplifies optimization (Farsang et al., 15 May 2026).

7. Distinct but closely related usage: RLoop in reinforcement learning for verifiable rewards

A distinct framework with similar naming, "RLoop: An Self-Improving Framework for Reinforcement Learning with Iterative Policy Initialization," addresses RL overfitting in reinforcement learning for verifiable rewards rather than carrying the exact ReLoop spelling. Its motivating pathology is that on-policy RL training of large reasoning models can raise the in-distribution training reward $\hat y_i^{(t-1)}$81 while out-of-distribution performance peaks early and then plateaus or declines. The paper attributes this to policy over-specialization and catastrophic forgetting, and reports that checkpoint-to-checkpoint evaluations lose a non-trivial fraction, $\hat y_i^{(t-1)}$82–$\hat y_i^{(t-1)}$83, of problems solved by an earlier policy step. RLoop replaces a single long RL run with $\hat y_i^{(t-1)}$84 short RL runs interleaved with supervised consolidation. Each iteration initializes from $\hat y_i^{(t-1)}$85, runs RL for $\hat y_i^{(t-1)}$86 steps on

$\hat y_i^{(t-1)}$87

collects $\hat y_i^{(t-1)}$88, filters successful trajectories into

$\hat y_i^{(t-1)}$89

optionally restricting to “hard” prompts with empirical success rate below $\hat y_i^{(t-1)}$90, and then re-initializes from $\hat y_i^{(t-1)}$91 before a Rejection-sampling Fine-Tuning step

$\hat y_i^{(t-1)}$92

The paper states that no extra KL or entropy regularizer is required in the RFT phase because the policy is explicitly re-initialized from the stable base $\hat y_i^{(t-1)}$93 (Zhiyuan et al., 6 Nov 2025).

The experimental configuration uses Qwen-2.5-7b-Math, DAPO-17k, AIME-2024, MinervaMath, OmniMath, and MATH-500. The RL baseline runs $\hat y_i^{(t-1)}$94 steps with group size $\hat y_i^{(t-1)}$95 and max length $\hat y_i^{(t-1)}$96; RLoop uses $\hat y_i^{(t-1)}$97 iterations, each with $\hat y_i^{(t-1)}$98 RL steps plus $\hat y_i^{(t-1)}$99 epoch of RFT on successful trajectories from “hard” prompts with success $\{(x_i,y_i,\hat y_i^{(t-1)})\}$00. The average over four benchmarks is reported as Avg@32 accuracy Base $\{(x_i,y_i,\hat y_i^{(t-1)})\}$01 RL $\{(x_i,y_i,\hat y_i^{(t-1)})\}$02 RLoop $\{(x_i,y_i,\hat y_i^{(t-1)})\}$03, and Pass@32 Base $\{(x_i,y_i,\hat y_i^{(t-1)})\}$04 RL $\{(x_i,y_i,\hat y_i^{(t-1)})\}$05 RLoop $\{(x_i,y_i,\hat y_i^{(t-1)})\}$06, corresponding to relative gains over RL of $\{(x_i,y_i,\hat y_i^{(t-1)})\}$07 accuracy and $\{(x_i,y_i,\hat y_i^{(t-1)})\}$08 pass@32. Per-dataset pass@32 improvements are AIME $\{(x_i,y_i,\hat y_i^{(t-1)})\}$09 pp $\{(x_i,y_i,\hat y_i^{(t-1)})\}$10, MATH $\{(x_i,y_i,\hat y_i^{(t-1)})\}$11 pp $\{(x_i,y_i,\hat y_i^{(t-1)})\}$12, OmniMath $\{(x_i,y_i,\hat y_i^{(t-1)})\}$13 pp, and Minerva $\{(x_i,y_i,\hat y_i^{(t-1)})\}$14 pp. The stated limitations are additional overhead from multiple short RL runs and repeated fine-tuning, dependence on binary or sparse rewards in the current version, and tuning sensitivity in $\{(x_i,y_i,\hat y_i^{(t-1)})\}$15 and iteration count $\{(x_i,y_i,\hat y_i^{(t-1)})\}$16. Despite the spelling difference, its inclusion clarifies a frequent source of confusion: ReLoop is not a single universally standardized framework, but a label repeatedly attached to loop-based self-improvement mechanisms in several neighboring literatures.