Recurrence Curriculum Strategies

• Recurrence Curriculum is an umbrella concept encompassing iterative task redistribution, recursive data generation, and staged increases in recurrent computation.
• It enables dynamic reinforcement learning methods by repeatedly updating training distributions based on current performance signals to improve convergence.
• It also integrates supervised and LLM training strategies by strategically reintroducing high-value data and progressively increasing model depth to enhance efficiency.

Recurrence curriculum, as suggested by several strands of curriculum-learning research, denotes training schemes in which the curriculum is not a single fixed easy-to-hard ordering, but is instead revisited, recomputed, recursively expanded, or staged through repeated increases in internal computation. In the literature represented here, that idea appears in at least five distinct forms: iterative re-estimation of task distributions from current competence, re-use of curriculum knowledge across training runs, recursive decomposition of hard examples into easier subproblems, strategically timed reintroduction of important data, and explicit curricula over recurrent depth or latent reasoning steps (Klink et al., 2020, Portelas et al., 2020, Zhao et al., 23 Feb 2026, Bergsma et al., 29 Sep 2025, McLeish et al., 10 Nov 2025, Yang et al., 11 Jun 2026). This suggests that “recurrence curriculum” is best understood as an umbrella concept rather than a single standardized algorithm.

1. Conceptual scope

The common feature across these works is not recurrence in the narrow architectural sense, but repeated curriculum formation. In some papers, the recurrence lies in repeatedly updating a task distribution during reinforcement learning. In others, it lies in restarting training under a distilled curriculum from an earlier run, recursively generating easier data from harder data, or progressively increasing the amount of recurrent computation used by the model.

Form of recurrence	Representative papers	Core mechanism
Iterative task redistribution	(Klink et al., 2020, Wen et al., 22 May 2026)	Recompute task distributions from current competence
Multi-run curriculum reuse	(Portelas et al., 2020)	Learn a curriculum once, then retrain from scratch
Recursive data generation	(Zhao et al., 23 Feb 2026)	Decompose hard examples into easier subproblems
Timed data revisitation	(Bergsma et al., 29 Sep 2025)	Reinsert high-value data at TREC minima
Recurrence-depth staging	(McLeish et al., 10 Nov 2025, Yang et al., 11 Jun 2026)	Gradually increase recurrent computation

The boundaries of the concept are equally important. “Curriculum Learning for Handwritten Text Line Recognition” changes the difficulty of the training distribution for RNNs by biasing sampling toward shorter sequences, but it “does not alter the recurrence equations themselves” (Louradour et al., 2013). “Curriculum Learning with a Progression Function” introduces a framework based on progression functions and mapping functions; the supplied material indicates that the framework is recurrence-capable in principle, but gives no evidence that the paper itself explicitly studies revisitation, cyclic scheduling, or backward progression (Bassich et al., 2020). By contrast, “Curriculum Learning for Cumulative Return Maximization” defines a curriculum as a sequence “with no repeated tasks,” so it is explicitly non-recurrent in the sense of revisiting earlier tasks (Foglino et al., 2019).

2. Iterative task-distribution curricula in reinforcement learning

A central recurrence pattern in reinforcement learning is repeated re-estimation of the training-task distribution from the learner’s current competence. “Self-Paced Deep Reinforcement Learning” formulates curriculum generation as an inference problem over context distributions. Instead of training only on the target distribution $\mu(c)$, the method learns a training distribution $p_\nu(c)$ by optimizing

$$\max_{\nu,\omega}\; J(\nu,\omega) - \alpha\, D_{\mathrm{KL}}(p_\nu(c)\,\|\,\mu(c)),$$

then alternates between policy improvement and context-distribution update under a trust-region constraint,

$$D_{\mathrm{KL}}(p_{\nu_{i+1}}\|p_{\nu_i}) \le \epsilon.$$

The curriculum is therefore “updated every training iteration based on the current policy and value estimates,” making it recurrence-like in the explicit sense of repeated task-distribution re-estimation over rounds (Klink et al., 2020).

“Curriculum reinforcement learning with measurable task representation learning” extends this iterative logic to settings where raw task-space interpolation is misleading. Its ACRL method learns a latent task representation through a variational autoencoder that reconstructs reward and next-state behavior, then repeatedly regenerates the curriculum distribution with two update operators: Exploration Bound Update and Latent Space Prediction. The latent interpolation step is

$z'=\alpha z_{\text{tar}} + (1-\alpha)z,$

followed by decoding back to a new context. The curriculum is therefore “continuously updated,” and the paper explicitly describes a repeated loop of policy learning, task-representation learning, and curriculum regeneration (Wen et al., 22 May 2026).

Taken together, these papers define a strong technical sense of recurrence curriculum: the curriculum is not precomputed, but is repeatedly recomputed from current performance signals. A plausible implication is that recurrence in curriculum learning can be formalized at the distributional level even when no task is literally revisited by name.

3. Recurrence across runs, and the contrast with one-pass sequencing

A different recurrence pattern appears when curriculum knowledge is transferred across training runs. “Trying AGAIN instead of Trying Longer: Prior Learning for Automatic Curriculum Learning” is explicit on this point: the first run uses exploratory ALP-GMM to discover “progress niches,” and the second run retrains the same learner from scratch under a distilled “expert curriculum.” The method AGAIN combines this distilled prior with a low-exploration ALP-GMM teacher. The paper reports “50% improvements on average over the current state of the art,” and, more importantly for the concept of recurrence curriculum, states that “for all tested variants, retraining the DRL agent from scratch in the second run gave superior end performances than fine-tuning using the weights of the first run” (Portelas et al., 2020).

The first run produces a sequence of LP-weighted Gaussian mixtures over task parameters,

$\mathcal{C}_{raw} = \{p(1), \ldots, p(T)\},$

which are then filtered by a learning-progress threshold to form the second-run curriculum. The recurrence here is neither architectural nor within-episode; it is curriculum recurrence across learners and training runs. This is why the paper frames the broader direction as “Classroom Teaching.”

This stands in sharp contrast to “Curriculum Learning for Cumulative Return Maximization.” That paper optimizes curricula for critical RL settings by maximizing cumulative return on the final task,

$$\mathcal{J}(c,m_f)=\sum_{i=1}^{N}\mathbb{E}[G_f^i],$$

but defines a curriculum as a sequence “without repetitions,”

$$c=\langle m_1,\ldots,m_l\rangle,\qquad i\neq j \Rightarrow m_i\neq m_j.$$

It is therefore a static one-pass curriculum, optimized offline in simulation and then fixed, with no revisitation and no adaptive resequencing (Foglino et al., 2019). This contrast clarifies an important misconception: curricula optimized over multiple tasks are not automatically recurrence curricula.

4. Recursive data construction and sequence-aware curricula

In supervised learning, recurrence curriculum often appears as recursive or staged transformation of the training data itself. “Learning to Solve Complex Problems via Dataset Decomposition” makes this explicit through Decomp, a teacher-student framework in which the teacher “recursively generate[s] easier versions of examples.” Starting from an example $(q, cot, a)$, the teacher splits the chain-of-thought into steps, assigns concept tags, generates verified subproblems, and recursively repeats the process until a maximum depth $D$ is reached. Difficulty is then scored by

$p_\nu(c)$0

combining structural complexity and conceptual depth, and training proceeds in staged easy-to-hard bins. On MATH-500 with Qwen2.5-1.5B, the paper reports $p_\nu(c)$1 for SFT-Decomp and $p_\nu(c)$2 for SFT-Decomp + Curriculum; on AIME2025 with Qwen3-4B-Base, it reports $p_\nu(c)$3 for SFT-Decomp and $p_\nu(c)$4 for SFT-Decomp + Curriculum (Zhao et al., 23 Feb 2026).

An earlier and simpler form of recurrence-aware curriculum appears in “Curriculum Learning for Handwritten Text Line Recognition.” There, the main curriculum variable is target sequence length $p_\nu(c)$5, and the probability of sampling an example is

$p_\nu(c)$6

with $p_\nu(c)$7, $p_\nu(c)$8 initialized to $p_\nu(c)$9, and $$\max_{\nu,\omega}\; J(\nu,\omega) - \alpha\, D_{\mathrm{KL}}(p_\nu(c)\,\|\,\mu(c)),$$0 linearly decreased to $$\max_{\nu,\omega}\; J(\nu,\omega) - \alpha\, D_{\mathrm{KL}}(p_\nu(c)\,\|\,\mu(c)),$$1 over the first $$\max_{\nu,\omega}\; J(\nu,\omega) - \alpha\, D_{\mathrm{KL}}(p_\nu(c)\,\|\,\mu(c)),$$2 epochs. The paper reports that the curriculum “substantially speeds convergence” on IAM, Rimes, and OpenHaRT, and that on IAM it improves CER from about $$\max_{\nu,\omega}\; J(\nu,\omega) - \alpha\, D_{\mathrm{KL}}(p_\nu(c)\,\|\,\mu(c)),$$3 to $$\max_{\nu,\omega}\; J(\nu,\omega) - \alpha\, D_{\mathrm{KL}}(p_\nu(c)\,\|\,\mu(c)),$$4 (Louradour et al., 2013).

These two papers illustrate different meanings of recurrence. Decomp is recursively generative: a problem expands into a tree of easier subproblems. The handwritten text curriculum is recurrence-aware in a weaker sense: it changes the effective temporal credit-assignment horizon seen by the recurrent model, but remains a predefined sampling schedule rather than a recursively updated one.

5. Reintroduction timing in LLM training

In LLM training, recurrence curriculum can concern not only which data appear, but when important data should reappear. “Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMs” introduces the training re-evaluation curve (TREC),

$$\max_{\nu,\omega}\; J(\nu,\omega) - \alpha\, D_{\mathrm{KL}}(p_\nu(c)\,\|\,\mu(c)),$$5

which measures the loss of the final model on the batch originally seen at training step $$\max_{\nu,\omega}\; J(\nu,\omega) - \alpha\, D_{\mathrm{KL}}(p_\nu(c)\,\|\,\mu(c)),$$6. The paper connects TREC to AdamW’s implicit EMA coefficients,

$$\max_{\nu,\omega}\; J(\nu,\omega) - \alpha\, D_{\mathrm{KL}}(p_\nu(c)\,\|\,\mu(c)),$$7

and proposes the predictive form

$$\max_{\nu,\omega}\; J(\nu,\omega) - \alpha\, D_{\mathrm{KL}}(p_\nu(c)\,\|\,\mu(c)),$$8

The empirical claim most relevant to recurrence curriculum is that the best place for high-quality data is often not the end of training, but the minimum of the TREC for that schedule. In the 610M placement experiments, placing the $$\max_{\nu,\omega}\; J(\nu,\omega) - \alpha\, D_{\mathrm{KL}}(p_\nu(c)\,\|\,\mu(c)),$$9B-token code blend into the lowest-TREC decile yielded the best code-blend validation loss across a $$D_{\mathrm{KL}}(p_{\nu_{i+1}}\|p_{\nu_i}) \le \epsilon.$$0 decay schedule, a decay-to-zero schedule, and a step-drop schedule. In the 3.9B continual pre-training experiment, inserting a $$D_{\mathrm{KL}}(p_{\nu_{i+1}}\|p_{\nu_i}) \le \epsilon.$$1B-token high-quality math segment halfway through CPT outperformed placing it at the end across all tested learning rates (Bergsma et al., 29 Sep 2025).

This work reframes revisitation as retention-aware timing. It is “not mainly a difficulty-based curriculum”; rather, it is a curriculum over when recurring high-value data should be concentrated. The paper is also careful about scope: within a fixed schedule, TREC minima are reliable placement signals, but across schedules, lower absolute TREC does not consistently imply better downstream validation.

6. Curricula over recurrent depth and latent reasoning

A more literal form of recurrence curriculum appears when the model itself is recurrent and the training process gradually increases the amount of recurrence. “Teaching Pretrained LLMs to Think Deeper with Retrofitted Recurrence” retrofits pretrained non-recurrent transformers into depth-recurrent models with a prelude $$D_{\mathrm{KL}}(p_{\nu_{i+1}}\|p_{\nu_i}) \le \epsilon.$$2, recurrent block $$D_{\mathrm{KL}}(p_{\nu_{i+1}}\|p_{\nu_i}) \le \epsilon.$$3, and coda $$D_{\mathrm{KL}}(p_{\nu_{i+1}}\|p_{\nu_i}) \le \epsilon.$$4: $$D_{\mathrm{KL}}(p_{\nu_{i+1}}\|p_{\nu_i}) \le \epsilon.$$5 The key training device is a curriculum over the mean of the Poisson-Lognormal train-time recurrence distribution. The appendix gives a linear schedule,

$$D_{\mathrm{KL}}(p_{\nu_{i+1}}\|p_{\nu_i}) \le \epsilon.$$6

and the paper reports that linearly scheduling the mean recurrence up to $$D_{\mathrm{KL}}(p_{\nu_{i+1}}\|p_{\nu_i}) \le \epsilon.$$7 gives “slightly better validation loss as a function of training steps” and “much better validation loss as a function of FLOPs” than training from the start at full mean recurrence (McLeish et al., 10 Nov 2025).

“Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning” uses a different but related curriculum. SWITCH introduces explicit boundary tokens <swi> and </swi> and trains the model with a “visible-to-latent curriculum.” At latent positions, the input embedding is replaced by the previous hidden state,

$$D_{\mathrm{KL}}(p_{\nu_{i+1}}\|p_{\nu_i}) \le \epsilon.$$8

Phase 1 teaches the model where to enter and exit latent mode; Phase 2 progressively replaces visible text inside switch spans with <latent> positions according to

$$D_{\mathrm{KL}}(p_{\nu_{i+1}}\|p_{\nu_i}) \le \epsilon.$$9

with $z'=\alpha z_{\text{tar}} + (1-\alpha)z,$0 and $z'=\alpha z_{\text{tar}} + (1-\alpha)z,$1. The paper states that “a one-shot replacement is too aggressive,” because “the model lets the block collapse into a no-op,” and that the parallel schedule is “substantially better” than sequential conversion. After Switch-SFT, MATH-500 accuracy is $z'=\alpha z_{\text{tar}} + (1-\alpha)z,$2; after Switch-GRPO it rises to $z'=\alpha z_{\text{tar}} + (1-\alpha)z,$3 (Yang et al., 11 Jun 2026).

These papers treat recurrence curriculum as a stabilization strategy for recurrent computation itself. The curriculum does not merely order examples; it controls how much iterative internal computation the model is asked to perform at each stage.

7. Boundaries, misconceptions, and open questions

The literature makes clear that recurrence curriculum is heterogeneous. In some settings it means adaptive recomputation of task distributions; in others, recursive data construction, retraining under prior curricula, strategically timed data recurrence, or staged increases in recurrent depth. This suggests that the term is best used descriptively, not as the name of a single settled formalism.

Several misconceptions follow from that heterogeneity. First, recurrence curriculum is not synonymous with recurrent neural networks: the handwritten text work is recurrence-aware but operates through data sampling, not altered recurrence equations (Louradour et al., 2013). Second, a framework can be recurrence-capable without being recurrence-realized: the progression-function/mapping-function abstraction can support revisitation in principle, but the supplied evidence does not show that it was explicitly implemented that way (Bassich et al., 2020). Third, repeated-task revisitation is not the only form of recurrence; repeated re-estimation of a task distribution or repeated regeneration of a curriculum also qualifies in a broader sense (Klink et al., 2020, Wen et al., 22 May 2026).

The open technical questions are correspondingly diverse. “Self-Paced Deep Reinforcement Learning” notes that its implemented $z'=\alpha z_{\text{tar}} + (1-\alpha)z,$4 schedule is a “naïve heuristic” and that more expressive context distributions than Gaussians may be needed (Klink et al., 2020). ACRL depends on a parametric context space and on successful-enough trajectories to learn a meaningful latent task space (Wen et al., 22 May 2026). AGAIN amortizes exploration across runs, but the quality of the second-run curriculum depends on the success of the first run (Portelas et al., 2020). Decomp depends on a strong teacher and an expensive recursive verification pipeline (Zhao et al., 23 Feb 2026). TREC currently provides the strongest guidance within a fixed schedule rather than across schedules (Bergsma et al., 29 Sep 2025). SWITCH shows that its latent representation is shaped “primarily by the Phase 2 curriculum rather than directly by the RL objective,” which raises the broader question of how far RL alone can discover useful hidden-state recurrence without a carefully designed curriculum (Yang et al., 11 Jun 2026).

Across these variations, the durable idea is consistent: curriculum learning need not be a single forward traversal through a static difficulty ordering. It can be a repeated process of revisitation, redistribution, decomposition, or deepening, with the recurrence occurring in the training distribution, the learner population, the data construction pipeline, or the model’s own internal computation.