Curriculum Top-K Annealing in Sequence Prediction

• Curriculum Top-K Annealing is a training mechanism that uses fixed top-k candidate sets with annealed replacement probability and temperature to smoothly transition from teacher forcing towards self-conditioning.
• It addresses exposure bias in sequence prediction by substituting gold tokens with either semantically similar nearest neighbors or model-generated predictions in a controlled, curriculum-based schedule.
• Empirical results on benchmarks like Penn Treebank and WikiText-2 show notable improvements in perplexity, demonstrating enhanced model stability and performance over standard methods.

“Curriculum Top-K Annealing” is best understood as an Editor’s term for a family of curriculum mechanisms in which training is constrained by a truncated top-$k$ candidate set while some aspect of exposure to that set is annealed over training time. In the literature provided here, the clearest direct instantiation is the sequence-prediction method "Nearest-Neighbor Replacement Sampling" (NNRS) from "$k$-Neighbor Based Curriculum Sampling for Sequence Prediction," where previous gold tokens are replaced by sampled semantic neighbors drawn from a token’s top-$k$ embedding neighborhood under an epoch-level curriculum and a temperature-controlled similarity distribution (Neill et al., 2021). The phrase itself is not used in that paper. This suggests that “Curriculum Top-K Annealing” is more accurately treated as a descriptive umbrella for related designs than as a standardized author-defined term.

1. Terminological scope and provenance

The most direct source for the concept is the NNRS mechanism in sequence prediction, but several adjacent papers instantiate only parts of the phrase “Curriculum Top-K Annealing.” Some are genuinely top-$k$-based without annealing, some are annealing-based without top-$k$, and some use curriculum staging over ranked bins rather than an explicit top-$k$ cutoff.

Work	Relation to the term	Core mechanism
"$k$-Neighbor Based Curriculum Sampling for Sequence Prediction" (Neill et al., 2021)	Closest direct instantiation	Fixed top-$k$ neighbor truncation with annealed replacement rate and temperature
"PCC: Paraphrasing with Bottom-k Sampling and Cyclic Learning for Curriculum Data Augmentation" (Lu et al., 2022)	Adjacent, not identical	Bottom-$k$ decoding for hard paraphrases plus cyclic curriculum bins
"HuCurl: Human-induced Curriculum Discovery" (Elgaar et al., 2023)	Adjacent, not exact top-$k$	Difficulty buckets with time-varying logistic weights and optional reassignment
"Differentiable Top-k Classification Learning" (Petersen et al., 2022)	Top-$k$0 without curriculum schedule	Fixed distribution over multiple $k$1 values in a differentiable loss
"Thermodynamics of Reinforcement Learning Curricula" (Adamczyk et al., 12 Mar 2026)	Annealing theory without top-$k$2	Geometric schedules minimizing excess thermodynamic work
"What Makes a Good Curriculum? Disentangling the Effects of Data Ordering on LLM Mathematical Reasoning" (Jia et al., 21 Oct 2025)	Ranking/tier analysis without explicit top-$k$3 annealing	Offline ordering and tier-wise curricula under multiple difficulty metrics
"Feature-based tuning of simulated annealing applied to the curriculum-based course timetabling problem" (Bellio et al., 2014)	Disambiguating usage of “curriculum” and “annealing”	Simulated annealing for CB-CTT, with no explicit top-$k$4 move mechanism

The principal ambiguity is therefore structural. In the direct sequence-modeling case, “top-$k$5” refers to a truncated neighborhood of semantically similar tokens and “annealing” refers to the evolution of replacement frequency and sampling temperature (Neill et al., 2021). In later adjacent work, “top-$k$6” may instead refer to decoding support, rank-based classification objectives, or quantile buckets, while “annealing” may mean cyclic traversal, logistic reweighting, or thermodynamic schedule design (Lu et al., 2022, Petersen et al., 2022, Elgaar et al., 2023, Adamczyk et al., 12 Mar 2026).

2. Sequence prediction origin: exposure bias and curriculum replacement

The direct motivating problem is exposure bias in autoregressive sequence prediction. In the formulation used for NNRS, a generic sequence model is trained on paired sequences $k$7, with language modeling as the special case $k$8 and $k$9. The training objective is standard maximum likelihood,

$k$0

which conditions on true past outputs during training. At inference, however, the model conditions on its own past predictions rather than teacher-forced histories. The paper identifies this mismatch as exposure bias and emphasizes that the resulting error can compound along the sequence, “in the worst case leading to errors quadratic in $k$1” (Neill et al., 2021).

NNRS modifies the distribution of conditioning histories rather than the likelihood itself. At time step $k$2, the previous gold token $k$3 need not be fed unchanged as the next input. Instead, training may substitute either the model prediction $k$4, as in scheduled sampling, or a sampled nearest neighbor $k$5 of the gold token in a pretrained embedding space. The important conceptual interpolation is

$k$6

rather than the direct scheduled-sampling transition

$k$7

This supplies a semantically local perturbation regime between pure teacher forcing and full self-conditioning (Neill et al., 2021).

Under this interpretation, Curriculum Top-K Annealing denotes a controlled broadening of the training history distribution. The perturbation space is not the full vocabulary and not initially the model’s own predictive distribution. It is a truncated semantic neighborhood, which makes the induced corruption bounded and embedding-structured. A plausible implication is that the method is best viewed as a curriculum over history corruption severity: initially almost deterministic, then increasingly stochastic, but within a top-$k$8 neighborhood whose internal entropy is itself controlled.

3. Mechanism: top-$k$9 neighborhoods, replacement schedules, and temperature control

In NNRS, each vocabulary item $k$0 is associated with its top $k$1 nearest neighbors in a pretrained embedding space. Cosine similarity is the neighborhood criterion,

$k$2

Only the top-$k$3 most similar words are retained, stored in a matrix $k$4 whose row $k$5 defines the truncated neighborhood for $k$6. Sampling therefore occurs from a fixed support of size $k$7, not from the full vocabulary (Neill et al., 2021).

The probability of selecting neighbor $k$8 given source word $k$9 is a softmax over cosine similarities with temperature $k$0:

$k$1

Small $k$2 makes the distribution more peaked and approaches nearest-neighbor determinism; large $k$3 flattens the distribution and explores more of the top-$k$4 list. The paper also mentions an alternative weighted-mean neighbor representation,

$k$5

but the reported experiments focus on discrete neighbor sampling (Neill et al., 2021).

A crucial terminological point is that, in this paper, $k$6 itself is fixed. The annealed variables are the probability of performing NNRS replacement, denoted $k$7, and the temperature $k$8 governing how probability mass is distributed within the top-$k$9 set. The paper states that in its experiments $k$0, and that pretrained GoogleNews skip-gram embeddings are used to construct the neighbor graph (Neill et al., 2021).

The curriculum is epoch-indexed. Scheduled sampling uses rate $k$1, NNRS uses rate $k$2, and both are monotonically increased using linear, exponential, inverse-sigmoid or S-shaped, and static schedules. Token selection during training is stochastic per sample and per time step: random variables are drawn, compared against $k$3 and $k$4, and the input is chosen as the model prediction, a sampled neighbor, or the gold token. If both scheduled sampling and NNRS fire on the same step, the paper’s prose specifies random tie-breaking with equal probability. If neither fires, teacher forcing is retained (Neill et al., 2021).

A second annealing process adapts $k$5 using validation perplexity. The rule is described more clearly in prose than in the printed equation: if validation loss worsens or fails to improve, $k$6 is increased to encourage broader exploration among top-$k$7 neighbors; if validation loss improves, $k$8 is decreased so the model focuses more on the closest neighbors. The text states $k$9 to avoid making the distribution too degenerate or too uniform (Neill et al., 2021).

This gives the most faithful technical interpretation of Curriculum Top-K Annealing in the sequence-modeling literature: fixed top-$k$0 support, curriculum over replacement rate, and annealing of effective support within that fixed set through temperature.

4. Relation to scheduled sampling, implementation profile, and empirical behavior

NNRS is explicitly positioned relative to scheduled sampling. Both perturb previous-token inputs during training to address exposure bias, and both use curricula that change over training. The difference is that scheduled sampling replaces a gold token with the model’s own prediction, whereas NNRS replaces it with a semantically similar token chosen externally from a pretrained embedding space. The paper argues that this creates a smoother path from teacher forcing to self-conditioning and also claims that, because the replacement policy is not generated by the model itself, NNRS more easily preserves a “proper” scoring-rule interpretation than scheduled sampling as criticized by Huszár (Neill et al., 2021).

Operationally, the method is designed to be straightforward, online, and memory-efficient. The nearest-neighbor table is precomputed from pretrained embeddings; training-time overhead is then largely limited to indexed categorical sampling from stored top-$k$1 lists. The paper contrasts storing embeddings of size $k$2 with storing a transition matrix of size $k$3, emphasizing that $k$4 is the practically relevant regime. For larger vocabularies, the discussion notes that exact or approximate $k$5-NN search could use KD-trees, metric trees, or cover trees, although this was unnecessary in the reported experiments (Neill et al., 2021).

The reported benchmarks are Penn Treebank and WikiText-2, with perplexity as the main language-modeling metric and self-BLEU and WMD or self-WMD as additional generation analyses. Main perplexity results are reported for a 2-hidden-layer LSTM, with additional quality and diversity tables including LSTM, GRU, and Highway networks. Baselines include no sampling or standard MLE teacher forcing, TPRS, scheduled sampling, NNRS alone, and the combined SS-NNRS system (Neill et al., 2021).

The main empirical findings are consistent across the summary provided. NNRS alone can improve over teacher forcing, but SS-NNRS usually performs best. Successful settings generally use $k$6 and $k$7. Exponential schedules, and sometimes static low replacement rates, outperform linear or sigmoid schedules; sampling too aggressively too early hurts. The gains are larger on WikiText-2 than on PTB, which the authors attribute to the larger vocabulary, more rare words, and greater usefulness of semantically local replacements. The paper reports about 8 perplexity points improvement on WikiText-2 over no sampling and about 2.75 perplexity points improvement on PTB with the best approach, with best test perplexities around 120.76 on WikiText-2 and 69.06 on PTB for SS+NNRS configurations (Neill et al., 2021).

These results give the phrase “Curriculum Top-K Annealing” its strongest concrete meaning: a semantically local curriculum over top-$k$8 perturbations whose stochasticity is introduced gradually and works especially well in conjunction with scheduled sampling.

5. Adjacent formulations in curriculum, top-$k$9, and annealing research

Outside sequence prediction, related papers realize analogous components without reproducing the same mechanism. In curriculum data augmentation, PCC uses top-$k$0 and bottom-$k$1 decoding regimes to generate paraphrases, ranks them by textual similarity, partitions them into six curricula, and traverses them cyclically. Its bottom-$k$2 procedure does not sample from the least probable $k$3 words; instead, it excludes the top-$k$4 most probable words and samples from the remainder for the first $k$5 decoding steps, then falls back to top-$k$6 and top-$k$7. Difficulty is defined by paraphrase similarity rather than perturbation ratio, and the system passes through the curricula twice. This is strongly suggestive for curriculum-controlled truncation policies, but the paper does not define an explicit annealing schedule over the value of $k$8 itself (Lu et al., 2022).

In sample-ranking curricula, HuCurl partitions examples into $k$9 ordered difficulty groups using annotation entropy or average baseline loss, then applies group-specific logistic weights

$k$0

over normalized training time. Its non-monotonic extension arises not from a non-monotone pacing function but from dynamic reassignment of examples across bins according to current loss relative to group mean. This is a bucket-wise annealing framework rather than a hard top-$k$1 selector, but it provides a soft approximation to changing ranked prefixes and explicitly shows that top-performing curricula are often non-monotonic (Elgaar et al., 2023).

Objective-side top-$k$2 appears in differentiable classification learning, where a random variable $k$3 defines a fixed distribution over top-$k$4 targets and the model is optimized with

$k$5

This supplies a principled multi-$k$6 loss and shows that mixing multiple $k$7 values can outperform optimizing a single $k$8, but the paper does not propose a time-varying curriculum $k$9 (Petersen et al., 2022).

Annealing without top-$k$0 is formalized in reinforcement learning by the thermodynamic curriculum framework, where task parameters or entropy temperature define a path $k$1 on a task manifold and optimal curricula minimize excess thermodynamic work. The leading-order objective

$k$2

induces a friction metric and geodesic schedules. This directly motivates geometry-aware annealing, but the paper does not treat explicit top-$k$3 selection (Adamczyk et al., 12 Mar 2026).

Offline ranking analyses for LLM reasoning provide further evidence that curriculum behavior is metric-dependent. The 2025 mathematical-reasoning study compares forward and reverse curricula, Single Group Curriculum, and grouped forward and reverse curricula over difficulty metrics such as ACC, SLP, LG, SLE, TLE, and VACC. It reports that no curriculum strategy dominates universally and that, for several metrics, the medium tier rather than the extreme easy or hard tier can be most useful. This does not define top-$k$4 annealing, but it materially weakens the assumption that a monotone expansion from the easiest prefix is universally optimal (Jia et al., 21 Oct 2025).

Finally, the simulated-annealing paper on curriculum-based course timetabling is a terminological counterexample. Its “curriculum-based” refers to the CB-CTT problem formulation, and its annealing is classical simulated annealing over timetabling moves; the paper explicitly does not contain an explicit top-$k$5 move-selection mechanism (Bellio et al., 2014).

6. Limitations, misconceptions, and conceptual implications

Several misconceptions are directly corrected by the source material. The first is that NNRS anneals $k$6. It does not. In the direct sequence-prediction formulation, top-$k$7 candidate truncation is fixed; annealing happens over replacement probability and over the temperature $k$8 of the top-$k$9 sampling distribution (Neill et al., 2021). The label “Curriculum Top-K Annealing” is therefore faithful only if “annealing” is understood to mean annealing within a top-$k$00 support or annealing the probability of invoking that support.

The second misconception is that stronger perturbation is always beneficial. The NNRS analysis states that too much replacement too early hurts, that larger $k$01 increases diversity at the expense of generation quality, and that $k$02 corresponds to ML training. Exponential schedules help because they defer most stochasticity until later in training (Neill et al., 2021). PCC reaches a closely related conclusion from a different direction: harder examples generated by bottom-$k$03 decoding are useful, but they are embedded in a cyclic curriculum rather than introduced once in a purely monotone schedule (Lu et al., 2022).

A third misconception is that semantic locality guarantees contextual validity. NNRS depends on pretrained embedding quality, and a near neighbor in embedding space may still be syntactically or contextually inappropriate for the current history. The method assumes that semantic neighborhoods provide useful smoothing, which helps more on WikiText-2 than PTB but remains task- and data-dependent (Neill et al., 2021).

The broader curriculum literature in the provided sources also argues against a universal one-way annealing doctrine. HuCurl reports that top-performing discovered curricula are often non-monotonic, while the 2025 LLM reasoning study finds that the relative effectiveness of forward versus reverse ordering depends jointly on model capability and task complexity and that different difficulty tiers produce distinct gains depending on task demands (Elgaar et al., 2023, Jia et al., 21 Oct 2025). A plausible implication is that hard top-$k$04 expansion schedules should often be replaced by bucketed, cyclic, or non-monotonic exposure rules when difficulty estimates are noisy or when the most useful training signal lies in a medium-difficulty band rather than at the extremes.

Taken together, the literature supports a precise but limited meaning of Curriculum Top-K Annealing. In its most concrete form, it refers to fixed top-$k$05 truncation combined with an annealed policy over when and how to sample from that truncated set, as in NNRS for sequence prediction (Neill et al., 2021). In a broader, inferred sense, it denotes curriculum systems in which ranked or truncated candidate sets are exposed under time-varying control, possibly through bottom-$k$06 decoding, bucket-wise logistic weights, multi-$k$07 objectives, or thermodynamically informed schedules (Lu et al., 2022, Elgaar et al., 2023, Petersen et al., 2022, Adamczyk et al., 12 Mar 2026). The provided papers collectively support the design pattern; they do not support a single canonical definition.