Improved Curriculum Learning (ICL)

• Improved Curriculum Learning is a design pattern that refines the traditional easy-to-hard approach by incorporating explicit difficulty measures, structured scheduling, and mechanisms to preserve early learning.
• ICL has been applied across domains such as prompt ordering in LLMs, medical imaging, and reinforcement learning, using tailored metrics like logit separability and expert kappa.
• The strategy improves training dynamics and test performance by coupling ordered data exposure with techniques like elastic coupling and progressive input adjustment, yielding measurable gains.

Improved Curriculum Learning (ICL) denotes a family of curriculum-learning refinements in which the standard easy-to-hard paradigm is augmented by more explicit difficulty measurers, more structured schedulers, or curriculum-aware objectives. In the recent literature, the term is not fully standardized: it appears in supervised training, multimodal reinforcement learning, medical image analysis, instruction tuning, and prompt construction for frozen LLMs. Taken together, these works suggest that ICL is best understood as a design pattern rather than a single algorithm: the central question is not merely whether examples are ordered, but how difficulty is defined, how exposure is staged, and whether information learned in early phases is preserved in later phases (Wang et al., 2020, Saglietti et al., 2021).

1. Scope and nomenclature

Curriculum learning is defined in the survey literature as learning from easier data to harder data, with the general abstraction of a Difficulty Measurer + Training Scheduler. The same survey broadens the notion further: a curriculum can be a sequence of reweightings of the target training distribution, or more generally a sequence of training criteria over steps, including data, tasks, loss functions, supervision schemes, or hypothesis-space changes (Wang et al., 2020). Within that broader frame, recent work labeled as ICL spans both training-time and inference-time settings.

A notable feature of current usage is acronym overload. In prompt-based LLM research, ICL can mean In-Context Curriculum Learning, where demonstrations are ordered from easy to hard without changing model parameters (Liu et al., 2024). In robotics, ICL can mean Incremental Curriculum Learning, where a long instruction is decomposed into progressively harder stages for deep reinforcement learning (Muttaqien et al., 2024). In emotion recognition in conversation, Improved Curriculum Learning denotes a difficulty measurer based on weighted emotional shifts that governs easy-to-hard training (Li et al., 21 Jul 2025). This suggests that the unifying concept is not the acronym itself but the recurrence of staged exposure structured by an explicit notion of difficulty.

Regime	Difficulty signal	Representative works
Frozen LLM prompting	Demonstration difficulty, PSL steps, logit separability, curvature	(Liu et al., 2024, Ma et al., 21 Feb 2025, Zixiao et al., 2024, Chung et al., 13 May 2026)
Supervised vision and medicine	Expert kappa, uncertainty, patch size	(Jiménez-Sánchez et al., 2020, Jiménez-Sánchez et al., 2020, Fischer et al., 27 Oct 2025)
Sequential and multimodal learning	Sub-instruction stage, weighted emotional shift	(Muttaqien et al., 2024, Li et al., 21 Jul 2025)
Instruction tuning and SSLMs	Subject stage, Bloom level, acquisition-theoretic categories	(Lee et al., 2023, Salhan et al., 2024)

The significance of this breadth is methodological. ICL is no longer confined to minibatch ordering in SGD; it now includes prompt ordering, curriculum-aware retrieval, curriculum masking in MLM, and stagewise decomposition of sequential tasks.

2. Core formal structure

The survey formulation describes a curriculum as a sequence

$$\mathcal{C}=\left\langle Q_1,\dots,Q_T \right\rangle,$$

where each criterion reweights the target distribution,

$Q_t(z)\propto W_t(z)P(z).$

In the original formulation, entropy increases over time, weights only increase, and the final curriculum recovers the target distribution (Wang et al., 2020). This provides the general mathematical backbone for most later ICL variants.

Several works make the two design variables explicit. In deep-network curriculum learning, the problem is decomposed into a scoring function $f$ that ranks examples by difficulty and a pacing function $g_\vartheta$ that determines how much of the sorted dataset is available at iteration $i$. Training then samples mini-batches from the easiest $g_\vartheta(i)$ examples after sorting (Hacohen et al., 2019). In the unified curriculum formulation for fracture classification, the same ranked data can drive three distinct mechanisms: reordering via a permutation $\pi$, subset growth via a pacing function $g$, and loss weighting via $\alpha$. The epoch-dependent loss introduces a scoring function $s$, curriculum probabilities $Q_t(z)\propto W_t(z)P(z).$0, the permutation $Q_t(z)\propto W_t(z)P(z).$1, the pacing function $Q_t(z)\propto W_t(z)P(z).$2, and weighting $Q_t(z)\propto W_t(z)P(z).$3, making reordering, subset selection, and weighted loss special cases of a common framework (Jiménez-Sánchez et al., 2020).

A major theoretical refinement appears in the teacher-student analysis of curriculum phases. That work shows that, in batch learning with standard losses, curriculum alone does not generally improve asymptotic generalization because later optimization can erase early-phase information. The proposed remedy is an explicit Gaussian prior or elastic coupling across phases,

$Q_t(z)\propto W_t(z)P(z).$4

which biases the later-phase solution toward the earlier one and preserves useful structure learned on easier data (Saglietti et al., 2021). This is a decisive shift from curriculum as ordering alone to curriculum as ordering plus memory.

These formulations collectively sharpen the meaning of “improved” in ICL. Improvement may arise from a better difficulty estimator, from a better schedule, or from an objective that prevents early curriculum stages from being forgotten.

3. Prompt-time and in-context curricula for LLMs

In LLM prompting, ICL moves curriculum from training to inference. The few-shot In-Context Curriculum Learning (ICCL) method keeps the LLM fixed and reorders only the demonstrations in the prompt, from easy to hard. Human experts determine difficulty in the main experiments using factors such as context length, complexity of the label format, deviation from common knowledge, and perceived solving difficulty. On SciCite, SciERC, and SciNLI, using 4–5 fixed demonstrations per task and two inference rounds, the paper reports that the average improvement across open-source models is 1.91% in $Q_t(z)\propto W_t(z)P(z).$5; for example, Qwen-72B-Chat improves from 49.48 to 52.23 average $Q_t(z)\propto W_t(z)P(z).$6, while GPT-4 decreases from 57.10 to 54.54. The same study also reports that base models are unstable under ICCL, with an average 7.16% decrease, and that model-generated difficulty ordering gives an average 0.29% decrease relative to random ordering (Liu et al., 2024). The immediate implication is that prompt curricula can help, but the benefit is model-dependent and appears to be linked to instruction tuning rather than pretraining alone.

A more structured variant is problem-solving logic guided curriculum ICL, which represents each example by a QDMR-derived operator sequence, or problem-solving logic (PSL). Demonstrations are selected when their PSL is a subsequence of the query PSL starting from the first operator, and difficulty is measured simply by the number of PSL steps, $Q_t(z)\propto W_t(z)P(z).$7. The resulting demonstrations are ordered from easy to hard. On SVAMP, AQuA, Gsm8k, CommonsenseQA, and StrategyQA, the method reports accuracies of 83.4%, 50.8%, 81.1%, 75.0%, and 71.6%, with an average of 72.37%, described as a 2.24% average accuracy improvement over prior ICL methods. An efficiency analysis reports that the prioritize-diversity strategy uses only 2.74 demonstrations on average and can reduce cost by 9% to 67% (Ma et al., 21 Feb 2025). Here, curriculum is not just ordering; it is logic-structured retrieval plus easy-to-hard sequencing.

A third line of work, MICL/LICL, bases curriculum construction on logit separability. Candidate label-related words are filtered by distribution separability and point-biserial testing, samples are scored by class-aligned logits or rank positions, and demonstrations are then ordered by score. The method also replaces weak class names with better label words and can insert multiple class-related words into each sample-label pair. Across seven classification datasets, the paper reports average gains of 3.11% on Llama2-7b and 11.58% on GPT2-xl over baselines in 1-shot settings, with ordering itself contributing average improvements of 0.39% and 5.86%, respectively (Zixiao et al., 2024). This broadens curriculum design beyond sample order to include label verbalization.

Long-context reasoning adds a further complication: more demonstrations do not automatically produce a better curriculum. In many-shot CoT-ICL, semantic similarity helps on non-reasoning tasks but fails on reasoning tasks because question-level similarity poorly predicts procedural compatibility. The paper proposes Curvilinear Demonstration Selection (CDS), which orders demonstrations to minimize total curvature in embedding space. The local curvature is defined by the turning angle

$Q_t(z)\propto W_t(z)P(z).$8

and the best ordering minimizes the sum of these angles. CDS yields up to a 5.42 percentage-point gain on geometry with 64 demonstrations, while smoother orderings correlate with higher accuracy (Chung et al., 13 May 2026). This result is important because it reframes the long context window as a curriculum rather than a retrieval buffer.

4. Domain-informed ICL in vision and medical imaging

Medical image classification supplied some of the clearest early demonstrations of ICL. In proximal femur fracture classification, curriculum order is derived from medical priors rather than generic heuristics. One paper compares uniform, frequency, AO-based, and kappa-based curricula using a ResNet-50 pretrained on ImageNet. On the 7-class AO task, mean / median weighted $Q_t(z)\propto W_t(z)P(z).$9-scores are 0.5662 / 0.5731 for random ordering, 0.5757 / 0.5923 for uniform ordering, 0.6757 / 0.6783 for AO ordering, and 0.6893 / 0.6900 for kappa ordering. The work reports about 15% improvement over baseline and notes that anti-curriculum performs worse (Jiménez-Sánchez et al., 2020). Difficulty here is medically grounded: AO encodes structural difficulty, while Cohen’s kappa encodes disagreement-based difficulty.

The later unified formulation generalizes this idea by combining class-level prior knowledge with sample-level uncertainty. For fracture classification, prior scores are taken from Cohen’s kappa agreement,

$f$0

while uncertainty is estimated by predictive entropy under Monte Carlo dropout,

$f$1

The same ranked samples can then be used for reordering, subsets, or weights. On the 7-class task, the best prior-knowledge reorder improves from 56.62 $f$2 to 68.93, and the best uncertainty subset reaches 65.51. On MNIST, the same framework is tested under limited data, class imbalance, and label noise, with the paper recommending reordering when domain knowledge exists, subsets for limited data or imbalance, and uncertainty-based weighting for noisy labels (Jiménez-Sánchez et al., 2020). This work is notable because it turns ICL into a modular scheduling framework rather than a single hand-crafted order.

In 3D medical image segmentation, ICL is instantiated as Progressive Growing of Patch Size (PGPS). Instead of ranking examples semantically, PGPS treats smaller patches as easier because they improve class balance, reduce background clutter, and reduce computational cost. Patch sizes increase linearly and stepwise, with one axis increased per stage, and the maximal patch size is used at inference. Two modes are reported: PGPS-Efficiency, which keeps batch size constant and reduces training time to only 44% of the constant patch size baseline on average, and PGPS-Performance, which reinvests freed memory into batch size and achieves a statistically significant relative mean performance gain of 1.28% in Dice Score while reducing training time to only 89%. The performance mode surpasses the constant patch size baseline on all 15 segmentation tasks and improves UNet, UNETR, and SwinUNETR (Fischer et al., 27 Oct 2025). This case is particularly significant because it shows that an effective curriculum need not be defined over labels or samples at all; it can be defined over input geometry.

5. Human-structured, multimodal, and sequential curricula

In deep reinforcement learning for mobile robots, Incremental Curriculum Learning is defined as stage-wise decomposition of a long instruction into easier sub-instructions. A canonical task, “Find the bread, take it, go to the fridge, and place the bread inside,” is split into four stages ranging from “Find the bread” to the full sequence. The agent operates in AI2-THOR with discrete actions, a first-person RGB camera, and text instructions, using a Multimodal Deep Q Network with a frozen ResNet18 image branch and an LSTM text branch. Training begins at Stage 1, proceeds over 12,000 episodes, and uses a goal-reaching reward between 5.0 and 20.0 with a -0.05 step penalty. The paper reports that without curriculum the agent struggles to learn, whereas ICL makes stage-wise transfer feasible, improves trainability, and supports adaptation from 3 objects to 9 objects (Muttaqien et al., 2024). Here the curriculum is a decomposition of task structure itself.

In emotion recognition in conversation, the proposed ICL addresses class imbalance by defining difficulty through weighted emotional shifts. Emotion similarity is computed on a 2D arousal-valence wheel, and each shift is weighted as

$f$3

Conversation difficulty is then

$f$4

After sorting conversations and partitioning them into $f$5 bins, training proceeds from the easiest bucket to harder ones. In the full LSDGNN+ICL system, weighted $f$6 reaches 70.24 on IEMOCAP and 64.07 on MELD; removing ICL drops performance to 68.92 and 63.78, respectively (Li et al., 21 Jul 2025). The method is important because difficulty is not frequency-based alone; it is tied to confusable emotional transitions.

Human-inspired curricula also appear in instruction tuning and language acquisition modeling. Instruction Tuning with Human Curriculum constructs a synthetic instruction-response dataset from 1.8K courses, 45 subjects, and 5.6K fine-grained concepts, using Bloom-style templates over Remember, Understand, and Apply. The preferred interleaving strategy revisits subjects cyclically while increasing cognitive difficulty. Compared with random shuffling, the paper reports gains of +4.76 on TruthfulQA, +2.98 on MMLU, +2.8 on OpenbookQA, and +1.28 on ARC-hard, all without extra computational cost (Lee et al., 2023). In cross-lingual small-scale language modeling, age-ordered CDS corpora and acquisition-theoretic curricula such as GROWING, INWARDS, and MMM are used to mask different token classes over developmental stages; the paper finds that fine-grained language-specific curricula, especially MMM (SEM), can outperform non-curriculum baselines, while universal curricula give mixed gains (Salhan et al., 2024). These studies extend ICL from difficulty scheduling to cognitively structured pedagogy.

6. Theory, limitations, and recurring misconceptions

A recurring misconception is that any easy-to-hard ordering is beneficial. The survey literature explicitly states that curriculum learning is not always beneficial, and that failure modes often arise from a poor difficulty measure, a poorly chosen scheduler, or a mismatch between human-defined and model-defined difficulty (Wang et al., 2020). Empirical work supports this caution. In deep-network training, random scoring behaves similarly to vanilla training, while self-paced scoring relative to the current hypothesis can hurt performance; transfer-based or bootstrapped difficulty estimation is substantially more reliable (Hacohen et al., 2019). In prompt-time curricula, human ordering outperforms model self-sorting, and GPT-4 degrades under the reported ICCL setup (Liu et al., 2024).

A second misconception is that curriculum alone changes final performance in all regimes. The analytical teacher-student theory draws a sharper distinction: curriculum can speed up learning dynamics in an online setting, especially when a sparse signal is hidden among irrelevant features, but does not provide generalization benefit in ordinary batch learning to convergence with standard losses. Large gains appear only when curriculum phases are coupled by a Gaussian prior or elastic penalty that preserves early learned structure (Saglietti et al., 2021). This point is consistent with the practical observation that many successful ICL systems add more than ordering: they add uncertainty-aware weighting, logic-aware retrieval, patch-size schedules, or regularization across phases.

A third misconception is that similarity is an adequate stand-in for curriculum. In reasoning-oriented many-shot CoT-ICL, semantic similarity-based retrieval fails because it does not capture procedural compatibility; order variance grows with more demonstrations, and smoother conceptual progressions outperform arbitrary permutations (Chung et al., 13 May 2026). In logic-guided curriculum ICL, a small set of well-chosen demonstrations with diverse difficulty can outperform a larger set of weakly relevant examples, further indicating that curriculum quality is not reducible to example count (Ma et al., 21 Feb 2025).

Taken together, these results suggest that the most durable form of ICL is not a universal easy-to-hard rule but a domain-specific coupling of three ingredients: an operational definition of difficulty, a scheduler matched to model dynamics, and a mechanism that retains what earlier phases teach. That interpretation aligns the otherwise diverse literature on prompt ordering, uncertainty-guided sampling, task decomposition, patch-size progression, and pedagogically structured instruction tuning.