Adaptive Sampling for Curriculum (AdaC)

• AdaC is a meta-framework that adaptively aligns training progression with evolving model competence through dynamic difficulty estimation and competence tracking.
• It utilizes techniques like multi-armed bandits, bilevel meta-optimization, and online learning strategies to optimize task selection and mitigate gradient starvation or forgetting.
• AdaC has demonstrated improvements in sample efficiency and performance robustness across applications in language models, robotics, and medical imaging.

Adaptive Sampling for Curriculum (AdaC) is a meta-framework that algorithmically aligns the progression of training samples or tasks with an agent’s evolving competence during learning. AdaC encompasses a spectrum of methods, including multi-armed bandit strategies, competence-progress tracking, bilevel meta-optimization, regret-minimizing online learning, and difficulty estimation pipelines. These mechanisms are unified by their core function: to adaptively control the distribution over training items (samples, goals, tasks, losses, or teachers) so as to maximize total learning progress, sample efficiency, or performance robustness.

1. Fundamental Principles and Motivation

AdaC addresses inherent limitations of static or hand-crafted curricula in machine learning systems. Manually ordered syllabi often mismatch evolving model capabilities, leading to gradient starvation, inefficient computation on already mastered examples, catastrophic forgetting, or premature overfitting to difficult instances. AdaC methodologies are formulated to:

• Estimate and quantify instance or task “difficulty” using model-intrinsic signals (win rates, validation loss, prediction gain, empirical solve rates, uncertainty, etc.).
• Track the learner’s competence or progress, typically on sliding windows or buffers.
• Adaptively sample or prioritize data to synchronize presented difficulty with current mastery.
• Provide rigorous theoretical guarantees (regret minimization, convergence) when cast in bandit or bilevel frameworks.

Early formalizations appeared in neural sequence modeling via multi-armed bandit-driven syllabi (Graves et al., 2017), and have since evolved into sample- and task-level adaptivity across language, vision, and robotics domains.

2. Key Methodologies and Algorithms

AdaC algorithms commonly instantiate one or more of the following computational motifs:

2.1. Difficulty Estimation

Difficulty may be precomputed (e.g., using held-out performance, solve rates, or annotation) or adaptively estimated:

• Coarse-to-fine estimation: A multi-stage process where an initial coarse binning (e.g., according to model correctness frequencies) is refined by dense stochastic evaluation to assign precise, continuous difficulty scores to each item (Li et al., 12 Nov 2025).
• Empirical solve rates: Monte Carlo or moving-average accuracy, such as the “win statistic” in AdaSTaR, provides an on-the-fly difficulty metric for each data point (Koh et al., 22 May 2025).
• Proxy metrics: In RL, goal proximity, reward thresholds, or required precision (e.g., $\epsilon$-accuracy in goal-reaching) serve as intrinsic difficulty scalars (Fournier et al., 2018, Niu et al., 2023).

2.2. Adaptive Scheduling and Competence Tracking

Curriculum scheduling operates by dynamically partitioning the data sorted by estimated difficulty into buckets or skill levels, and expanding the active training distribution in response to measured competence:

• Bucketed expansion: Sorted data are split into $K$ buckets; as the average competence score surpasses thresholds, new (harder) buckets are merged in, while prior buckets remain accessible to mitigate forgetting (Li et al., 12 Nov 2025).
• Competence progress weighting: Tasks or difficulty levels are sampled with probability proportional to recent competence progress raised to a power $\beta$, focusing training at levels exhibiting maximal growth (Fournier et al., 2018).
• Bilevel pacing: Only a fraction (e.g., $\alpha^2$) of recently sampled examples have their difficulty statistics updated per iteration, thus early training is biased toward easier items, while hard instance focus increases as global training accuracy $\alpha$ rises (Koh et al., 22 May 2025).

2.3. Online Bandit and Meta-Optimization Strategies

AdaC often employs probabilistic schedulers grounded in online learning or optimization theory:

• Exp3.S bandit policies: Tasks are arms, with selection probabilities updated via exponential weighting of observed progress (e.g., accuracy, complexity gain), importance-corrected by the bandit policy (Graves et al., 2017).
• Validation-driven bandit subset selection: Each arm corresponds to a submodular batch selection rule, and the best-performing arm (in terms of validation loss drop) is identified and exploited through a mixture of greedy exploration and exploitation, yielding provable no-regret convergence to the best curriculum in hindsight (Chanda et al., 28 Nov 2025).
• Online Stochastic Mirror Descent (OSMD): The sampling policy is parameterized and updated using entropy-regularized, gradient-based minimization of surrogate regret objectives, ensuring both adaptivity and theoretical performance bounds (Gu et al., 24 Feb 2026).

2.4. Explicit Data Revisitation and Forgetting Mitigation

By maintaining cumulative unions of curriculum buckets, AdaC prevents catastrophic forgetting. Previous (easier) samples persist in the training set even as harder examples are introduced and prioritized (Li et al., 12 Nov 2025).

3. Representative Domains and Concrete Implementations

AdaC has been deployed across a broad spectrum of domains:

3.1. LLM Reinforcement Learning

Curriculum RL in LLMs incorporates AdaC via:

• Dynamic difficulty estimation coupled with incremental bucket scheduling and competence tracking (Li et al., 12 Nov 2025).
• Bandit-guided online problem selection with explicit regret reduction objectives, providing dynamic curation during RL-finetuning (Gu et al., 24 Feb 2026).

3.2. Self-Improving Reasoning Models

The AdaSTaR framework adapts sample priorities in the Self-Taught Reasoner (STaR) loop by adjusting update frequencies of example statistics based on current model accuracy, blending easy/hard sampling in proportion to global skill (Koh et al., 22 May 2025).

3.3. Reinforcement Learning in Robotics

• Goal-conditioned RL: Curriculum over goals is realized by continuous interpolation between initial “easy” and target “hard” goal distributions, often via Wasserstein barycenter geodesics (e.g., GOATS) (Niu et al., 2023).
• Adaptive contrastive curriculum: Buffer trajectories are prioritized via a dynamic balance between diversity and quality signals, with norm-constrained contrastive learning used to refine the actionable curriculum (Wang et al., 2 Mar 2026).
• Grounded curricula and performance monitoring: Task-space latent representations (e.g., VAE-encoded maps) are combined with exponential moving-average performance estimates and alternating sampling from real and synthetic (teacher-generated) tasks (Wang et al., 5 Aug 2025).

3.4. Multi-Task and Budget-Constrained Learning

Meta-learning of task mixtures under explicit resource constraints proceeds via bilevel optimization over task-sampling logits, using validation-based objectives to dynamically allocate training effort (Kadasi et al., 4 Dec 2025).

3.5. Medical Imaging and Data Imbalance

Progressive transition from “easy” (lesioncentered) to “hard” (background, hard negatives) examples is orchestrated by mixing-schedule coefficients and dynamically updated hard-negative mining weights (Jesson et al., 2018).

A summary of representative approaches, their key algorithmic features, and results is presented below.

Approach	Adaptivity Mechanism	Empirical Setting/Remarks
AdaCuRL (Li et al., 12 Nov 2025)	Coarse-to-fine difficulty, competence-driven bucket expansion, sparse KL regularization	RL-finetuning for LLM reasoning; prevents forgetting, avoids policy degradation
AdaSTaR (Koh et al., 22 May 2025)	Dynamic priority update tied to model strength (accuracy), quadratic pacing	LLM reasoning (STaR); reduces false positives, boosts compute efficiency
GOATS (Niu et al., 2023)	Wasserstein interpolation over factorized goal spaces	Robotic water scooping; lowest sim-to-real error, sample-complexity reduction
ACTOR-CURATOR (Gu et al., 24 Feb 2026)	Bandit-guided mirror descent on utility-improvement, policy-improvement reward	LLM RL post-training; 28.6–30.5% test gains, 80% speedup vs baselines
CASED (Jesson et al., 2018)	Curriculum mixing, hard-negative mining	Lung CT nodule detection; state-of-the-art sensitivity on LUNA16

4. Theoretical Guarantees and Analysis

AdaC methods often provide non-asymptotic performance guarantees grounded in online learning theory:

• No-regret bounds: Bandit-driven curriculum algorithms (e.g., ONLINESUBMOD) yield cumulative regret $R_T=O(\log T)$ relative to the best static or nonstationary curriculum, even under partial or noisy feedback (Chanda et al., 28 Nov 2025, Gu et al., 24 Feb 2026).
• Mirror descent properties: AdaC as realized in actor-curator OSMD optimizes an entropy-regularized regret surrogate, converging toward the optimal (possibly nonstationary) curriculum within $O(T^{2/3})$ cumulative regret, modulated by nonstationarity of the reward landscape (Gu et al., 24 Feb 2026).
• Competence progress prioritization: Stochastic prioritization $p_i\propto (\mathrm{CP}_i)^\beta$ provably concentrates training on regions of highest learning dynamics, as empirically verified in continuous-control RL (Fournier et al., 2018).
• Bilevel optimization stability: Meta-learning approaches such as ADAPT enforce mixture diversity through entropy maximization, empirically avoiding collapse to degenerate task selection (Kadasi et al., 4 Dec 2025).

5. Empirical Impact and Benchmarks

Some prominent task-specific findings:

• LLM RL: AdaCuRL achieves significant performance gains and sample efficiency improvements on diverse mathematical and logical reasoning tasks. For example, curriculum-based RL in LLMs prevents gradient starvation and catastrophic forgetting, while conditional KL regularization avoids policy collapse on invalid samples (Li et al., 12 Nov 2025).
• Post-training LLMs: ACTOR-CURATOR surpasses uniform and curriculum learning baselines, with up to 80% speedup to equivalent accuracy and 28.6–30.5% relative metric gains (Gu et al., 24 Feb 2026).
• Medical imaging: CASED obtains 88.35% average sensitivity in lung nodule detection, outperforming all non-curriculum and two-stage methods on LUNA16 data (Jesson et al., 2018).
• Robotics: GOATS reduces scooping error to 5.46–8.71% in multi-goal water-tank tasks and maintains a <2–3% real-to-sim gap; ACDC reduces time-to-threshold and cumulative regret by 40–85% and 50–70% respectively in high-dimensional manipulation (Niu et al., 2023, Wang et al., 2 Mar 2026).
• Multi-task learning: ADAPT reallocates token budgets to harder tasks with substantial savings (2.6–23× fewer tokens to reach lowest loss) and no loss in downstream macro-average accuracy (Kadasi et al., 4 Dec 2025).

6. Limitations and Open Questions

While AdaC methods provide broad efficacy, certain limitations are recurrently observed:

• Reward model and feedback reliability: Bandit-based AdaC requires sufficiently informative intrinsic or extrinsic rewards; noisy or ill-posed validation metrics can misalign curriculum progression (Gu et al., 24 Feb 2026, Chanda et al., 28 Nov 2025).
• Early training stability: Warm-starts or capped exploration phases are necessary to stabilize curriculum selection in early stages where gradients or competence estimates are unreliable (Graves et al., 2017, Li et al., 12 Nov 2025).
• Design of difficulty metrics: Reliance on suitable, task-specific difficulty indicators remains a challenge, particularly in high-dimensional or non-factorizable domains.
• Computational overhead: Online estimation of per-instance difficulty, competence tracking, and submodular maximization introduce nontrivial overheads, which can be offset by convergent gains only for sufficiently large tasks (Chanda et al., 28 Nov 2025, Gu et al., 24 Feb 2026).
• Generalization across domains: While AdaC methods have been shown to be robust across a range of settings, their transferability to settings lacking clear difficulty signals or with streaming, non-i.i.d. data is an active area of study.

7. Broader Implications and Future Directions

AdaC principles generalize beyond classical curriculum learning, providing a spectrum of algorithmic tools for adaptive data selection, online prioritization, resource scheduling, and continual learning. Future research is oriented toward:

• Formal unification of bandit, bilevel, and RL-based AdaC frameworks under meta-learning theory.
• Instance-level adaptive curricula within massive, streaming, or continual datasets, for both pretraining and finetuning paradigms.
• Automated metric selection for difficulty and competence, possibly leveraging unsupervised or self-supervised representation learning.
• Cross-domain AdaC: e.g., transfer of curricula or task-weighting across language, vision, and control tasks, incorporating context-rich problem embeddings and domain-aware reward shaping.

AdaC remains a critical ingredient for unlocking efficient, robust, and scalable learning in systems where model capabilities, task difficulty, and data complexity evolve non-trivially throughout training (Li et al., 12 Nov 2025, Koh et al., 22 May 2025, Niu et al., 2023, Gu et al., 24 Feb 2026, Chanda et al., 28 Nov 2025, Kadasi et al., 4 Dec 2025, Fournier et al., 2018, Jesson et al., 2018, Wang et al., 2 Mar 2026).