Test-Time Scaling Performance Model (TTSPM)

• TTSPM is a framework that applies curriculum-based interaction scaling at test time to progressively adjust difficulty and evaluation complexity.
• It integrates dynamic scheduling and adaptive sampling methods to maximize sample efficiency and improve robustness during inference.
• Empirical applications in RL, LLM fine-tuning, and robotics show that TTSPM can accelerate convergence and enhance task mastery.

Curriculum-Based Interaction Scaling is a methodological paradigm for structuring and growing the complexity, diversity, or volume of agent–environment interactions in machine learning systems. The canonical form is a curriculum: a staged or graded decomposition of the task, data, or environment space along dimensions that control learning difficulty. The objective is to maximize sample efficiency, accelerate optimization, and achieve higher asymptotic performance by aligning the agent’s exposure to progressively harder or more informative challenges, while often managing auxiliary trade-offs such as safety, generalization, or interaction costs.

1. Core Principles and Definitions

Curriculum-based interaction scaling refers to any formalism in which the training process is organized as a progression through an explicit, potentially adaptive sequence of tasks, data, or environment configurations, each designed to scale up pertinent aspects of interaction:

• Interaction Units: Atoms of interaction may be task instances (e.g., proof statements (Polu et al., 2022)), dialogue episodes (Li et al., 2023), RL environments (Peng et al., 2024), control-parameter subspaces (Murali et al., 2017), or agent populations (Long et al., 2020).
• Curriculum Scheduling: The system presents these units in an ordered manner—static (predefined sequence) or dynamic (adaptive based on agent performance or learning progress)—with increasing complexity.
• Scaling Axes: Interaction scaling can be realized along various axes: data difficulty, environment complexity (number of agents, obstacles, task constraints), action or observation space dimensionality, allowed interaction horizon, or data quality/utility (Li et al., 1 Apr 2026).
• Advancement Rule: Advancement through the curriculum may be controlled by fixed schedules, empirical metrics (learning progress, solve rate, validation loss), or reward-driven adaptive mechanisms.
• Sample Efficiency: The approach is justified by its ability to focus interaction on tasks that are neither trivial nor unlearnable, thereby maximizing information gain per unit cost.

2. Approaches and Formal Algorithms

The instantiation of curriculum-based interaction scaling varies substantially by domain but falls into several formal patterns:

a. Expert Iteration and Statement Curricula (Formal Mathematics)

A model $\theta$ is interleaved through proof search over pooled statements and model re-training, where the curriculum $\mathrm{St} = U_1 \cup \cdots \cup U_M$ partitions problems by intrinsic difficulty (e.g., synthetic depth, olympiad statements). Attempt counts per subset $\alpha_m$ allocate search budget, with scaling controlled via

$a(s) = \alpha_m \quad \forall s \in U_m$

and possible adaptive updates

$$a_{k+1}(s) = F(\textrm{pass-rate}_{\theta_k}(s), v_{\theta_k}(s))$$

(Polu et al., 2022).

b. Data-Centric and Difficulty-Based Sorting (LLMs)

Curricula are realized by sorting data by length, attention-variance, or loss-derived metrics. For a prompt $p$:

• Prompt length: $d_{\mathrm{length}}(p) = |T(p)|$
• Attention-variance: $$d_{\mathrm{att}}(p) = \frac{1}{L}\sum_{l=1}^L \mathrm{Var}_l(p)$$
• Loss-based: $$d_{\mathrm{loss}}(p) = -\sum_{i=1}^{|y|} \log p_\theta(y_i|x, y_{<i})$$

An epoch-wise schedule starts with random order, then sorts by ascending difficulty, presenting the easiest cases before harder ones in subsequent epochs (Kim et al., 2024).

c. Multi-Armed Bandit Curriculum Scheduling (RL and Robotics)

Interaction complexity is parameterized (e.g., number of agents), and a non-stationary multi-armed bandit assigns sampling probabilities to curriculum arms:

$$p_i(t) = (1-\eta) \frac{e^{w_i(t)}}{\sum_j e^{w_j(t)}} + \frac{\eta}{N+1}$$

with weight updates $\mathrm{St} = U_1 \cup \cdots \cup U_M$0 after each episode, where $\mathrm{St} = U_1 \cup \cdots \cup U_M$1 is the importance-weighted, normalized return (Peng et al., 2024).

d. Learning Progress-Based Automatic Task Sampling

For task set $\mathrm{St} = U_1 \cup \cdots \cup U_M$2, per-task learning progress is estimated as

$\mathrm{St} = U_1 \cup \cdots \cup U_M$3

and the next sampling distribution is a Boltzmann softmax over $\mathrm{St} = U_1 \cup \cdots \cup U_M$4,

$\mathrm{St} = U_1 \cup \cdots \cup U_M$5

This focuses sampling on tasks with maximal current return gains (Li et al., 24 Jan 2026).

e. Stage-Wise and Evolutionary Population Growth (Multi-Agent RL)

Populations are doubled ($\mathrm{St} = U_1 \cup \cdots \cup U_M$6), and ensembles of parameter sets are evolved by mix-and-match, mutation (fine-tuning), and selection based on adaptability at each new scale (Long et al., 2020).

f. Information-Based Data Valuation

Interaction curricula are built by scoring data according to gradient-based metrics (e.g., TracIn),

$\mathrm{St} = U_1 \cup \cdots \cup U_M$7

and then using these for importance weighting in the training objective. The schedule may involve a three-phase weighting ramp (Li et al., 1 Apr 2026).

3. Domains and Empirical Results

Mathematics and Automated Theorem Proving

Expert-iteration frameworks for formal mathematics achieve log-linear scaling of pass@1 score with iteration number:

$\mathrm{St} = U_1 \cup \cdots \cup U_M$8

and demonstrate substantially steeper compute-performance slopes than pure proof search, validating the efficacy of curriculum-induced data self-generation. State-of-the-art results are achieved on the miniF2F benchmark (e.g., pass@1: 29.6%, pass@8: 34.5%) (Polu et al., 2022).

Reinforcement Learning and Robotics

• Multi-agent curricula (EPC) yield a normalized test score of 1.00 at large scales (24 sheep/16 wolves), far surpassing standard MARL baselines (Long et al., 2020).
• Adaptive curriculum RL in rough-terrain locomotion achieves a $\mathrm{St} = U_1 \cup \cdots \cup U_M$92–4$\alpha_m$0 speedup in convergence and higher task mastery rates in ANYmal quadruped experiments (Li et al., 24 Jan 2026).
• Reward-driven curricula for self-driving robustly increase test success rates, notably from 77.5% to 100% (no SVs) and 64.5% to 75.5% (6 SVs) (Peng et al., 2024).

LLM Instruction and Data Ordering

• Curriculum data ordering, particularly by attention-variance, yields modest but consistent improvement (e.g., 67.5% avg. accuracy for Gemma-7B on Orca-math, outperforming random schedules by $\alpha_m$10.6–0.7 points) (Kim et al., 2024).
• Small LMs benefit most from sequential conversational→QA curricula for robust fine-tuning gains, while merged curricula better preserve zero-shot generalization (Capone et al., 29 Oct 2025).

Synthetic Data and Scaling Laws

Layered synthetic curricula in recommender LLMs—item definition, collaborative filtering, unbiased user histories—enable the first robust power-law scaling laws (e.g., perplexity in UIH: $\alpha_m$2). SASRec recall@100 shows a $\alpha_m$3 improvement over real logs, establishing pedagogically structured synthetic data as necessary for predictable scaling in this domain (Zhang et al., 7 Feb 2026).

Game-Theoretic and Motion Planning

Gradient-based data valuation induces curricula that reduce planning errors (planADE: 1.704 vs 1.822 with metadata heuristics; $\alpha_m$4), with low variance and greater sample efficiency (Li et al., 1 Apr 2026).

Education and Human Interaction

Curriculum-driven chatbots (Edubot) construct dialogue sessions with graded topic complexity and CEFR-level control, outperforming general LLMs in metrics of guidance, topic alignment, and conversational realism (Li et al., 2023). In classroom settings, network analysis reveals that curricula with integrated group work and whole-class synthesis scale both the breadth and depth of peer interaction networks, as measured by average degree and transitivity (Commeford et al., 2020).

4. Scheduling, Selection, and Advancement Methods

The primary schedules for curriculum progression include:

• Static schedules: Fixed epochs per stage, sorted data, or staged addition/removal of features or agents. Employed in legal LLMs (document length/Qtype) (Upadhyay et al., 26 Apr 2025) and classical staged curricula (Long et al., 2020, Capone et al., 29 Oct 2025).
• Performance-based advancements: Adaptive rules based on agent pass-rate, learning progress, or value-head confidence.
• Multi-armed bandit sampling: Automated importance weighting of curriculum arms, driven by normalized and importance-scaled episodic returns (Peng et al., 2024).
• Learning progress optimization: Boltzmann softmax over per-task learning gradients (Li et al., 24 Jan 2026).
• Attention/variance-based data ordering: Sorting examples for LLM fine-tuning (Kim et al., 2024).
• Gradient-similarity weighting: Directly weighting samples by their expected contribution to loss reduction (Li et al., 1 Apr 2026).
• Information-richness optimization: In robot fostering, interaction scaling is tied to optimizing a composite metric $\alpha_m$5 over long trajectories, constrained by governance and privacy standards (Pablo-Marti et al., 28 Sep 2025).

5. Theory, Transfer, and Generalization

• Emergent curriculum climbing: In agent-environment settings with suitable diversity, models can self-discover and ascend intrinsic hierarchies of problem difficulty, even without explicit difficulty labels or ground-truth answers (Polu et al., 2022).
• Sample efficiency and robustness: Curriculum-based interaction scaling demonstrably improves sample efficiency across RL, robotics, and LLM domains, sometimes yielding order-of-magnitude improvements in convergence or final task performance (Li et al., 24 Jan 2026, Zhang et al., 7 Feb 2026).
• Generalizability: The underlying abstraction—ordering interaction along meaningful axes of complexity—transfers across disciplines: language modeling, automated theorem proving, robotics, education, and more (Capone et al., 29 Oct 2025).

6. Limitations and Open Challenges

• Most current methods fix curriculum decomposition and stage boundaries by hand or via heuristics; fully automated, performance-sensitive scheduling is an open research direction (Polu et al., 2022, Li et al., 24 Jan 2026).
• Some regimes (notably, small LMs and zero-shot evaluation) expose trade-offs: tightly focused curricula can induce catastrophic forgetting or reduced generalization (Capone et al., 29 Oct 2025).
• In certain high-dimensional settings, obtaining the “correct” axis of curriculum decomposition is nontrivial; data-centric or gradient-based metrics offer improvements but require costly computation (Li et al., 1 Apr 2026).
• Governance and privacy constraints pose unique demands when scaling interaction in real-world human-facing systems, necessitating integrated data-pipeline, annotation, and audit procedures (Pablo-Marti et al., 28 Sep 2025).
• Theoretical guarantees for universal mastery under arbitrary interaction scaling regimes remain to be established, especially when tasks are heterogeneous or reward signals are sparse and noisy (Li et al., 24 Jan 2026).

7. Outlook and Practical Guidelines

• Whenever possible, design curricula to maximize agent learning progress at the “edge of competence”—present tasks of intermediate difficulty or high expected information gain.
• For large-scale data-driven models (LLMs, RL agents), preference should be given to curriculum orderings derived from model-centric metrics (attention, loss, gradient similarity) over static feature heuristics—yielding more stable and sample-efficient training (Kim et al., 2024, Li et al., 1 Apr 2026).
• In resource-limited settings, employ sequential or phased curricula to extract maximal fine-tuning gain, but monitor generalization to prevent over-specialization (Capone et al., 29 Oct 2025).
• For domains with privacy, safety, or regulatory requirements, curriculum-based scaling must be harmonized with privacy-preserving interaction, staged evaluation gates, and post-hoc auditability (Pablo-Marti et al., 28 Sep 2025).
• The design of synthetic, layered curricula (especially with compositional logic and bias-free sequences) is emerging as a cornerstone for scaling in recommendation and other domains where real interactions are biased or sparse (Zhang et al., 7 Feb 2026).

Curriculum-based interaction scaling thus constitutes a foundational strategy for orchestrating the growth of agent capabilities, balancing exploration and exploitation, maximizing information throughput, and negotiating deployment constraints in contemporary machine learning systems.