IRT-Based Scheduling (PUDF)

• The paper introduces PUDF, a framework that dynamically aligns item difficulty with learner ability using IRT models.
• It employs deep reinforcement learning to amortize experimental design, achieving up to 50x speed improvements over classical methods.
• PUDF integrates classical IRT-based approaches with modern deep learning extensions to optimize adaptive testing and curriculum scheduling.

Item Response Theory-Based Scheduling, notably instantiated in the PUDF (Psychology-based Unified Dynamic Framework), leverages Item Response Theory (IRT) models for scheduling and selection in adaptive testing and curriculum learning. PUDF encompasses both classical IRT-driven approaches for experimental design and recent deep learning-based extensions that enable real-time, information-optimal scheduling in both psychometrics and machine learning (Keurulainen et al., 2023, Meng et al., 2024).

1. Item Response Theory Foundations in Scheduling

Item Response Theory provides a probabilistic model for the likelihood that a learner (human or machine) responds correctly to an item, as a function of that learner’s latent ability and the item’s parameters. The canonical IRT models include:

• 1PL (Rasch) Model: For item $i$ with difficulty $b_i$ and subject ability $\theta$:

$$p_i(\theta; b_i) = \frac{1}{1+\exp\left[-(\theta - b_i)\right]}$$

• 2PL/3PL Extensions: Incorporate discrimination $a_i$ and, for 3PL, guessing $c_i$:

$$p_i^{(2PL)}(\theta; a_i, b_i) = \frac{1}{1+\exp\left[-a_i(\theta - b_i)\right]}$$

$$p_i^{(3PL)}(\theta;a_i,b_i,c_i) = c_i + (1-c_i) \frac{1}{1+\exp\left[-a_i(\theta - b_i)\right]}$$

In scheduling, these IRT models furnish interpretable, theory-grounded notions of item difficulty and learner ability, enabling principled selection of data or test items (Keurulainen et al., 2023, Meng et al., 2024).

2. IRT-Based Data Difficulty Estimation

A core element in PUDF is the assignment of scalar difficulty scores to items. In educational or psychological testing, item difficulties are learned from subject responses. In curriculum learning for machine learning, PUDF introduces "IRT-AC," which computes difficulties using an artificial crowd (AC) of pretrained LLMs (PLMs):

• Assemble a response matrix $Z$ for $J$ reference models (AC members) over $b_i$0 items.
• Fit an IRT model (typically 1PL or via variational inference for scalability) to obtain global, model-independent item difficulties $b_i$1.
• The value $b_i$2 corresponds directly to the latent ability scale: a model with $b_i$3 has a 50% chance of success on item $b_i$4 (Meng et al., 2024).

This approach yields difficulty values consistent across data and models, supporting meaningful, interpretable scheduling for both humans and machines.

3. Amortised Experimental Design via Reinforcement Learning

PUDF extends classical Optimal Experimental Design (OED) by training a deep reinforcement learning (DRL) agent to amortise the adaptive selection process:

• POMDP Formulation: State encodes the sequence of past item-response pairs $b_i$5. Actions are tuple $b_i$6: choice of next item and current ability estimate.
• Reward: Immediate reward $b_i$7 incentivises accurate and rapid ability estimation.
• Policy Learning: A recurrent network, with fully connected and mean-pooling layers, takes the belief state and outputs both the item-selection policy and amortised ability estimate $b_i$8.
• Training: Synthetic data (randomised over plausible ability/difficulty priors) trains the agent with Proximal Policy Optimization (PPO), amortising both selection and inference (Keurulainen et al., 2023).

Once pre-trained, each next-item decision requires a single network forward-pass ($b_i$92–5 ms), representing a 10x–50x speedup over non-amortised, classical OED methods.

4. Dynamic Data Scheduling in Curriculum Learning

PUDF delivers an integrated dynamic scheduler for curriculum learning—Dynamic Data Selection via Model Ability Estimation (DDS-MAE):

• Offline Preprocessing: Compute item difficulties $\theta$0 via IRT-AC.
• Per-Epoch Scheduling:

1. Estimate current model ability $\theta$1 by MLE using a random batch of training examples.
2. Select curriculum data $\theta$2 for the epoch (items the model has at least 50% success probability on).
3. Fine-tune model only on $\theta$3 for that epoch.

• Adaptive Control: If $\theta$4 plateaus for two epochs, it is incremented to ensure continued curriculum progression (Meng et al., 2024).

This strategy maintains theoretical consistency between difficulty and ability while enabling per-epoch dynamic adaptation.

5. Empirical Performance and Component Analysis

PUDF's empirical evaluations demonstrate:

• In Adaptive Testing: On synthetic education datasets (200 students, 50 items), PUDF achieves a mean-MSE of $\theta$5, significantly outperforming both random ($\theta$6) and non-adaptive ($\theta$7) designs. Ablation confirms full adaptivity is critical for optimal estimation (Keurulainen et al., 2023).
• In Curriculum Learning: On GLUE tasks, PUDF surpasses all tested curriculum baselines in both accuracy and training time (e.g., DeBERTaV3: 90.95% acc, 97 min vs. RL-teacher: 90.68%, 233 min; no-curriculum: 90.03%, 179 min). Statistical testing confirms significance of both accuracy and efficiency gains (Meng et al., 2024).

A comprehensive component analysis reveals:

• IRT-AC is critical for accuracy.
• DDS-MAE is essential for efficiency.
• Only the combination of both components yields optimal convergence speed and performance.

Method	Accuracy (%)	Training Time (min)
No Curriculum	90.03	179
RL-teacher	90.68	233
PUDF (IRT-AC + DDS-MAE)	90.95	97

6. Extensions and Broader Applicability

PUDF generalises to several settings:

• Other IRT Models: Immediate extension to 2PL/3PL models, multidimensional IRT ($\theta$8), and cognitive diagnosis frameworks is feasible by modifying the simulation environment and network inputs.
• Alternate Instructional Design: Online tutoring, knowledge-tracing, or skill-path planning by treating curricular modules as “items” and scheduling adaptively for diagnostic value.
• Computational Scalability: Variational inference and sampling mitigate the cost of IRT model fitting for large datasets, and curriculum schedules can be recalibrated as models or data evolve (Keurulainen et al., 2023, Meng et al., 2024).

A plausible implication is increased interest in offline amortisation strategies for other adaptive, theory-driven scheduling domains.

7. Implementation Considerations and Best Practices

In practice:

• Offline IRT Fitting: Pre-compute $\theta$9 scores for the dataset using variational inference IRT (e.g., py-irt/Pyro) and an artificial crowd of reference models. This step is amortised across all future runs.
• Ability Estimation: At each schedule update, estimate ability using a subset ($$p_i(\theta; b_i) = \frac{1}{1+\exp\left[-(\theta - b_i)\right]}$$01k examples) to limit computational overhead.
• Dynamic Schedule Update: Strategies for schedule progression and plateau-escape are crucial to prevent stalling on “easy” data.
• Ablation Guidance: Use both theoretically grounded DM (IRT-AC) and adaptive TS (DDS-MAE/DRL) for optimal balance of accuracy and efficiency.
• Reporting: Average results and p-tests over multiple runs are necessary to ensure reproducibility and statistical significance (Meng et al., 2024).

Failure to align data difficulty and schedule adaptivity can result in suboptimal learning or redundant data usage.

---

PUDF, as realised in both deep reinforcement learning for adaptive experiment design (Keurulainen et al., 2023) and psychometric curriculum learning (Meng et al., 2024), establishes a principled, extensible framework for real-time, theory-consistent scheduling across education, psychology, and machine learning domains.