Parameterized Student Proficiency Simulation

• PS² is a simulation paradigm that uses LLMs to generate synthetic student responses based on discrete, continuous, and multidimensional proficiency parameters.
• It employs varied simulation protocols, including role-play prompting and model-level control, to mimic realistic student error patterns and evolving learning trajectories.
• Integration with psychometric models like IRT enables scalable item analysis and calibration, supporting rapid assessment and refined test design in education.

Parameterized Student Proficiency Simulation (PS$^2$) is a methodological paradigm that leverages LLMs to synthetically generate student response data conditioned on explicit, fine-grained proficiency parameters. PS$^2$ provides a flexible and scalable alternative to traditional human pilot studies for educational item analysis, enabling automated estimation of item difficulty, discrimination, and error distributions. The core approach involves simulating a population of students with diverse ability profiles, prompting LLMs to generate responses as if from each simulated persona, and calibrating the aggregate results via psychometric models such as Item Response Theory (IRT). Distinct PS$^2$ frameworks have been advanced, including prompt-level role play, model-level interpolation, knowledge graph–based cognitive prototyping, and direct parameter control in the LLM's forward process.

1. Formalization and Proficiency Parameterization

PS$^2$ methods formalize the simulation of student responses as a function $$\pi(\text{response}~|~\text{item},~\text{proficiency})$$, where the "proficiency" parameter can take several forms:

• Discrete Proficiency Levels: E.g., NAEP skill bands ("Below Basic," "Basic," "Proficient," "Advanced") (Acquaye et al., 15 Jan 2026); "weak/average/strong" personas as prompt variables (Li et al., 21 Dec 2025).
• Continuous Proficiency Control: Real-valued interpolation between LLMs of differing ability, yielding a graded spectrum of simulated students parameterized by $p \in [0,1]$ (Liu et al., 31 Jan 2026).
• Multidimensional Profiles: Vectors $\boldsymbol{\beta}$ of mastery across $K$ knowledge components (KCs), with each component $k_i$ or $C_{i,k}$ representing grasp or misconception of atomic concepts (Lu et al., 2024, Wu et al., 26 May 2025).
• Dynamic State Models: Differential equations specify evolving knowledge $Z_j(t)$, operability $r(t)$, and cumulative effort $P(t)$ over time, supporting simulation of learning trajectories (Mayer, 2013).

Table 1 summarizes three canonical parameterizations:

Framework	Proficiency Param	Granularity
Prompt persona	Discrete $p$	Ability levels
Logit blend	Continuous $p$	Finer-grained score
Knowledge graph	Vector $\theta$	KC-specific

Explicit definition and operationalization of these parameters is fundamental for mapping simulated response patterns to real-world student heterogeneity and for downstream psychometric analysis.

2. Simulation Protocols and Model Architectures

PS$^2$ instantiations configure synthetic classrooms via one or more of the following approaches:

Role-Play Prompting

Instruction-tuned LLMs are prompted to "be" students of specified grade and ability. Key factors include:

• Prompt templates: E.g., "You are a {skill level} student in the {grade}th grade..." (Acquaye et al., 15 Jan 2026), "Suppose you are a {weak/average/strong} student..." (Li et al., 21 Dec 2025).
• Identity cues: Assigning names, IDs, or demographic attributes to improve realism and alignment (e.g., stratified by race/gender) (Acquaye et al., 15 Jan 2026).
• Knowledge component–based: Profiles specify mastery, confusion, and unknowns per KC, with inlined example responses to guide LLM simulation (Lu et al., 2024).
• Teacher-as-predictor: LLM predicts likely student errors given a cognitive profile, enhancing simulation authenticity (Lu et al., 2024).
• Batch protocols: Simulate $N$ students per item, aggregate for percent-correct or item statistics (Acquaye et al., 15 Jan 2026).

Model-Level Proficiency Control

In logit interpolation frameworks, proficiency is a parameter of the model's forward pass:

• Upper-bound and lower-bound LLMs: $M_u$ (strong) and $M_\ell$ (error-informed weak). Output logits are blended as $z(p) = (1 − p) z^\ell + p z^u$ (Liu et al., 31 Jan 2026).
• Hybrid ratio $p$: Directly scales the strength of response generation, mapping to academic performance through calibration.

Cognitive Prototype Construction

Student models adhere to knowledge graphs encoding explicit mastery/confusion per concept (Wu et al., 26 May 2025):

• Prototype vector $\theta_i$ synthesized from past behaviors and concept mastery.
• Behavior and solution generation: Map $\theta_i$ to new tasks by similarity mapping, reference retrieval, and self-refinement loops.

Dynamic State Evolution

Ordinary differential equation (ODE)–based simulation captures knowledge growth/decay, operability fatigue, and effort-knowledge coupling (Mayer, 2013):

• Learning phases: Training (acquisition) and break (decay/recovery).
• Parameters: Assimilation/forgetting rates, complexity, cumulative work thresholds.

3. Psychometric Item Modeling and Validation

The juxtaposition of synthetic response matrices with IRT models enables robust item analytics:

• Rasch (1PL) and GPCM IRT Models: Fit to the binary or ordinal simulated responses, extracting item difficulties $\delta_i$, abilities $\beta_n$, and discrimination parameters (Acquaye et al., 15 Jan 2026, Scarlatos et al., 7 Jul 2025).
• Direct Preference Optimization (DPO): Simulator LLMs are fine-tuned such that response likelihoods align with ground-truth IRT probabilities, using calibrated preference pairs $(r^w, r^\ell)$ over simulated responses (Scarlatos et al., 7 Jul 2025).
• Population-level simulation: Run over empirically sampled or stratified ability histograms for realistic difficulty distribution recovery.

Validation metrics used for external alignment include:

• Pearson $r$, Spearman $\rho$: Correlations between simulated and human percent-correct/IRT item statistics.
• AUCs, FID, MAUVE, Div.KL: Discriminative and distributional alignment metrics on item and student response spaces (Liu et al., 31 Jan 2026).
• Empirical overlap: Analysis of distractor selection and error modes compared to real students (Lu et al., 2024).

Typical top-line results indicate $r=0.75$–$0.82$ on NAEP-aligned math items using role-play LLM simulations, with lower-bound models more accurately recapitulating difficulty rankings than superhuman LLMs (Acquaye et al., 15 Jan 2026). DPO-aligned simulators outperform prompt-only or SFT-only baselines for cold-start item difficulty prediction (Scarlatos et al., 7 Jul 2025).

4. Empirical Properties, Model Selection, and Identity Effects

Numerous empirical findings and design insights emerge from systematic PS$^2$ experimentation:

• Weaker “math” or student-specialized models yield superior alignment: Models such as Gemma-2-9B (72% item accuracy) outperform stronger solvers (Llama-3-70B at 92%) on simulated-versus-real correlations (e.g., $r=0.61$ vs $r=0.44$ at grade 8) (Acquaye et al., 15 Jan 2026).
• Minimal persona cues significantly enhance calibration: Use of unique names or stratified demographics (across race/gender) improves predictive correlation beyond anonymous or ID-tagged agents (Acquaye et al., 15 Jan 2026).
• Prompt engineering dominates zero-shot conditioning: Chain-of-thought exemplars and teacher-as-predictor paradigm yield more realistic error distributions than abstract persona statements (Lu et al., 2024, Li et al., 21 Dec 2025).
• Monotonic proficiency control and ordering: Model-level PS$^2$ via logit interpolation ensures strict ordering of accuracy across proficiency levels, whereas prompt-based methods often break this monotonicity when simulating binaries (Liu et al., 31 Jan 2026).
• Distributional fidelity and error diversity require error-informed models: Synthetic cognitive errors must reflect both procedural and conceptual error types for lower-proficiency alignments; simply noising an upper-bound model yields degraded distributional metrics (Liu et al., 31 Jan 2026).
• Simulation size trade-offs: Increasing classroom size $N$ improves signal but incurs compute cost; $N=50$ is efficient for prototyping, $N=300$ for high-fidelity evaluation (Acquaye et al., 15 Jan 2026).

5. Domain Extensions and Advanced PS$^2$ Formulations

PS$^2$ is generalizable across multiple domains and architectures:

• Knowledge component–oriented simulation: Profile partitioning across “mastery/confusion/unknown” enables highly granular simulation in domains ranging from heuristic evaluation (Lu et al., 2024) to programming (Wu et al., 26 May 2025).
• Epistemic State Specification (ESS): Formalizes simulation as constrained generation under explicit knowledge and misconception variables: $\mathcal{S}_t = (K_t, M_t, R_t)$, enabling consistency, error-mode control, and learning-dynamics simulation (Yuan et al., 9 Jan 2026).
• Dynamic learning simulation: ODE frameworks and state-space models allow for simulation of longitudinal student trajectories, integrating fatigue, teacher effects, and clickstream data for adaptation (Mayer, 2013).
• Scalability and token limitations: Prompt chunking and profile selection allow PS$^2$ to adapt to different domain sizes and data availability (Xu et al., 2023).

6. Limitations, Controversies, and Open Challenges

Despite clear progress, PS$^2$ frameworks face substantial methodological and practical limitations:

• Competence paradox and introspection gap: Strongest LLMs cannot authentically replicate human error patterns—being too accurate impedes simulation; most models cannot reliably predict their own failure modes (AUROC $\approx0.55$–$0.67$) (Li et al., 21 Dec 2025).
• Shallow fidelity of prompt-based personas: Zero-shot personas frequently fail to shift response distributions meaningfully, especially for low proficiency, necessitating fine-tuning or explicit error modeling (Li et al., 21 Dec 2025).
• Ground-truth calibration constraints: Real-student response logs are labor-intensive or restricted, and synthetic error datasets may insufficiently capture the true diversity of human misconceptions (Liu et al., 31 Jan 2026).
• Ethical considerations: Synthetic data avoids privacy concerns but may inadvertently encode or reinforce demographic or ability-based stereotypes via prompt or model biases (Acquaye et al., 15 Jan 2026).
• Agentic and longitudinal simulation remains underexplored: Most systems simulate static proficiency; integrating knowledge tracing, forgetting, and environment-dependent progression is a recognized research frontier (Yuan et al., 9 Jan 2026).

7. Practical Implications and Future Directions

PS$^2$ underpins a variety of applications in educational technology, psychometrics, and test item development:

• Low-cost, scalable difficulty screening: Enables rapid pretesting of new item sets prior to expensive human pilots (Acquaye et al., 15 Jan 2026, Scarlatos et al., 7 Jul 2025).
• Diagnosis of question flaws: Synthetic response data can highlight ambiguous or misleading distractors and inform question revision (Lu et al., 2024).
• Item bank augmentation: Supports cold-start estimation in adaptive learning or assessment systems (Scarlatos et al., 7 Jul 2025).
• Personalized AI tutoring: Fine-grained simulation may allow tutors to anticipate error modes and adapt interventions to simulated "student" profiles (Liu et al., 31 Jan 2026).
• Psychometric validation and new metrics: Supports automated psychometric analyses, including IRT-based item scoring and student ability estimation.

Future research directions include extending PS$^2$ with richer misconception taxonomies, dialogic and collaborative simulations, agentic learning/forgetting dynamics, and continual calibration on emergent response data. Robust evaluation frameworks that move beyond correlation to structural and behavioral alignment will be essential for PS$^2$'s maturation as a foundational technology for AI-augmented education research (Yuan et al., 9 Jan 2026, Li et al., 21 Dec 2025).