Generative Item Response Theory (G-IRT)

• G-IRT is a generative reformulation of classical IRT that transforms response data into latent traits via an explicit generative mechanism.
• It uses a diagnosis function and reconstruction module to perform inductive inference, allowing quick diagnosis of new learners without re-optimization.
• The model blends classical psychometrics with deep generative methods, addressing calibration bottlenecks and enhancing adaptive testing strategies.

Generative Item Response Theory (G-IRT) is a generative reformulation of item response theory in which diagnostic variables are produced by an explicit generation mechanism from response data rather than being obtained only through transductive parameter fitting. In the explicit formulation introduced in "Generative Cognitive Diagnosis" (Li et al., 13 Jul 2025), G-IRT is the generative reworking of classical 2PL IRT: a learner’s response vector is mapped to latent traits by a generative diagnosis function, while the ordinary IRT response function is retained as a reconstruction module during training. Related work uses the same generative logic more broadly, treating item responses as draws from latent-variable models with Bayesian nonparametric priors, probabilistic autoencoders, adversarial variational inference, or LLM simulators (Duck-Mayr et al., 2020, Chang et al., 2019, Luo et al., 15 Feb 2025, Ormerod, 5 Jan 2026).

1. Classical IRT background and the motivation for a generative reformulation

Classical IRT models observed responses as conditionally independent draws from a latent trait model. In the standard two-parameter form used as the starting point for G-IRT, the probability of a correct response is

$$P(y_{ij}=1 \mid \theta_i,a_j,b_j) = \frac{1}{1+\exp\{-a_j(\theta_i-b_j)\}}.$$

Here, $\theta_i$ is learner ability, $a_j$ is item discrimination, and $b_j$ is item difficulty (Li et al., 13 Jul 2025). In the three-parameter logistic model, the response function is

$$p_{ij} (\theta_j )= c_i + \frac{1- c_i}{1+ e^{-a_i (\theta_j-b_i)} },$$

with $c_i$ the guessing parameter (Lalor et al., 2016).

The classical estimation regime is transductive: latent person and item parameters are optimized so that the response model fits the observed score matrix. "Generative Cognitive Diagnosis" states that this paradigm has two key issues: no instant diagnosis for new learners without expensive re-optimization, and limited reliability because latent estimates may be non-identifiable or hard to interpret (Li et al., 13 Jul 2025). The same psychometric background also motivates IRT-based evaluation beyond education. In NLP evaluation, IRT was used to show that high accuracy does not necessarily imply a high latent ability score, because response patterns depend on item difficulty and discrimination rather than raw item counts alone (Lalor et al., 2016).

The generative reformulation arises from this bottleneck. Instead of treating latent traits as quantities that must be re-estimated whenever new response data arrive, G-IRT treats diagnosis as a learned generation problem: response patterns are transformed into latent traits directly by a parameterized generator, and reconstruction through an IRT response model supplies the training signal (Li et al., 13 Jul 2025).

2. The generative diagnosis paradigm and the formal G-IRT construction

The central object in G-IRT is the generative diagnosis function

$g_\phi : \mathbb{R}^Z \to \Theta \times \Psi,$

parameterized by $\phi$, which maps learner and item response vectors to latent traits (Li et al., 13 Jul 2025). The paper describes the generative process in two stages: data aggregation, which converts sparse response logs into compact learner and item response vectors, and feature generation, which maps those vectors to latent learner and item traits. This shifts the optimization target from latent variables $(\Theta,\Psi)$ to the GDF parameters $\phi$ (Li et al., 13 Jul 2025).

Training is reconstruction-based but diagnosis is generative. The overall objective is

$\theta_i$0

with generated traits $\theta_i$1 (Li et al., 13 Jul 2025). The paper characterizes G-IRT mathematically as being built by taking the inverse of the 2PL-IRT response function and replacing unobserved quantities with proxy parameters. Final trait estimates are then obtained by averaging across all non-missing observations for each learner or item (Li et al., 13 Jul 2025).

Two reliability conditions are emphasized. The first is identifiability: identical learner response vectors should imply identical learner traits, and identical item response vectors should imply identical item features. The second is monotonicity: higher mastery should not imply lower probability of correct responses on relevant items (Li et al., 13 Jul 2025). In this formulation, proxy parameters are internal generators rather than the final diagnostic outputs. The diagnostic outputs are the generated $\theta_i$2, $\theta_i$3, and $\theta_i$4 that are subsequently passed through the IRT reconstruction module (Li et al., 13 Jul 2025).

This architecture yields inductive inference. For a new learner, the procedure is to transform the learner’s responses into a response vector, apply the trained generator, and return $\theta_i$5, with no re-optimization of latent parameters required (Li et al., 13 Jul 2025). That property is the main departure from conventional IRT calibration.

3. Broader generative IRT model classes

The explicit G-IRT formulation sits within a larger family of generative latent-trait models. One major line is Bayesian nonparametric IRT. GPIRT is presented as a fully Bayesian, nonparametric generative model in which each respondent has a latent ability $\theta_i$6, each item has an unknown latent response function $\theta_i$7, and binary responses are generated via

$\theta_i$8

Each $\theta_i$9 receives an independent Gaussian process prior, allowing non-monotonic, non-saturating, asymmetric, or otherwise nonstandard item response functions while retaining joint inference over respondent traits and item functions (Duck-Mayr et al., 2020).

A dynamic extension appears in "A Dynamic, Ordinal Gaussian Process Item Response Theoretic Model" (Chen et al., 3 Apr 2025). GD-GPIRT combines Gaussian-process priors over latent trait trajectories with Gaussian-process priors over item response functions, and uses an ordinal threshold likelihood

$a_j$0

In this construction, latent trajectories are generated from a GP over time rather than a random walk, and ordinal observations are generated by thresholding a flexible latent utility induced by a GP-based item response function (Chen et al., 3 Apr 2025).

Another line interprets IRT through deep latent-variable modeling. "Probabilistically-autoencoded horseshoe-disentangled multidomain item-response theory models" (Chang et al., 2019) treats IRT as the decoder in a probabilistic autoencoder and couples it to a Bayesian neural-network encoder for amortized scoring. The same paper uses a sparsity-promoting horseshoe prior to factor items into latent domains directly within the IRT model, rather than requiring post hoc exploratory factor analysis (Chang et al., 2019). "Generative Adversarial Networks for High-Dimensional Item Factor Analysis" (Luo et al., 15 Feb 2025) updates this perspective by replacing standard VAE inference with adversarial variational Bayes and an importance-weighted objective. The decoder remains the psychometric response model, but the approximate posterior over latent traits is made more expressive through a discriminator network that estimates the KL term (Luo et al., 15 Feb 2025).

The broader literature also preserves the generative semantics of classical IRT while focusing on calibration and scalability rather than redefining diagnosis. "Regularized Bayesian calibration and scoring of the WD-FAB IRT model" (Chang et al., 2020) treats the graded response model as a generative probabilistic model and shows that regularized Bayesian calibration predicts held-out response patterns better than marginal maximum likelihood. "Scalable Learning of Item Response Theory Models" (Frick et al., 2024) exploits the equivalence between alternating 2PL subproblems and logistic regression, then compresses these subproblems using coresets to make large-scale latent-variable learning tractable.

This suggests that G-IRT has both a narrow and a broad usage. In the narrow usage, it denotes the explicit generative diagnosis paradigm of (Li et al., 13 Jul 2025). In the broader usage, it refers to a family of models that retain the IRT idea of responses being generated from latent person-item interactions while replacing fixed calibration pipelines with more flexible generative mechanisms.

4. LLMs, open-ended responses, and response simulation

A recent development extends generative IRT into response simulation with LLMs. "Reconstructing Item Characteristic Curves using Fine-Tuned LLMs" (Ormerod, 5 Jan 2026) fine-tunes Qwen-3 dense models with LoRA to simulate student responses across a spectrum of latent abilities. The model is conditioned on discrete ability descriptors such as Critical, Basic, Proficient, and Exemplary, and the probability assigned to the correct option is used to reconstruct synthetic item characteristic curves. The standard 2PL form is written as

$a_j$1

while multiple-choice responses are related to a nominal response model through a softmax over answer-option logits (Ormerod, 5 Jan 2026). In this setting, the LLM functions as a conditional response generator whose class-conditional output probabilities approximate a discrete ICC.

A separate expansion appears in "gencat: Generative computerized adaptive testing" (Feng et al., 23 Feb 2026). Its GIRT model is designed for open-ended programming responses rather than binary correctness. Student knowledge is represented in both an explicit KC mastery vector and a low-dimensional latent vector, linked by

$a_j$2

The model predicts the actual response code $a_j$3 conditioned on question text and student knowledge, using a supervised fine-tuning objective together with a KC-alignment loss (Feng et al., 23 Feb 2026). To align generation with student mastery, the paper adds a second-stage preference optimization step after supervised fine-tuning. At test time, the model updates the learner-specific latent vector $a_j$4 while keeping global model parameters fixed, then samples possible responses to candidate questions for question selection (Feng et al., 23 Feb 2026).

The adaptive-testing layer is intrinsically generative. GENCAT proposes three question-selection criteria based on sampled responses: uncertainty, semantic diversity, and information. The information criterion is defined as

$a_j$5

In contrast to standard CAT, which typically selects questions using correctness probabilities alone, this framework selects questions based on distributions over generated open-ended responses (Feng et al., 23 Feb 2026).

5. Empirical evidence and operational behavior

The empirical record for G-IRT is heterogeneous because different papers target different tasks: new-learner diagnosis, flexible calibration, adaptive testing, or synthetic ICC reconstruction. The following results are representative.

Setting	Representative result	Source
Generative diagnosis for new learners	about a $a_j$6 speedup for diagnosis of new learners	(Li et al., 13 Jul 2025)
ASSIST score reconstruction	accuracy $a_j$7, F1 $a_j$8, RMSE $a_j$9	(Li et al., 13 Jul 2025)
Math1 score reconstruction	accuracy $b_j$0, F1 $b_j$1, RMSE $b_j$2	(Li et al., 13 Jul 2025)
Identifiability in G-IRT	IDS for learner traits $b_j$3, IDS for item traits $b_j$4 on both datasets	(Li et al., 13 Jul 2025)
Early-stage adaptive testing	AUC improvement of up to $b_j$5 in the key early testing stages	(Feng et al., 23 Feb 2026)
CodeWorkout at $b_j$6	GENCAT (Diversity) AUC $b_j$7; 1PL_IRT AUC $b_j$8; NCAT AUC $b_j$9	(Feng et al., 23 Feb 2026)
Knowledge-response alignment	SFT Pearson $$p_{ij} (\theta_j )= c_i + \frac{1- c_i}{1+ e^{-a_i (\theta_j-b_i)} },$$0; DPO Pearson $$p_{ij} (\theta_j )= c_i + \frac{1- c_i}{1+ e^{-a_i (\theta_j-b_i)} },$$1	(Feng et al., 23 Feb 2026)
Grade 6 ELA ICC reconstruction	Qwen-14B: 1PL difficulty Pearson $$p_{ij} (\theta_j )= c_i + \frac{1- c_i}{1+ e^{-a_i (\theta_j-b_i)} },$$2, RMSE $$p_{ij} (\theta_j )= c_i + \frac{1- c_i}{1+ e^{-a_i (\theta_j-b_i)} },$$3; 2PL discrimination Pearson $$p_{ij} (\theta_j )= c_i + \frac{1- c_i}{1+ e^{-a_i (\theta_j-b_i)} },$$4, RMSE $$p_{ij} (\theta_j )= c_i + \frac{1- c_i}{1+ e^{-a_i (\theta_j-b_i)} },$$5	(Ormerod, 5 Jan 2026)
BEA 2024 difficulty prediction	Qwen-8B: Pearson $$p_{ij} (\theta_j )= c_i + \frac{1- c_i}{1+ e^{-a_i (\theta_j-b_i)} },$$6, RMSE $$p_{ij} (\theta_j )= c_i + \frac{1- c_i}{1+ e^{-a_i (\theta_j-b_i)} },$$7	(Ormerod, 5 Jan 2026)

These results separate several operational claims. First, the explicit G-IRT diagnosis paradigm supports inductive scoring with no retraining requirement and a substantial inference-time gain for new learners (Li et al., 13 Jul 2025). Second, generative adaptive testing with open-ended responses improves early-stage question selection quality relative to binary-response CAT baselines (Feng et al., 23 Feb 2026). Third, LLM-based response simulation can reconstruct ICCs and appears especially effective at discrimination estimation, though the output is still evaluated against psychometric baselines rather than used as a full replacement for field testing (Ormerod, 5 Jan 2026).

Outside the explicit G-IRT label, related generative models also report stronger fit or flexibility. GPIRT emphasizes recovery of nonstandard response curves and active learning (Duck-Mayr et al., 2020); GD-GPIRT reports advantages in correlation between estimated and true traits, RMSE of item characteristic curves, and response prediction accuracy, especially for ordinal data (Chen et al., 3 Apr 2025); AVB- and IWAVB-based item factor analysis achieves higher likelihood than standard VAE or IWAE baselines, with especially clear gains under multimodal latent distributions (Luo et al., 15 Feb 2025).

6. Misconceptions, limitations, and ongoing debates

One recurring misconception is to equate generative IRT with neural text generation. The literature is broader than that. GPIRT, GD-GPIRT, Bayesian GRM calibration, and multidomain horseshoe-autoencoded IRT are all explicitly generative in the latent-variable sense without using LLMs or textual generators (Duck-Mayr et al., 2020, Chen et al., 3 Apr 2025, Chang et al., 2020, Chang et al., 2019). Conversely, some LLM-based systems described as G-IRT-style are best interpreted as applications or extensions of the generative logic rather than replacements for psychometric structure.

A second issue is identifiability. The generative diagnosis paper treats identifiability as a first-class design constraint (Li et al., 13 Jul 2025), and the continuous-response $$p_{ij} (\theta_j )= c_i + \frac{1- c_i}{1+ e^{-a_i (\theta_j-b_i)} },$$8-IRT literature shows why this matters. $$p_{ij} (\theta_j )= c_i + \frac{1- c_i}{1+ e^{-a_i (\theta_j-b_i)} },$$9-IRT suffers from a symmetry problem in which a response near 1 can be explained either by high ability relative to difficulty with positive discrimination or by low ability relative to difficulty with negative discrimination. $c_i$0-IRT addresses this by decomposing discrimination into sign and magnitude and reports a reduction in inverted discrimination signs from roughly $c_i$1–$c_i$2 in $c_i$3-IRT to roughly $c_i$4–$c_i$5 in $c_i$6-IRT (Ferreira-Junior et al., 2023). This suggests that generative expressiveness and psychometric interpretability remain tightly coupled.

A third issue is scope. Person-fit analysis for distinguishing human and generative AI responses to multiple-choice assessments uses IRT to model human response regularities and flags aberrant patterns with $c_i$7, $c_i$8, $c_i$9, and $g_\phi : \mathbb{R}^Z \to \Theta \times \Psi,$0, but the paper explicitly frames this as an extension of IRT in application and interpretation, not a new IRT model class (Strugatski et al., 2024). It also reports that detection weakens as AI cheating becomes more prevalent, and unusual person-fit can arise from non-cheating causes such as atypical response processes (Strugatski et al., 2024).

LLM-based calibration and generative CAT have additional limitations. The ICC-reconstruction study states that the method is not yet reliable enough to replace live field testing for high-stakes use and notes prompt sensitivity, small datasets, and a simplifying assumption of uniform incorrect-choice behavior for BEA (Ormerod, 5 Jan 2026). GENCAT reports computational cost from sampling multiple responses per candidate question, dependence on human-annotated KCs, and domain limitation to programming datasets (Feng et al., 23 Feb 2026).

The current literature therefore supports a restrained interpretation. G-IRT is not a single model family but a developing research program organized around a common principle: diagnosis, calibration, or adaptive testing is reformulated so that latent psychometric structure is generated from response data under explicit probabilistic or neural generative mechanisms. The strongest empirical evidence so far concerns inductive diagnosis speed, flexible response-function modeling, and improved early-stage adaptive testing; the main unresolved questions concern reliability under distribution shift, identifiability under richer parameterizations, and how far generative simulators can substitute for conventional calibration data (Li et al., 13 Jul 2025, Chen et al., 3 Apr 2025, Ormerod, 5 Jan 2026, Feng et al., 23 Feb 2026).