KT4EQG: Personalized EQG via Knowledge Tracing

• KT4EQG is a personalized educational question generation framework that customizes questions based on detailed modeling of a student’s knowledge state.
• It employs a hierarchical Hidden-Markov model for knowledge tracing to assess concept mastery and select the optimal concept for question generation.
• The system couples KT with reinforcement learning and alignment training to ensure generated questions are both concept-faithful and highly answerable.

Searching arXiv for the target paper and closely related work on educational question generation and knowledge tracing. Search query: KT4EQG personalized exercise question generation knowledge tracing arXiv KT4EQG is a personalized Educational Question Generation (EQG) framework that couples knowledge tracing (KT) with large-language-model-based question generation in order to synthesize exercise questions for individual students (Gao et al., 24 Apr 2026). Its central premise is that an effective EQG system should personalize questions by modeling a student’s knowledge state, selecting the knowledge concept that offers the greatest prospective learning benefit, and then generating a question faithfully grounded in that concept. In the reported experiments on XES3G5M and MOOCRadar, KT4EQG consistently generates more effective questions than methods with limited or no personalization, while maintaining high answerability (Gao et al., 24 Apr 2026).

1. Problem setting and conceptual scope

Educational Question Generation aims to synthesize customized exercise questions that enhance student learning. The formulation underlying KT4EQG emphasizes that personalization should be fine-grained: the system should not merely produce plausible questions, but should choose what the student should practice next, conditional on the student’s historical performance (Gao et al., 24 Apr 2026).

The framework addresses a gap identified in prior EQG practice: few existing EQG approaches are able to achieve such fine-grained personalization. KT4EQG explores how EQG can benefit from knowledge tracing, which models students’ knowledge states based on historical performance and predicts future performance. The resulting architecture separates personalization into two coupled subproblems. First, a KT model estimates concept-level mastery and evaluates the educational value of candidate concepts. Second, an LLM-based generator produces a question aligned with the selected concept.

This suggests a definition of personalization that is stronger than prompt-level conditioning alone. In KT4EQG, personalization is tied to an explicit latent-state model of mastery and to a decision rule that optimizes expected improvement in overall knowledge mastery, rather than to surface adaptation of question wording.

2. Knowledge tracing backbone

KT4EQG uses the KT² model, described as a hierarchical Hidden-Markov tree, to trace concept-level mastery (Gao et al., 24 Apr 2026). Let $\mathcal{C}$ be the set of knowledge concepts (KCs). For each concept $c \in \mathcal{C}$, $K_c \in \{0,1\}$ is a hidden variable indicating mastery. Each practice question $X_t$ is tagged with a single KC $\psi(X_t)$, and the student’s response $A_t \in \{0,1\}$ depends only on $K_{\psi(X_t)}$.

The generative assumptions are factorized over the KC tree. The transition model is

$$p\bigl(\{K_c\}_{c\in\mathcal{C}}\bigr) = \prod_{c\in\mathcal{C}} p\bigl(K_c\mid K_{P(c)}\bigr),$$

where

$$p\bigl(K_c=1\mid K_{P(c)}=m\bigr) = \begin{cases} 1 & \text{if } m=1,\ \gamma_c & \text{if } m=0, \end{cases}$$

and $\gamma_c$ is a learnable “slip” parameter; root nodes omit the parent. The emission model is

$c \in \mathcal{C}$0

where $c \in \mathcal{C}$1 is the “learned” correct-if-mastered probability, fixed to medium difficulty in evaluation, and $c \in \mathcal{C}$2 is the guess-rate.

Given a student history $c \in \mathcal{C}$3, posterior mastery is computed for each concept as $c \in \mathcal{C}$4 via the upward–downward algorithm on the KC tree. The model also predicts next-question correctness for any candidate concept $c \in \mathcal{C}$5:

$c \in \mathcal{C}$6

As new $c \in \mathcal{C}$7 pairs arrive, the history $c \in \mathcal{C}$8 is augmented and posteriors are updated incrementally. In the learning assumption used in the framework description, these arrivals are taken to satisfy $c \in \mathcal{C}$9.

All parameters $K_c \in \{0,1\}$0 are fit by Expectation–Maximization to maximize

$K_c \in \{0,1\}$1

Within the framework, KT therefore functions not only as a predictor of correctness but as the state estimator that defines the optimization target for personalization.

3. Concept selection as an educational objective

The next-practice concept is selected by maximizing total mastery over all concepts after a hypothetical correct attempt (Gao et al., 24 Apr 2026). The objective is

$K_c \in \{0,1\}$2

An equivalent expected-gain formulation is also given:

$K_c \in \{0,1\}$3

with the expectation collapsing to $K_c \in \{0,1\}$4 under the framework’s learning assumption.

Operationally, for each candidate concept $K_c \in \{0,1\}$5, KT² inference is run in a hypothetical history $K_c \in \{0,1\}$6 to compute posterior $K_c \in \{0,1\}$7 for all $K_c \in \{0,1\}$8 and then sum those values. This is done exhaustively over all leaf-concepts.

This formulation makes the personalization criterion explicit. The selected concept is not simply the least-mastered concept, nor merely the concept with highest predicted immediate correctness. Instead, the objective is the concept whose successful practice would maximize overall mastery over the KC set. A plausible implication is that the framework embeds prerequisite structure directly into pedagogical choice through the hierarchical KT² state space.

4. Question generation and alignment training

After concept selection, KT4EQG uses an LLM-based question generator to produce a question conditioned on the target concept (Gao et al., 24 Apr 2026). A pretrained LLM $K_c \in \{0,1\}$9—Qwen3-8B—is fine-tuned to map from a concept token $X_t$0 to question text $X_t$1. In practice, a small prompt template is prepended, embedding the instruction “Generate a question for concept: c=…”.

The supervised fine-tuning objective is standard cross-entropy over aligned concept–question pairs:

$X_t$2

To increase concept fidelity, KT4EQG introduces reinforcement learning with an alignment reward. A verifier $X_t$3 is trained to score alignment using a contrastive InfoNCE loss:

$X_t$4

The scalar alignment reward is standardized and passed through a sigmoid:

$X_t$5

where $X_t$6 are the mean and standard deviation over concepts for a fixed $X_t$7. The frozen base LLM acts as a KL-constraint, and $X_t$8 is optimized by a policy gradient method, exemplified by GRPO, to maximize expected reward minus a KL-penalty:

$X_t$9

By using $\psi(X_t)$0 as reward, the generator is explicitly grounded to produce questions whose content the verifier judges as aligned with the target concept. In the terminology of the framework, this is the principal fidelity constraint on generation.

5. End-to-end operation

The single-round integration pipeline has four stages (Gao et al., 24 Apr 2026). First, for each candidate KC, the framework computes

$\psi(X_t)$1

and sets $\psi(X_t)$2. Second, it samples a question $\psi(X_t)$3. Third, the student practices; under the simulation assumption, the observed response is $\psi(X_t)$4, and the history is updated as $\psi(X_t)$5. Fourth, the posterior $\psi(X_t)$6 is recomputed by upward–downward inference.

In multi-round simulation, the same process is repeated for $\psi(X_t)$7 practice steps. In a live system, real student responses $\psi(X_t)$8 replace the “always correct” assumption, and the history is updated accordingly.

This separation between concept optimization and textual generation is structurally important. The educational decision is made in the KT space, while the LLM is constrained to realize that decision as a concept-faithful question. A plausible implication is that failures can be localized: weak concept choice and weak question fidelity are treated as distinct error modes.

6. Experimental configuration and empirical results

The experimental study uses two datasets and evaluates both learning-oriented effectiveness and generation quality (Gao et al., 24 Apr 2026).

Dataset	Overall scale	Modules used
XES3G5M	18 066 students, 7 652 questions, 865 KCs	Application: 300 students, 66 leaf KCs; Computation: 300/40 KCs; Counting: 300/28 KCs
MOOCRadar	14 224 students, 2 513 questions, 5 600 fine-grained KCs	Wine: 300/4; Circuit: 260/8; Theory: 130/3

Student-level 80/20 train/test splits are used, and from each student truncated histories are created at lengths $\psi(X_t)$9. Evaluation metrics include Exam Score, defined by generating a fixed exam of one question per concept after $A_t \in \{0,1\}$0 simulated practice rounds and reporting average predicted correctness under the KT model; Answerability, defined as the fraction of generated questions deemed solvable by a separate LLM judge, Qwen-3-4B; and, for ablations, Alignment Score $A_t \in \{0,1\}$1 and NLL.

Across all six modules, KT4EQG achieves the highest Exam Score and substantially outperforms zero-shot LLMs (Qwen3-8B), ScaleQuest, PromptCoT, and their “+Oracle” variants, where oracle $A_t \in \{0,1\}$2 is provided. The reported gain is not merely relative to weak baselines: KT4EQG closes $A_t \in \{0,1\}$3–$A_t \in \{0,1\}$4 of the gap between a Random baseline and an Oracle upper bound. Answerability remains $A_t \in \{0,1\}$5 and comparable to the pretrained base model.

The ablation on the Application module isolates the contribution of alignment training:

Model stage	Alignment $A_t \in \{0,1\}$6	NLL
Backbone (Qwen3)	0.78	1.48
+ SFT warm-up	0.88	2.32
+ Alignment RL	0.94	2.27

For the same ablation, Answerability is reported as $A_t \in \{0,1\}$7 for the Backbone, $A_t \in \{0,1\}$8 for + SFT warm-up, and $A_t \in \{0,1\}$9 for + Alignment RL. The paper’s interpretation is that alignment training drives a large gain in $K_{\psi(X_t)}$0 with minimal degradation in fluency, measured by NLL, and preserves answerability.

In multi-turn simulation over $K_{\psi(X_t)}$1 practice rounds, KT4EQG exhibits the steepest gains in exam score, whereas zero-shot LLM baselines plateau early. The concept-selection behavior is also described qualitatively: KT4EQG adaptively picks easier concepts for low-mastery students and increasingly challenging ones as mastery rises, demonstrating a natural “zone of proximal development.”

7. Interpretation, scope, and technical significance

KT4EQG combines a probabilistic KT² model for fine-grained concept-gain estimation with an LLM question generator trained under explicit alignment rewards (Gao et al., 24 Apr 2026). The framework’s empirical claim is therefore twofold: it produces personalized questions that maximize simulated learning gains and it does so while keeping generated questions high-quality and answerable.

Its scope is shaped by the assumptions of the evaluation protocol. Exam Score is computed under the KT model after simulated practice rounds, and the single-round learning assumption sets the newly observed response to $K_{\psi(X_t)}$2. The framework description explicitly distinguishes this simulation regime from live deployment, where real student responses replace the assumption. This indicates that the reported effectiveness is anchored in a KT-mediated simulation of learning rather than in a live classroom intervention.

At the methodological level, KT4EQG is significant because it operationalizes personalization as a closed loop between latent mastery estimation, concept-level decision-making, and concept-faithful generation. This suggests a broader research program in which educational generation systems are evaluated not only by linguistic quality or topical relevance, but also by their effect on a model of student state. Within that framing, KT4EQG can be understood as a tightly integrated architecture for personalized exercise selection and realization, rather than as a standalone question generator.