---
title: 'KT-PSP: Process-Aware Knowledge Tracing'
url: https://www.emergentmind.com/topics/knowledge-tracing-leveraging-problem-solving-process-kt-psp
type: topic
---

# KT-PSP: Process-Aware Knowledge Tracing

Knowledge Tracing Leveraging Problem-Solving Process (KT-PSP) denotes a class of knowledge tracing methodologies that incorporate detailed student problem-solving process data—beyond mere correctness labels—to capture granular, multidimensional models of learner proficiency and improve prediction fidelity. Recent KT-PSP frameworks utilize diverse sources of process data including code submissions, stepwise mathematical solutions, procedural annotations, and solution traces, extracting and leveraging domain-specific intermediate signals for both enhanced predictive accuracy and improved interpretability. These approaches address the limitations of classical KT models (e.g., DKT, BKT), which operate exclusively on correctness/outcome sequences, by explicitly modeling students’ in-problem behaviors, thereby facilitating personalized, diagnostic feedback in adaptive learning systems.

## 1. Problem Formulation and Motivation

KT-PSP extends the standard knowledge tracing paradigm by augmenting the student interaction sequence. Classical KT models a student’s knowledge evolution via
$$
S_\text{conv} = \{ (q_1, c_1, r_1), \ldots, (q_t, c_t, r_t) \},
$$
where $q_t$ is the $t$th problem, $c_t$ its knowledge concept(s), and $r_t \in \{0,1\}$ the correctness label. The prediction task is to estimate
$$
P(r_{t+1}=1 \mid q_{t+1}, c_{t+1}, S_\text{conv}).
$$
In KT-PSP, each interaction is further annotated with the observed problem-solving process, leading to
$$
S_\text{PSP} = \{ (q_1, c_1, r_1, p_1), \ldots, (q_t, c_t, r_t, p_t) \},
$$
where $p_t$ encodes the student’s process trace (e.g., source code, handwritten solution steps, multi-step logs). The predictive model then leverages the expanded history to estimate
$$
P(r_{t+1}=1 \mid q_{t+1}, c_{t+1}, S_\text{PSP}),
$$
and, in advanced KP-PSP frameworks such as StatusKT, incorporates process-derived intermediate proficiency vectors $m_t \in \mathbb{R}^D$, where $D$ is the number of proficiency dimensions (e.g., conceptual understanding, procedural fluency) [2512.00311].

KT-PSP is motivated by the empirical limitations of correctness-only KT, which fails to exploit the diagnostic value in partial solutions, error types, intermediate states, and domain-specific behaviors that constitute the bulk of student learning dynamics. Early work on Code-DKT [2206.03545] and KCQRL [2410.01727] illustrates the substantial AUC gains achievable by embedding process-aware signals into the sequential KT pipeline.

## 2. Datasets and Problem-Solving Process Representations

The principal benchmark for mathematical KT-PSP is the KT-PSP-25 dataset [2512.00311], comprising 22,289 digital math sessions with OCR-transcribed solution traces, knowledge concept tags, correctness, and timing metadata. Each $p_t$ is a multi-line (≥5 lines) logically ordered LaTeX representation of a student’s handwritten solution, curated for high process fidelity. Problems span 2,696 items and 490 distinct knowledge concepts, supporting fine-grained mapping between solution steps and conceptual targets.

In programming KT-PSP, the Code-DKT dataset encompasses 410 students and 50 Java programming problems, where each process trace $c_t$ is the raw source code of a student submission [2206.03545].

For general math KT, process annotation can be generated automatically using LLM-based chain-of-thought prompts, followed by step-wise knowledge concept (KC) mapping and semantic alignment [2410.01727]. Automated tools extract solution steps, annotate KCs, and align process steps to domain concepts, supplying detailed input representations for downstream KT modules.

## 3. Architectures and Process-Sensitive Modeling

KT-PSP methodologies instantiate several architectural innovations to process and exploit problem-solving process data. The principal architectural modules include:

- **Process Feature Extraction:** In Code-DKT [2206.03545], code submissions are parsed into ASTs, and code-paths are sampled and encoded via an attention-weighted code2vec variant, yielding a process feature vector $z_t$ for each attempt. For mathematical problem solving (StatusKT [2512.00311]), OCR-extracted solution traces feed into LLM-based pipelines that decompose the process into natural-language proficiency indicators and map evidence of mastery to explicit scores.

- **Intermediate Signal Construction:** StatusKT [2512.00311] deploys a three-stage LLM pipeline:
  1. **Teacher LLM:** Generates lists of problem-specific proficiency indicators covering distinct dimensions (CU, SC, PF, AR).
  2. **Student LLM:** Maps each indicator to a candidate response, as if answering rubric questions with student solutions.
  3. **Teacher LLM:** Evaluates responses for indicator satisfaction, yielding binary scores $e_{t,d,j}$. Averaged by dimension, these form the MP (mathematical proficiency) ratio vector $m_t \in [0,1]^4$.
  These $m_t$ serve as auxiliary inputs to the KT model at each timestep.

- **Process-Enhanced KT Backbone:** The status-aware KT model concatenates or injects process-derived features ($z_t$, $m_t$) with traditional correctness histories and passes them through LSTM, Transformer, or memory-augmented architectures (e.g., DKVMN), enhancing both predictive accuracy and explanatory transparency [2512.00311, 2206.03545].

- **Task-Specific Objective Functions:** Models optimize a composite loss, typically
$$
L = \sum_{t=1}^T \big[ \text{BCE}(r_t, r_{\text{pred},t}) + \alpha \sum_{d} (m_{t,d} - m_{\text{pred},t,d})^2 \big],
$$
balancing response prediction and proficiency regression [2512.00311].

## 4. Model Learning, Evaluation, and Empirical Results

KT-PSP frameworks are typically trained and evaluated on split datasets using metrics such as AUC and RMSE for next-response correctness, and mean-squared error for proficiency prediction. Notable empirical findings:

- StatusKT [2512.00311] achieves consistent performance improvements (ΔAUC up to 0.0149) across ten canonical KT algorithms on KT-PSP-25, including DKT, DKVMN, SAINT, AKT, stableKT, robustKT. Gains are robust to process noise and task type, and ablations show that both the proficiency regression loss and the LLM-based process signal extraction are critical.
- Code-DKT [2206.03545] yields 3.07–4.00% AUC improvement over DKT across five programming assignments, with maximum gains on tasks exhibiting recurring code structures and concept overlap.
- KCQRL [2410.01727] demonstrates generalized process-level gains across 15 KT models (relative AUC increases 1–7%) by employing automated, process-anchored KC annotation and contrastively aligned question representations.
- GRATE [2210.09013] introduces adaptive attempt aggregation and rank-based temporal smoothing to mitigate process noise in complex, multi-concept problems, securing statistically significant improvements over standard tensor factorization and memory-network KT models.

A summary of selected quantitative results appears below:

| Model / Approach | Dataset         | Baseline AUC | KT-PSP AUC | ΔAUC      |
|------------------|----------------|--------------|------------|-----------|
| StatusKT (DKVMN) | KT-PSP-25      | 0.6095       | 0.6220     | +0.0125   |
| Code-DKT         | Java Assignments| See text     | +3.07–4%   |           |
| KCQRL (IEKT)     | XES3G5M        | 82.24        | 82.82      | +0.58     |
| GRATE            | MasteryGrids   | ≈0.688       | 0.7035     | +0.0155   |

These consistent AUC and RMSE gains confirm the hypothesis that rich process information provides actionable, predictive signal unavailable to correctness-only KT models.

## 5. Intermediate Signals and Interpretability

A key advantage of KT-PSP is the explicit modeling and prediction of human-interpretable proficiency vectors at each timestep. In StatusKT [2512.00311], for each problem and student, the model provides estimated scores for Conceptual Understanding (CU), Strategic Competence (SC), Procedural Fluency (PF), and Adaptive Reasoning (AR). These are derived by evaluating LLM-generated, process-specific indicators against the student’s solution trace, facilitating strand-level diagnostics with native interpretability.

For example, a predicted vector
$$
m_{\text{pred}} = [\text{CU}=0.80, \text{SC}=0.60, \text{PF}=0.40, \text{AR}=0.25]
$$
indicates high conceptual mastery but low adaptive reasoning, guiding targeted instructional feedback. Case studies in [2512.00311] demonstrate the validity of this multidimensional scoring for formative assessment.

Process-derived embeddings in Code-DKT [2206.03545] and KCQRL [2410.01727] similarly contribute to interpretability by identifying domain-relevant code patterns or semantic skills aligned with knowledge concepts or solution-step representations.

## 6. Extensions: Process Granularity and Aggregation

KT-PSP research pursues finer process granularity while addressing signal noise and redundancy. GRATE [2210.09013] introduces dynamic attempt aggregation via a rank-based tensor factorization:
- Attempts are automatically merged to eliminate high-noise or uninformative time slices, smoothing knowledge traces while preserving key transitions.
- A soft monotonicity constraint stabilizes predicted mastery trajectories, mitigating the effects of slips and guesses.
- The approach provides interpretable Q-matrices mapping problems to latent concepts, frequently uncovering skill clusters orthogonal to textbook topic labels.

A plausible implication is that adaptive process aggregation will become essential as KT-PSP frameworks scale to more complex, open-ended domains.

## 7. Connections to Related Methodologies and Future Directions

KT-PSP unifies and extends multiple methodological lines:
- LLM-based semantic annotation and stepwise process interpretation [2410.01727, 2512.00311].
- Domain-specific feature extraction from raw code or solution logs [2206.03545].
- Tensor and memory-Augmented KT for multi-concept, multi-step interactions [2210.09013].
- Deep contrastive learning for embedding alignment and noise reduction [2410.01727].

Emerging themes include integration of multimodal process data (e.g., speech, gesture, collaborative logs), real-time proficiency estimation for adaptive intervention, and generalized frameworks for process-sensitive KT beyond mathematics and programming. As dataset scale and LLM sophistication increase, KT-PSP is positioned to support next-generation personalizable tutoring systems with both accurate predictions and fine-grained, explainable feedback.

---
**References:**
- "Tracing Mathematical Proficiency Through Problem-Solving Processes" [2512.00311]
- "Code-DKT: A Code-based Knowledge Tracing Model for Programming Tasks" [2206.03545]
- "Automated Knowledge Concept Annotation and Question Representation Learning for Knowledge Tracing" [2410.01727]
- "Knowledge Tracing for Complex Problem Solving: Granular Rank-Based Tensor Factorization" [2210.09013]

Source: https://www.emergentmind.com/topics/knowledge-tracing-leveraging-problem-solving-process-kt-psp