Proficiency Taxonomy Model (PTM)

Updated 9 July 2026

Proficiency Taxonomy Model is a computational framework that embeds a hierarchical taxonomy of coding proficiencies into student performance prediction.
It leverages educator-informed skills and deep learning components like LSTM and BERT to jointly learn proficiency profiles and predict task struggles.
Empirical evaluations on Java and Python datasets demonstrate improved ROC-AUC, highlighting the benefits of integrating coding history with explicit skill taxonomy.

The Proficiency Taxonomy Model (PTM) is a modeling paradigm in which an explicit taxonomy of proficiencies is embedded into a computational representation of learner state and then used for prediction, interpretation, or control. In its canonical formulation, PTM was introduced for early detection of struggling student programmers: it jointly learns a student’s coding-skill profile from coding history and predicts whether that student will struggle on a new task (Schwartz et al., 24 Aug 2025). Related work suggests a broader usage in which structured proficiency categories serve as intermediate state variables in knowledge tracing, simulation, or graded user modeling rather than remaining purely descriptive labels (Park et al., 29 Nov 2025, Breitwieser et al., 2021).

1. Canonical definition and problem setting

PTM was introduced to address a specific limitation in prior AI-based detectors of struggling student programmers: such systems did not explicitly reason about students’ programming skills in the model. The PTM formulation therefore embeds a taxonomy of coding proficiencies directly into the detection architecture. The prediction problem is defined as follows: given a student’s complete coding history up to a new task, the model predicts whether the student will struggle with that task (Schwartz et al., 24 Aug 2025).

The model is teacher-informed from the outset. Its taxonomy was developed through in-depth interviews with two Computer Science teachers with 30+ years of experience and two university-level CS1 course TAs. In these interviews, educators identified common struggle points including basic task comprehension, correct application of learned concepts, algorithmic design, debugging, and documentation. Survey data from educators and literature further emphasized incorrect application of programming concepts as a core indicator of struggle, alongside excessive time to solution, lower grades, and loss of motivation (Schwartz et al., 24 Aug 2025).

A central property of PTM is that the taxonomy is not external annotation alone. It is embedded into the model as a latent-but-structured proficiency profile, and this profile is optimized jointly with the downstream struggle classifier. This design gives PTM a dual role: it is both a representation of student ability and a mechanism for task-specific prediction (Schwartz et al., 24 Aug 2025).

2. Taxonomic foundations and representational structure

The coding taxonomy underlying PTM is hierarchical and educator-designed. It is inspired by Bloom’s Taxonomy and programming education best practices, and it aligns observable code-level skills such as loops, conditionals, and logical operators with higher-level or latent skills such as decomposition, advanced problem-solving, edge-case handling, testing, debugging, and documentation. The taxonomy covers five layers: Engagement, Understanding, Application, Problem Solving, and Quality Assurance (Schwartz et al., 24 Aug 2025).

Within the model, this taxonomy is instantiated as the Taxonomy-Based Proficiency Profile (TBPP). TBPP is a vector embedding representing proficiency in 13 skills: 10 observed skills, inferred from code and aligned with programming concepts, and 3 latent skills, intended to capture higher-order proficiencies such as decomposition and advanced problem-solving. Each skill score ranges from 0 to 1 (Schwartz et al., 24 Aug 2025).

Related work shows that PTM-like representations vary by domain while preserving the same underlying logic: a structured proficiency space is defined first, and learner or artifact state is then mapped into that space. In mathematical learning, StatusKT operationalizes mathematical proficiency as four observable strands—Conceptual Understanding (CU), Strategic Competence (SC), Procedural Fluency (PF), and Adaptive Reasoning (AR)—while omitting Productive Disposition (PD) because it is challenging to reliably assess from textual student traces. In AI-generated Python code analysis, proficiency is operationalized through a CEFR-inspired six-level taxonomy from A1 to C2 via static construct analysis. These variants indicate that PTM is best understood as a structured modeling template rather than a single fixed taxonomy (Park et al., 29 Nov 2025, Temkulkiat et al., 31 Mar 2026).

Context	Categories or levels	Operationalization
Student programming PTM	Engagement, Understanding, Application, Problem Solving, Quality Assurance	TBPP with 10 observed and 3 latent skills
Mathematical proficiency tracing	CU, SC, PF, AR; PD omitted	Indicator-based proficiency vector
Python code proficiency	A1, A2, B1, B2, C1, C2	`pycefr` maps constructs to levels

A recurrent theme across these systems is multidimensionality. Proficiency is not treated as a single scalar competence variable, but as a structured vector over partially separable skill dimensions. This suggests that PTM-style models are especially appropriate when the target task depends on heterogeneous subskills rather than a monolithic notion of ability.

3. PTM architecture and learning objective

The canonical PTM architecture is multi-task and sequence-based. Its inputs include the student history $H = \{H_1, H_2, \ldots, H_k\}$ , where each $H_i$ is a sequence of code submissions for task $i$ ; past struggling indicators $s_i$ ; the target student ID; and the target coding task, represented by its description and required programming concepts (Schwartz et al., 24 Aug 2025).

PTM has two main components. The first is the Taxonomy-Based Proficiency Profile (TBPP) estimator. For each task $i$ , the student’s submissions are embedded with CodeBERT and aggregated through an LSTM into a task vector $Z_i$ . The sequence $(Z_1, \ldots, Z_K)$ over prior tasks is then processed by another LSTM, producing a single representation that is fed through a linear layer to output the 13 skill scores. The second component is the struggle predictor. It combines TBPP with a BERT encoding of the target task text and a binary vector of required programming concepts (Schwartz et al., 24 Aug 2025).

Task alignment is implemented explicitly. PTM uses cross-attention between the task representation and TBPP so that the model can weight which proficiencies matter most for the new task. The required concepts, together with latent skills that are always active, form a skill weight vector that is processed by an MLP and multiplied element-wise with TBPP to produce a task-relevant weighted TBPP. This weighted TBPP and the cross-attended task representation are concatenated and passed through a final MLP for binary struggle classification (Schwartz et al., 24 Aug 2025).

PTM is trained with a joint objective: $L = a \cdot MAE(\hat{y}, y) + (1 - a) \cdot BCE(\hat{t}, t)$ where $MAE(\hat{y}, y)$ is the error between predicted and target TBPP, $BCE(\hat{t}, t)$ is the binary cross-entropy for struggle prediction, and $H_i$ 0 is a balancing hyperparameter set to $H_i$ 1 in the experiments. Ground-truth TBPP is estimated from a task-skill matrix $H_i$ 2 and a student score vector $H_i$ 3 using

$H_i$ 4

In the reported implementation, PTM is trained with Adam, hidden size 512, learning rate 0.0001, batch size 32, and 15 epochs (Schwartz et al., 24 Aug 2025).

The architectural significance of PTM lies in this coupling of longitudinal sequence modeling with explicit skill structure. The taxonomy does not merely annotate model outputs after the fact; it shapes the latent state, the auxiliary target, and the task-conditioning mechanism.

4. Empirical evaluation in programming education

PTM was evaluated on two introductory programming datasets. CodeWorkout (Java) contains approximately 630 students and 50 coding tasks in a university-level introductory programming setting with relatively uniform task structure. FalconCode (Python) contains approximately 1,330 students and 408 coding tasks, with a mix of small exercises and longer labs and greater variation in task order across students. A student is labeled as struggling on a task if they fail any unit test or require more attempts than 75% of their peers (Schwartz et al., 24 Aug 2025).

The evaluation protocol uses the first 30 tasks to predict struggle on the next 20 tasks, without retraining, under 5-fold cross-validation with student-level splits so that no student appears in both training and test sets. Baselines are DKT, Code-DKT, and SAKT, and the main evaluation metric is ROC-AUC, with statistical significance assessed via paired bootstrap and Bonferroni correction (Schwartz et al., 24 Aug 2025).

Model	CodeWorkout ROC-AUC (%)	FalconCode ROC-AUC (%)
DKT	76.27 ± 2.2	63.54 ± 1.8
Code-DKT	70.93 ± 4.3	62.17 ± 2.0
SAKT	76.11 ± 1.09	70.52 ± 0.8
PTM	77.09 ± 1.8	73.18 ± 0.9

Ablation studies isolate the contributions of history and taxonomy. A No-Tax No-Hist variant achieves 72.50 ± 3.5 on CodeWorkout and 69.13 ± 1.3 on FalconCode, while a No-Tax variant with history but without the taxonomy achieves 75.52 ± 1.7 and 71.33 ± 0.6 respectively. The full PTM yields the best scores on both datasets, indicating that coding history and taxonomy contribute complementary signal. Sensitivity analysis further shows that longer histories improve performance, with the best results obtained when at least 22–30 prior tasks are included (Schwartz et al., 24 Aug 2025).

The dataset-level pattern is also informative. PTM performs better on CodeWorkout than on FalconCode. The reported explanation is that CodeWorkout has more uniform task sequencing and fully online visible data, whereas FalconCode has more variable scheduling and local work that leads to missing observations for some steps (Schwartz et al., 24 Aug 2025). This suggests that PTM benefits from dense and consistently ordered evidence when estimating longitudinal proficiency profiles.

5. PTM-like extensions across domains

In mathematics education, the StatusKT framework extends knowledge tracing by incorporating the student’s problem-solving process into the interaction sequence and extracting explicit mathematical proficiency signals. The conventional knowledge tracing sequence

$H_i$ 5

is expanded to include a detailed process term $H_i$ 6, yielding

$H_i$ 7

StatusKT then uses a teacher-student-teacher LLM pipeline: a teacher LLM generates problem-specific proficiency indicators tagged to CU, SC, PF, or AR; a student LLM answers those indicators from the written solution trace; and a teacher LLM evaluates whether each indicator is satisfied. Per-dimension ratios are aggregated into a proficiency vector $H_i$ 8. On the KT-PSP-25 dataset, which contains 22,289 student-problem pairs with OCR-transcribed written solutions, StatusKT improves AUC and ACC for the majority of KT backbones while providing interpretable explanations grounded in explicit proficiency dimensions. The paper explicitly interprets this as a Proficiency Taxonomy Model in both the representational and modeling senses (Park et al., 29 Nov 2025).

A different extension appears in social-media proficiency modeling. Rather than treating proficiency as expert/non-expert, that literature models proficiency as a graded topic-dependent property. Users are represented by features derived from topic query words, with evaluated alternatives including TF, TF-IDF, User2Vec (U2V), Relative User2Vec (Rel-U2V), and LDA. A scalar proficiency score is defined as

$H_i$ 9

and the outputs can be interpreted as user-to-topic score vectors that populate a PTM. The reported findings show that U2V and Rel-U2V are superior or competitive to word-based baselines, whereas LDA is poorer and less reliable for proficiency because the induced topics often track style rather than knowledge domains (Breitwieser et al., 2021).

Proficiency taxonomies have also been applied to generated code rather than learners. In the analysis of AI-agent Python code, the pycefr tool maps static language constructs to CEFR-like levels from A1 through C2. Across 591 pull requests containing 5,027 Python files generated by three AI agents, more than 90% of constructs fall into A1 or A2, while less than 1% are C2. AI-generated and human pull requests exhibit broadly similar proficiency profiles, and high-proficiency code is concentrated in feature addition and bug-fixing tasks (Temkulkiat et al., 31 Mar 2026). This does not define PTM in the learner-modeling sense, but it demonstrates that taxonomy-based proficiency representations can be attached to artifacts and workflows as well as to students.

6. Interpretability, simulation, and terminological boundaries

One important extension of PTM-style thinking is simulation. The PS $i$ 0 framework for student proficiency simulation does not use the PTM name, but it is explicitly taxonomy-driven. It constructs a lower-bound student model by fine-tuning a weak LLM to exhibit conceptual and procedural errors, then interpolates between this lower-bound model and a strong upper-bound LLM using a hybrid ratio $i$ 1. At the logit level,

$i$ 2

with token probabilities generated by a softmax over the interpolated logits. The framework reports monotonic control of behavior via a KL-divergence argument, calibration to academic performance, and empirical gains in student behavior similarity and item difficulty prediction, measured with FID, MAUVE, Diversity KL, PCC, and SCC (Liu et al., 31 Jan 2026). A plausible implication is that PTM principles are not limited to diagnosis; they can also structure controllable simulation when proficiency levels must be synthesized rather than inferred.

Interpretability is a consistent motivation across these lines of work. In canonical PTM, the model exposes a 13-dimensional proficiency profile aligned with educator concepts. In StatusKT, indicator-level assessments are surfaced as explicit intermediate signals. In PS $i$ 3, taxonomy-driven error types make the lower end of the proficiency spectrum cognitively meaningful rather than random. These systems therefore differ from black-box predictors that optimize only final accuracy (Schwartz et al., 24 Aug 2025, Park et al., 29 Nov 2025, Liu et al., 31 Jan 2026).

A recurrent source of confusion is acronym overlap. PTM is also used for the Personalized Thinking Model, a different learner representation grounded in Marzano’s New Taxonomy of Educational Objectives. That PTM organizes learner journals into five layers—behavioral instances, behavioral patterns, cognitive utilization, metacognitive tendencies, and core values—and reports an overall atomic-information-point F1 of 74.57% before human-in-the-loop refinement and 75.48% after refinement, along with user ratings of 4.26 and 4.30 on a five-point scale (Hwang et al., 6 May 2026). Despite the shared acronym and its educational setting, this model is conceptually distinct from the Proficiency Taxonomy Model: it targets hierarchical modeling of thought and self-system structure rather than explicit proficiency-state estimation for prediction.

The literature also clarifies a common conceptual misconception: proficiency is not identical to expertise. In social-media proficiency modeling, proficiency is treated as a graded construct that includes autodidacts, hobbyists, and users with sustained practical exposure, whereas expertise imposes a stronger positive criterion (Breitwieser et al., 2021). That distinction is consistent with PTM-style systems, which typically assign structured, partial, and evolving proficiencies rather than forcing a binary expert classification.