---
title: Proficiency Taxonomy Model (PTM)
url: https://www.emergentmind.com/topics/proficiency-taxonomy-model-ptm
type: topic
---

# Proficiency Taxonomy Model (PTM)

The **Proficiency Taxonomy Model (PTM)** is a modeling paradigm in which an explicit taxonomy of proficiencies is embedded into a computational representation of learner state and then used for prediction, interpretation, or control. In its canonical formulation, PTM was introduced for early detection of struggling student programmers: it jointly learns a student’s coding-skill profile from coding history and predicts whether that student will struggle on a new task [2508.17353]. Related work suggests a broader usage in which structured proficiency categories serve as intermediate state variables in knowledge tracing, simulation, or graded user modeling rather than remaining purely descriptive labels [2512.00311][2110.08011].

## 1. Canonical definition and problem setting

PTM was introduced to address a specific limitation in prior AI-based detectors of struggling student programmers: such systems did not explicitly reason about students’ programming skills in the model. The PTM formulation therefore embeds a taxonomy of coding proficiencies directly into the detection architecture. The prediction problem is defined as follows: given a student’s complete coding history up to a new task, the model predicts whether the student will struggle with that task [2508.17353].

The model is teacher-informed from the outset. Its taxonomy was developed through in-depth interviews with two Computer Science teachers with 30+ years of experience and two university-level CS1 course TAs. In these interviews, educators identified common struggle points including basic task comprehension, correct application of learned concepts, algorithmic design, debugging, and documentation. Survey data from educators and literature further emphasized incorrect application of programming concepts as a core indicator of struggle, alongside excessive time to solution, lower grades, and loss of motivation [2508.17353].

A central property of PTM is that the taxonomy is not external annotation alone. It is embedded into the model as a latent-but-structured proficiency profile, and this profile is optimized jointly with the downstream struggle classifier. This design gives PTM a dual role: it is both a representation of student ability and a mechanism for task-specific prediction [2508.17353].

## 2. Taxonomic foundations and representational structure

The coding taxonomy underlying PTM is hierarchical and educator-designed. It is inspired by Bloom’s Taxonomy and programming education best practices, and it aligns observable code-level skills such as loops, conditionals, and logical operators with higher-level or latent skills such as decomposition, advanced problem-solving, edge-case handling, testing, debugging, and documentation. The taxonomy covers five layers: **Engagement**, **Understanding**, **Application**, **Problem Solving**, and **Quality Assurance** [2508.17353].

Within the model, this taxonomy is instantiated as the **Taxonomy-Based Proficiency Profile (TBPP)**. TBPP is a vector embedding representing proficiency in **13 skills**: **10 observed skills**, inferred from code and aligned with programming concepts, and **3 latent skills**, intended to capture higher-order proficiencies such as decomposition and advanced problem-solving. Each skill score ranges from 0 to 1 [2508.17353].

Related work shows that PTM-like representations vary by domain while preserving the same underlying logic: a structured proficiency space is defined first, and learner or artifact state is then mapped into that space. In mathematical learning, StatusKT operationalizes mathematical proficiency as four observable strands—**Conceptual Understanding (CU)**, **Strategic Competence (SC)**, **Procedural Fluency (PF)**, and **Adaptive Reasoning (AR)**—while omitting **Productive Disposition (PD)** because it is challenging to reliably assess from textual student traces. In AI-generated Python code analysis, proficiency is operationalized through a CEFR-inspired six-level taxonomy from **A1** to **C2** via static construct analysis. These variants indicate that PTM is best understood as a structured modeling template rather than a single fixed taxonomy [2512.00311][2604.00299].

| Context | Categories or levels | Operationalization |
|---|---|---|
| Student programming PTM | Engagement, Understanding, Application, Problem Solving, Quality Assurance | TBPP with 10 observed and 3 latent skills |
| Mathematical proficiency tracing | CU, SC, PF, AR; PD omitted | Indicator-based proficiency vector |
| Python code proficiency | A1, A2, B1, B2, C1, C2 | `pycefr` maps constructs to levels |

A recurrent theme across these systems is multidimensionality. Proficiency is not treated as a single scalar competence variable, but as a structured vector over partially separable skill dimensions. This suggests that PTM-style models are especially appropriate when the target task depends on heterogeneous subskills rather than a monolithic notion of ability.

## 3. PTM architecture and learning objective

The canonical PTM architecture is multi-task and sequence-based. Its inputs include the student history \(H = \{H_1, H_2, \ldots, H_k\}\), where each \(H_i\) is a sequence of code submissions for task \(i\); past struggling indicators \(s_i\); the target student ID; and the target coding task, represented by its description and required programming concepts [2508.17353].

PTM has two main components. The first is the **Taxonomy-Based Proficiency Profile (TBPP)** estimator. For each task \(i\), the student’s submissions are embedded with **CodeBERT** and aggregated through an **LSTM** into a task vector \(Z_i\). The sequence \((Z_1, \ldots, Z_K)\) over prior tasks is then processed by another LSTM, producing a single representation that is fed through a linear layer to output the 13 skill scores. The second component is the struggle predictor. It combines TBPP with a **BERT** encoding of the target task text and a binary vector of required programming concepts [2508.17353].

Task alignment is implemented explicitly. PTM uses **cross-attention** between the task representation and TBPP so that the model can weight which proficiencies matter most for the new task. The required concepts, together with latent skills that are always active, form a **skill weight** vector that is processed by an MLP and multiplied element-wise with TBPP to produce a task-relevant weighted TBPP. This weighted TBPP and the cross-attended task representation are concatenated and passed through a final MLP for binary struggle classification [2508.17353].

PTM is trained with a joint objective:
\[
L = a \cdot MAE(\hat{y}, y) + (1 - a) \cdot BCE(\hat{t}, t)
\]
where \(MAE(\hat{y}, y)\) is the error between predicted and target TBPP, \(BCE(\hat{t}, t)\) is the binary cross-entropy for struggle prediction, and \(a\) is a balancing hyperparameter set to \(0.5\) in the experiments. Ground-truth TBPP is estimated from a task-skill matrix \(A\) and a student score vector \(S\) using
\[
TBPP = A \cdot S .
\]
In the reported implementation, PTM is trained with Adam, hidden size 512, learning rate 0.0001, batch size 32, and 15 epochs [2508.17353].

The architectural significance of PTM lies in this coupling of longitudinal sequence modeling with explicit skill structure. The taxonomy does not merely annotate model outputs after the fact; it shapes the latent state, the auxiliary target, and the task-conditioning mechanism.

## 4. Empirical evaluation in programming education

PTM was evaluated on two introductory programming datasets. **CodeWorkout (Java)** contains approximately 630 students and 50 coding tasks in a university-level introductory programming setting with relatively uniform task structure. **FalconCode (Python)** contains approximately 1,330 students and 408 coding tasks, with a mix of small exercises and longer labs and greater variation in task order across students. A student is labeled as struggling on a task if they fail any unit test or require more attempts than 75% of their peers [2508.17353].

The evaluation protocol uses the first 30 tasks to predict struggle on the next 20 tasks, without retraining, under 5-fold cross-validation with student-level splits so that no student appears in both training and test sets. Baselines are **DKT**, **Code-DKT**, and **SAKT**, and the main evaluation metric is **ROC-AUC**, with statistical significance assessed via paired bootstrap and Bonferroni correction [2508.17353].

| Model | CodeWorkout ROC-AUC (%) | FalconCode ROC-AUC (%) |
|---|---:|---:|
| DKT | 76.27 ± 2.2 | 63.54 ± 1.8 |
| Code-DKT | 70.93 ± 4.3 | 62.17 ± 2.0 |
| SAKT | 76.11 ± 1.09 | 70.52 ± 0.8 |
| PTM | **77.09 ± 1.8** | **73.18 ± 0.9** |

Ablation studies isolate the contributions of history and taxonomy. A **No-Tax No-Hist** variant achieves **72.50 ± 3.5** on CodeWorkout and **69.13 ± 1.3** on FalconCode, while a **No-Tax** variant with history but without the taxonomy achieves **75.52 ± 1.7** and **71.33 ± 0.6** respectively. The full PTM yields the best scores on both datasets, indicating that coding history and taxonomy contribute complementary signal. Sensitivity analysis further shows that longer histories improve performance, with the best results obtained when at least 22–30 prior tasks are included [2508.17353].

The dataset-level pattern is also informative. PTM performs better on CodeWorkout than on FalconCode. The reported explanation is that CodeWorkout has more uniform task sequencing and fully online visible data, whereas FalconCode has more variable scheduling and local work that leads to missing observations for some steps [2508.17353]. This suggests that PTM benefits from dense and consistently ordered evidence when estimating longitudinal proficiency profiles.

## 5. PTM-like extensions across domains

In mathematics education, the StatusKT framework extends knowledge tracing by incorporating the student’s **problem-solving process** into the interaction sequence and extracting explicit mathematical proficiency signals. The conventional knowledge tracing sequence
\[
\mathcal{S}_{\text{conv}} = \{(q_1, c_1, r_1), \ldots, (q_t, c_t, r_t)\}
\]
is expanded to include a detailed process term \(p_t\), yielding
\[
\mathcal{S}_{\text{new}} = \{(q_1, c_1, r_1, p_1), \ldots, (q_t, c_t, r_t, p_t)\}.
\]
StatusKT then uses a **teacher-student-teacher** LLM pipeline: a teacher LLM generates problem-specific proficiency indicators tagged to CU, SC, PF, or AR; a student LLM answers those indicators from the written solution trace; and a teacher LLM evaluates whether each indicator is satisfied. Per-dimension ratios are aggregated into a proficiency vector \(\mathbf{m}_t\). On the **KT-PSP-25** dataset, which contains **22,289 student-problem pairs** with OCR-transcribed written solutions, StatusKT improves AUC and ACC for the majority of KT backbones while providing interpretable explanations grounded in explicit proficiency dimensions. The paper explicitly interprets this as a **Proficiency Taxonomy Model** in both the representational and modeling senses [2512.00311].

A different extension appears in social-media proficiency modeling. Rather than treating proficiency as expert/non-expert, that literature models proficiency as a graded topic-dependent property. Users are represented by features derived from topic query words, with evaluated alternatives including **TF**, **TF-IDF**, **User2Vec (U2V)**, **Relative User2Vec (Rel-U2V)**, and **LDA**. A scalar proficiency score is defined as
\[
\text{PS}_{u,c} = \frac{1}{|F_u|} \sum_{f_{i,u} \in F_u} f_{i,u},
\]
and the outputs can be interpreted as user-to-topic score vectors that populate a PTM. The reported findings show that U2V and Rel-U2V are superior or competitive to word-based baselines, whereas LDA is poorer and less reliable for proficiency because the induced topics often track style rather than knowledge domains [2110.08011].

Proficiency taxonomies have also been applied to generated code rather than learners. In the analysis of AI-agent Python code, the **pycefr** tool maps static language constructs to CEFR-like levels from **A1** through **C2**. Across **591 pull requests** containing **5,027 Python files** generated by three AI agents, more than **90%** of constructs fall into **A1** or **A2**, while less than **1%** are **C2**. AI-generated and human pull requests exhibit broadly similar proficiency profiles, and high-proficiency code is concentrated in feature addition and bug-fixing tasks [2604.00299]. This does not define PTM in the learner-modeling sense, but it demonstrates that taxonomy-based proficiency representations can be attached to artifacts and workflows as well as to students.

## 6. Interpretability, simulation, and terminological boundaries

One important extension of PTM-style thinking is simulation. The **PS\(^2\)** framework for student proficiency simulation does not use the PTM name, but it is explicitly **taxonomy-driven**. It constructs a lower-bound student model by fine-tuning a weak language model to exhibit **conceptual** and **procedural** errors, then interpolates between this lower-bound model and a strong upper-bound LLM using a hybrid ratio \(p \in [0,1]\). At the logit level,
\[
z_{i,t}(p) = (1-p)\, z^{l}_{i,t} + p\, z^{u}_{i,t},
\]
with token probabilities generated by a softmax over the interpolated logits. The framework reports monotonic control of behavior via a KL-divergence argument, calibration to academic performance, and empirical gains in student behavior similarity and item difficulty prediction, measured with **FID**, **MAUVE**, **Diversity KL**, **PCC**, and **SCC** [2602.00850]. A plausible implication is that PTM principles are not limited to diagnosis; they can also structure controllable simulation when proficiency levels must be synthesized rather than inferred.

Interpretability is a consistent motivation across these lines of work. In canonical PTM, the model exposes a 13-dimensional proficiency profile aligned with educator concepts. In StatusKT, indicator-level assessments are surfaced as explicit intermediate signals. In PS\(^2\), taxonomy-driven error types make the lower end of the proficiency spectrum cognitively meaningful rather than random. These systems therefore differ from black-box predictors that optimize only final accuracy [2508.17353][2512.00311][2602.00850].

A recurrent source of confusion is acronym overlap. **PTM** is also used for the **Personalized Thinking Model**, a different learner representation grounded in **Marzano’s New Taxonomy of Educational Objectives**. That PTM organizes learner journals into five layers—behavioral instances, behavioral patterns, cognitive utilization, metacognitive tendencies, and core values—and reports an overall atomic-information-point F1 of **74.57%** before human-in-the-loop refinement and **75.48%** after refinement, along with user ratings of **4.26** and **4.30** on a five-point scale [2605.04761]. Despite the shared acronym and its educational setting, this model is conceptually distinct from the Proficiency Taxonomy Model: it targets hierarchical modeling of thought and self-system structure rather than explicit proficiency-state estimation for prediction.

The literature also clarifies a common conceptual misconception: proficiency is not identical to expertise. In social-media proficiency modeling, proficiency is treated as a graded construct that includes autodidacts, hobbyists, and users with sustained practical exposure, whereas expertise imposes a stronger positive criterion [2110.08011]. That distinction is consistent with PTM-style systems, which typically assign structured, partial, and evolving proficiencies rather than forcing a binary expert classification.

Source: https://www.emergentmind.com/topics/proficiency-taxonomy-model-ptm