---
title: Dual Graph Attention-based Knowledge Tracing
url: https://www.emergentmind.com/topics/dual-graph-attention-based-knowledge-tracing-dgakt
type: topic
---

# Dual Graph Attention-based Knowledge Tracing

to=arxiv_search  彩神争霸app 开号地址json
{"query":"2211.12881 DGEKT: A Dual Graph Ensemble Learning Method for Knowledge Tracing", "max_results": 5}
Dual Graph Attention-based Knowledge Tracing (DGAKT) is a graph neural network model for knowledge tracing that predicts whether a student will answer a target exercise correctly by operating on a per-interaction subgraph extracted from an integrated student–exercise–knowledge concept graph. In the formulation reported for DGAKT, the model is designed to leverage higher-order information from subgraphs representing student-exercise-KC relationships, combining local neighbor attention with global subgraph attention while avoiding explicit recurrent or transformer sequence models. The approach is motivated by the need to exploit multi-relational educational structure without incurring the computational cost associated with full global graphs and long learning sequences [2507.18668].

## 1. Conceptual position within knowledge tracing

Knowledge tracing (KT) seeks to predict the probability of a correct response to a future exercise from a student’s prior interactions. In DGAKT, a student \(s\) has an interaction sequence \(\{(q_1,r_1,ts_1),\dots,(q_t,r_t,ts_t)\}\), and the prediction target is
\[
\hat{r}_\tau=\Pr(r_\tau=1\mid \text{history up to }\tau-1,q_\tau).
\]
Rather than modeling this history with an RNN or Transformer, DGAKT constructs a subgraph for each target interaction \((s,q_\tau)\) and learns predictive representations directly from graph structure and edge features [2507.18668].

A central feature of DGAKT is its integrated graph design. The model combines student–exercise interactions and exercise–KC relations in a single heterogeneous graph \(G=(V,E,R)\), where \(V=S\cup Q\cup C\), \(E=E(S,Q)\cup E(Q,C)\), and \(R=\{r_{SQ},r_{QC}\}\). Student–exercise edges encode observed interactions with timestamps and correctness, while exercise–KC edges encode exercise membership in knowledge concepts. The paper states that each KC node represents a hyperedge over exercises, following a conversion in which hyperedges are turned into KC nodes connected to their incident exercises, so that message passing can learn KC embeddings instead of merely using hyperedges as paths [2507.18668].

This integrated formulation places DGAKT at the intersection of graph-based KT and inductive subgraph learning. It differs from sequence-centric KT architectures such as DKT, AKT, SAKT, and SAINT, and it also differs from global-graph KT systems because it processes only the subgraph relevant to the target interaction. This suggests that DGAKT’s defining claim is not merely graph augmentation, but a specific coupling of higher-order relational learning and computationally bounded inference [2507.18668].

## 2. Integrated graph construction and subgraph extraction

DGAKT operates on a heterogeneous graph with two relation types. The adjacency matrices are \(A_{SQ}\in\{0,1\}^{|S|\times |Q|}\), where \(A_{SQ}(s,q)=1\) iff \((s,q)\in E(S,Q)\), and \(A_{QC}\in\{0,1\}^{|Q|\times |C|}\), where \(A_{QC}(q,c)=1\) iff \((q,c)\in E(Q,C)\). The heterogeneous adjacency \(A_H\) is the block matrix over students, exercises, and KCs, using \(A_{SQ}\) and \(A_{QC}\) in the cross-type blocks, together with symmetric counterparts if the graph is treated as undirected in local attention [2507.18668].

For each target interaction \((s,q_\tau,ts_\tau)\), DGAKT constructs a subgraph \(G_{sub}\) consisting of three node sets: exercises in the target student’s subsequence window \(W(s)\) of fixed length; students who have interacted with the target exercise \(q_\tau\); and all KCs connected to the exercises in the subsequence via \(E(Q,C)\). All edges among the selected nodes are included except the target interaction edge \((s,q_\tau)\), which is removed to prevent label leakage. The resulting subgraph mixes student–exercise and exercise–KC relations in a compact local context around the prediction target [2507.18668].

The model applies an IGMC-style labeling trick to initialize node features. The target student and target exercise are labeled as 0 and 1, neighboring students and exercises as 2 and 3, and the two types of KCs related to the exercises as 4 and 5. Node features are one-hot vectors over these labels. The paper states that these structural features generalize inductively. Local edge features are defined only on student–exercise edges and comprise a 3-dimensional vector: timestamp proximity, number of previous interactions capped at 128, and target-side response indicator. Timestamp normalization is given by
\[
\begin{aligned}
ts_{\text{abs}} &= |ts_{\tau}-ts|\\
ts_{\text{norm}} &= 1-\frac{ts_{\text{abs}}-\min(ts_{\text{abs}})}{\max(ts_{\text{abs}})-\min(ts_{\text{abs}})}.
\end{aligned}
\]
For exercise–KC edges, edge features are zero-padded, while node-type features are used in global attention [2507.18668].

The subgraph construction is central to DGAKT’s efficiency claim. Because the model processes only \(V_{sub}\) and \(E_{sub}\) per target interaction, it avoids the full-graph dependence characteristic of global graph KT systems. A plausible implication is that DGAKT’s inductive capability and efficiency arise from the same design decision: the use of local, labeled, target-conditioned subgraphs rather than a single transductive graph representation.

## 3. Dual attention architecture

The “dual” in DGAKT refers to a dual attention design that combines local neighbor attention with global subgraph attention. The model alternates these two components for \(L\) layers, with \(L=2\) in the reported implementation. Formally,
\[
\begin{aligned}
H^l &= \Psi^l(\mathcal{G}_{sub},H^{l-1},F)\\
h_g^l &= \Phi^l(\mathcal{G}_{sub},H^l,M),
\end{aligned}
\]
where \(H^l\) are node embeddings, \(F\) are local edge features, \(M\) are one-hot node-type features, and \(h_g^l\) is the subgraph embedding [2507.18668].

The local component is an edge-featured graph attention layer. For each edge \((i,j)\in E_{sub}\), attention logits are computed by concatenating transformed source-node, neighbor-node, and edge features:
\[
\epsilon(h_i,h_j,f_{ij})=\text{LeakyReLU}\big(\text{attn}([W_i h_i\ \|\ W_j h_j\ \|\ W_f f_{ij}])\big).
\]
The normalized coefficients are
\[
\alpha_{ij}=\frac{\exp(\epsilon(h_i,h_j,f_{ij}))}{\sum_{j'\in\mathcal{N}_i}\exp(\epsilon(h_i,h_{j'},f_{ij'}))},
\]
and multi-head aggregation yields
\[
h'_i=\text{ELU}\left(W_{\text{local}\ \Vert_{k=1}^K \sum_{j\in \mathcal{N}_i}\alpha^k_{ij}W_\Psi h_j}\right).
\]
This layer allows temporal proximity, cumulative activity, and response context to modulate neighbor influence on student–exercise edges [2507.18668].

The global component introduces a virtual subgraph node \(g\). All nodes \(j\in V_{sub}\) are connected to \(g\) through directed edges \(j\to g\), each carrying a node-type feature \(m_j\). Attention weights to the virtual node are
\[
\beta_j=\frac{\exp(\epsilon(h_g,h_j,m_j))}{\sum_{j'\in\mathcal{N}_g}\exp(\epsilon(h_g,h_{j'},m_{j'}))},
\]
and the global embedding is
\[
h'_g=\text{ELU}\left(W_{\text{global}\ \Vert_{k=1}^K \sum_{j\in \mathcal{N}_g}\beta^k_j W_\Phi h_j}\right).
\]
The paper characterizes this layer as a way to attend across the entire subgraph in one hop, thereby complementing local multi-hop propagation and mitigating over-smoothing [2507.18668].

This architecture encodes higher-order relationships in two ways. Stacked local layers propagate signals along paths such as \(S\to Q\to C\) and \(S\to Q\to S'\), while global attention lets indirectly connected nodes influence the subgraph representation in a single step. A frequent simplification is to describe DGAKT as merely a graph-attention analogue of sequential KT. The reported design is more specific: DGAKT does not use RNN/Transformer sequence modeling, and its temporal dynamics are injected through edge features rather than token-to-token temporal attention [2507.18668].

## 4. Representation fusion, prediction, and learning objective

After stacking \([\,\text{local}\rightarrow \text{global}\,]\) layers, DGAKT concatenates intermediate representations across layers. The subgraph-level representation is
\[
x_g=\text{concat}(h_g^1,h_g^2,\dots,h_g^L),
\]
and the target-specific representation is
\[
x_\tau=\text{concat}(h_s^1,\dots,h_s^L,\ h_e^1,\dots,h_e^L),
\]
where \(h_s^l\) and \(h_e^l\) are the target student and target exercise embeddings at layer \(l\) [2507.18668].

Prediction is performed by two MLP heads with sigmoid output. One head uses the global subgraph representation and the other uses the target node-pair representation:
\[
\begin{aligned}
\hat{r}_\Phi &= \sigma(\mathcal{W}_\Phi^1(\mathcal{W}_\Phi^0 x_g))\\
\hat{r}_\Psi &= \sigma(\mathcal{W}_\Psi^1(\mathcal{W}_\Psi^0 x_\tau))\\
\hat{r} &= \gamma \hat{r}_\Phi + (1-\gamma)\hat{r}_\Psi.
\end{aligned}
\]
The mixing coefficient \(\gamma\in[0,1]\) controls the contribution of global and local-target perspectives [2507.18668].

Training combines binary cross-entropy losses for both heads with a consistency regularizer:
\[
\mathcal{L}_{\Phi} = -\frac{1}{N}\sum_{i=0}^{N}\big(r_i\log(\hat{r}_{\Phi i})+(1-r_i)\log(1-\hat{r}_{\Phi i})\big),
\]
\[
\mathcal{L}_{\Psi} = -\frac{1}{N}\sum_{i=0}^{N}\big(r_i\log(\hat{r}_{\Psi i})+(1-r_i)\log(1-\hat{r}_{\Psi i})\big),
\]
\[
\mathcal{L}_{cons} = \sqrt{\frac{1}{N}\sum_{i=1}^{N}(\hat{r}_{\Phi i}-\hat{r}_{\Psi i})^2},
\]
\[
\mathcal{L} = \gamma \mathcal{L}_{\Phi} + (1-\gamma)\mathcal{L}_{\Psi} + \lambda \mathcal{L}_{cons}.
\]
Optimization uses Adam, and the activation function is ELU [2507.18668].

The two-head formulation reflects DGAKT’s division between “global higher-order” and “local target-pair” evidence. This suggests that the model is not attempting to collapse all predictive information into a single embedding. Instead, it maintains two predictive views and explicitly regularizes them toward agreement. In the reported framework, temporal modeling is achieved by reweighting graph messages with edge features rather than by storing hidden states over sequence positions [2507.18668].

## 5. Empirical results, robustness, and efficiency

DGAKT was evaluated on EdNet, ASSISTments 2017, and Junyi, with chronological 60/20/20 train/validation/test splits. The reported datasets contain 117,345 students, 13,517 exercises, 188 skills, and approximately 30M interactions for EdNet; 1,709 students, 3,162 exercises, 102 skills, and 942,816 interactions for ASSISTments 2017; and 162,303 students, 722 exercises, 49 skills, and 3M interactions for Junyi. Evaluation uses AUC and ACC [2507.18668].

| Dataset | DGAKT result | Reported gain over best baseline |
|---|---:|---:|
| EdNet | ACC 0.7630, AUC 0.8188 | 3.26% ACC, 3.95% AUC |
| ASSIST2017 | ACC 0.8151, AUC 0.8994 | 10.83% ACC, 11.22% AUC |
| Junyi | ACC 0.8576, AUC 0.9122 | 0.85% ACC, 3.57% AUC |

The paper reports that DGAKT consistently outperforms all baselines, including RNN-based models such as DKT and DKVMN, attention-based models such as AKT, SAKT, and SAINT, and graph-based models such as GKT, IGMC, PEBG, DGEKT, and IGMC-KC. The reported improvements over the best baselines are statistically significant with \(p<0.01\) using t-tests [2507.18668].

Ablation studies attribute performance gains to all major components. Removing local attention, removing global attention, or removing KCs degrades performance across datasets. Removing timestamp, previous-interaction count, or response features from student–exercise edges also hurts performance. The paper additionally reports that IGMC-KC is often the second-best baseline, which is presented as evidence that integrated student–exercise and exercise–KC structure is itself important, while DGAKT’s dual attention and subgraph design yield further gains [2507.18668].

The model is also evaluated on held-out exercise-type and KC generalization settings. DGAKT outperforms IGMC-KC across ASSIST2017_A/B/C and EdNet_A/B/C splits; for example, on ASSIST2017_A/B/C the reported DGAKT results are ACC 0.8708/0.8610/0.8557 and AUC 0.9480/0.9444/0.9394, compared with IGMC-KC ACC 0.8132/0.7758/0.7954 and AUC 0.8442/0.8445/0.8363. On EdNet_A/B/C, DGAKT reports ACC 0.7716/0.7225/0.7631 and AUC 0.8015/0.7757/0.7886, compared with IGMC-KC ACC 0.7657/0.7149/0.7576 and AUC 0.7949/0.7608/0.7755 [2507.18668].

In complexity analysis, DGAKT has time complexity \(O(n\cdot d^2+n^2\cdot d)\) and space complexity \(O(n^2+n\cdot d+d^2)\), where \(n\) is subgraph size and \(d\) is hidden dimension. The \(n^2\) term is attributed to attention normalization over subgraph neighborhoods, and the \(n\cdot d^2\) term to linear projections. The model is reported to have about 60k parameters, and the paper emphasizes reduced GPU memory and runtime relative to global-graph training and inference. This efficiency claim is integral to DGAKT’s identity: the method is presented not only as more accurate, but as addressing resource efficiency that prior KT methods had largely overlooked [2507.18668].

## 6. Relation to DGEKT, interpretive clarifications, and limitations

DGAKT is closely related to, but distinct from, the earlier “DGEKT: A Dual Graph Ensemble Learning Method for Knowledge Tracing” [2211.12881]. DGEKT builds two complementary graphs over interaction nodes: a hypergraph for heterogeneous exercise–concept associations and a directed graph for interaction transitions. It then learns separate graph representations and combines them through online knowledge distillation, where an adaptive teacher provides predictions on all exercises as extra supervision. In experiments against eight baselines on ASSIST09, ASSIST17, and EdNet, DGEKT is reported to achieve state-of-the-art performance, with headline AUCs of 76.56 on ASSIST09, 77.54 on ASSIST17, and 70.07 on EdNet [2211.12881].

The relationship between the two methods is conceptually important because the phrase “dual graph” can be interpreted in different ways. In DGEKT, “dual” refers to two separate relational structures—a concept-association hypergraph and a transition directed graph—combined through ensemble learning and online knowledge distillation. In DGAKT, by contrast, the reported model integrates student–exercise and exercise–KC relations into a single graph and uses a dual attention design consisting of local neighbor attention and global subgraph attention [2507.18668]. A common misconception is therefore to treat DGAKT as a direct renaming of DGEKT with attention added. The available descriptions indicate a more substantive architectural shift: DGAKT replaces global dual-graph encoding plus recurrent sequence modeling with inductive subgraph extraction, edge-featured local EGAT, global attention to a virtual subgraph node, and dual prediction heads.

The data also support several limitations. DGAKT depends on the quality and completeness of exercise–KC tagging; noisy or missing KC links weaken higher-order signals. Local edge features are defined only on student–exercise edges, leaving exercise–KC edges without time-aware attributes. Fixed subgraph construction may omit informative neighbors, such as other exercises attempted by peer students. The model captures temporal proximity through edge features, but it does not explicitly model long-term forgetting beyond the cap on previous interactions. The paper identifies adaptive subgraph expansion or sampling, extension of temporal features to other relation types, dynamic graph updates, and integration with LLMs as future directions [2507.18668].

Interpretability is one of the reported by-products of the architecture. Global attention weights over subgraph nodes can highlight influential exercises and KCs, and case studies on Junyi are said to show that high-attention nodes often align with prerequisite structures even though prerequisites are not explicitly provided [2507.18668]. This suggests that DGAKT’s global attention mechanism functions not only as an aggregator but also as a lens on which higher-order educational relations the model considers predictive.

In summary, DGAKT occupies a specific position within graph-based KT: it is an inductive, subgraph-based, attention-driven model that integrates student–exercise and exercise–KC information, captures higher-order signals through stacked local and global attention, and couples predictive performance with explicit efficiency objectives. Relative to prior graph KT approaches, including DGEKT, its main distinctive elements are the integrated graph, per-target subgraph inference, and the dual attention mechanism that replaces both global-graph transductive processing and explicit sequential encoders [2507.18668].

Source: https://www.emergentmind.com/topics/dual-graph-attention-based-knowledge-tracing-dgakt