Transition-Aware Graph Attention Network (TGA)

• The paper introduces TGA, highlighting a novel graph attention mechanism that efficiently models multi-behavior user interactions for enhanced conversion prediction.
• TGA constructs a structured sparse graph incorporating item-level, category-level, and neighbor-level transitions, ensuring linear computational complexity even with long sequences.
• Empirical results demonstrate that TGA achieves higher AUC scores and significant speed improvements over traditional transformer-based models in e-commerce recommendation systems.

The Transition-Aware Graph Attention Network (TGA) is a sequential modeling architecture explicitly designed for multi-behavior user interaction data, particularly in the context of large-scale e-commerce recommendation systems. TGA addresses the limitations of previous transformer-based architectures by leveraging structured sparse graphs that encode diverse transition types among user behaviors, enabling both improved modeling fidelity for evolving user preferences and linear computational complexity even with long interaction sequences (Jin et al., 21 Jan 2026).

1. Motivation and Task Definition

Multi-behavior recommendation data in e-commerce includes click, add-to-cart, favorite, and purchase actions, each providing distinct intent signals. Conventional sequential models often treat behavior sequences monolithically or incur prohibitive costs when scaling to long, heterogeneous sequences. The central insight motivating TGA is that the transitions among behaviors—such as the pathway from click to purchase—convey critical contextual information not captured by raw event order alone.

The primary task addressed is the post-click conversion rate (CVR) prediction, formally specified as follows:

• Input: A user profile vector $u_p$ and a multi-behavior sequence $\mathcal{S} = \{(i_n, b_n, t_n)\}_{n=1}^N$, where $i_n$ denotes the item, $b_n$ the behavior type, $t_n$ the timestamp (with implicit position $p_n$).
• Objective: For a candidate item $c$, predict the probability $\hat y \approx y$ where $y \in \{0,1\}$ indicates conversion.
• Training Data: $\mathcal{D} = \{(u_p, \mathcal{S}, c, y)\}$.

This setup seeks to model the nuanced interplay between sequence structure and user intention (Jin et al., 21 Jan 2026).

2. Structured Sparse Graph Construction

TGA transforms the input sequence into a directed graph $G = (V, E)$ that operationalizes multi-view transitions:

• Node construction: Each interaction $(i_n, b_n)$ is encoded as a node $e_n$ with embedding

$$e_n = [\,e_n^i \oplus e_n^b \oplus e_n^t \oplus e_n^p\,] \in \mathbb{R}^{4d}$$

where $e_n^i$, $e_n^b$, $e_n^t$, $e_n^p \in \mathbb{R}^d$ represent item, behavior, timestamp, and position embeddings, respectively.

• Edges encode three specific transition perspectives:
  • Item-level transitions: Connect $(i, b_x)$ to $(i, b_y)$ if the same item occurs across two behaviors.
  • Category-level transitions: Connect $(i_x, b_x)$ to $(i_y, b_y)$ when $\mathrm{cat}(i_x) = \mathrm{cat}(i_y)$ and actions flow from $b_x$ on $i_x$ to $b_y$ on $i_y$.
  • Neighbor-level transitions: Connect temporally adjacent interactions $(i_n, b_n) \to (i_{n+1}, b_{n+1})$.
• Sparsity constraint: Each node admits at most one predecessor and successor per transition view. On average, per node: 0.56 item-edges, 1.19 category-edges, and 2.00 neighbor-edges—promoting local connectivity and tractability (Jin et al., 21 Jan 2026).

3. Transition-Aware Graph Attention Mechanism

TGA utilizes stacked graph-attention layers that are sensitive to both node features and transition types. Each layer involves two principal phases:

3.1 Behavior-Aware Edge Transformations

For each directed edge $e_l \to e_c$ corresponding to transition type $b_l \to b_c$: $$e_c^{\mathrm{in}} = W^\mathrm{in}_{b_l \to b_c} [\,e_l \oplus e_c \oplus (e_c^t - e_l^t) \oplus (e_c^p - e_l^p)\,] + b^\mathrm{in}_{b_l \to b_c}$$ Similarly, for each outgoing edge $e_c \to e_r$ of type $b_c \to b_r$: $$e_c^{\mathrm{out}} = W^\mathrm{out}_{b_c \to b_r} [\,e_c \oplus e_r \oplus (e_r^t - e_c^t) \oplus (e_r^p - e_c^p)\,] + b^\mathrm{out}_{b_c \to b_r}$$ All transformed edge representations (separated by transition view and direction) are aggregated into the local neighborhood $\mathcal{N}(e_c)$ of each node.

3.2 Multi-Head Attention and Node Update

For each attention head $k$: $$\alpha_k(u, e_c) = \frac{\exp\!\bigl((W_k^K u)^\top (W_k^Q e_c)\bigr)} {\sum_{z \in \mathcal{N}(e_c)} \exp\!\bigl((W_k^K z)^\top (W_k^Q e_c)\bigr)}$$

$$\hat e_c^{(k)} = \sum_{u \in \mathcal{N}(e_c)} \alpha_k(u, e_c) W_k^V u$$

The outputs from all heads are concatenated and projected. Residual connections, LayerNorm, and a feed-forward network (FFN) are then applied: $$e'_c = \mathrm{LayerNorm}(e_c + \hat e_c), \quad \tilde e_c = \mathrm{LayerNorm}(e'_c + \mathrm{FFN}(e'_c))$$ Stacking $L$ such layers allows information propagation along $L$-hop transition paths (Jin et al., 21 Jan 2026).

4. Computational Properties

TGA achieves linear complexity relative to the sequence length $N$, a crucial distinction from self-attention approaches. Given ≤6 neighbors per node per layer:

• Transformation cost: $O(N \cdot 6 d^2)$
• Attention cost: $O(N \cdot 6 d)$
• Total for $L$ layers: $O(N L d^2)$

In contrast, a full transformer block operates at $O(N^2 d)$ per layer, or $O(N^2 L d)$ overall. This enables TGA to scale to industrial-size sequences previously impractical for Transformer-based models (Jin et al., 21 Jan 2026).

5. Training Objective

TGA employs a binary cross-entropy loss for conversion prediction: $$\hat y = \sigma(\mathrm{MLP}(u_p, \mathcal{S}^L, c))$$

$$\mathcal{L} = -\sum_{(u_p, \mathcal{S}, c, y) \in \mathcal{D}} \Bigl[y \log \hat y + (1-y) \log (1 - \hat y)\Bigr]$$

This objective directly targets accurate post-click conversion prediction in large-scale recommendation settings (Jin et al., 21 Jan 2026).

6. Empirical Results and Comparisons

TGA has been rigorously evaluated on both public (Taobao) and industrial-scale (Taobao logs) datasets, with sequences up to 1,024 events. Offline AUC and speed results are summarized below:

Model	AUC (Taobao)	Speed	AUC (Industrial)	Speed
Transformer	0.7276	1.0×	-	-
MB-STR	0.7334	0.8×	-	-
END4Rec	0.7405	1.8×	-	-
Reformer	0.7306	1.7×	0.8623	1.0×
Linear Trans.	0.7348	1.9×	0.8619	1.0×
Longformer	0.7244	2.1×	0.8617	1.2×
TGA	0.7454	5.8×	0.8635	3.4×

• A/B Testing in a production environment yielded +1.29% post-click CVR and +1.79% GMV improvements over a strong production baseline.

These findings establish TGA as the state-of-the-art in both accuracy and computational efficiency for this task class (Jin et al., 21 Jan 2026).

7. Ablation Analysis and Model Insights

Ablation studies confirm the necessity of all three transition types:

Model Variant	AUC (Industrial)	ΔAUC
TGA (full)	0.8635	-
w/o item-level	0.8618	–0.0017
w/o category-level	0.8614	–0.0021
w/o neighbor-level	0.8625	–0.0010

Category-level transitions contribute the most, but each transition view is critical for optimal performance. Increased TGA depth consistently enhances AUC, supporting the conclusion that higher-order transition modeling substantially benefits sequence understanding.

In summary, TGA constructs a behavior-transition-aware sparse graph, applies edge-type-conditioned linear transformations, and utilizes multi-head attention over a limited set of local, diverse neighbors. This design yields both improved predictive accuracy and linear-time scalability for complex, long multi-behavior interaction sequences in modern recommendation systems (Jin et al., 21 Jan 2026).