Boundary-Aware Multi-Behavior Dynamic Graph Transformer for Sequential Recommendation

Abstract: In the landscape of contemporary recommender systems, user-item interactions are inherently dynamic and sequential, often characterized by various behaviors. Prior research has explored the modeling of user preferences through sequential interactions and the user-item interaction graph, utilizing advanced techniques such as graph neural networks and transformer-based architectures. However, these methods typically fall short in simultaneously accounting for the dynamic nature of graph topologies and the sequential pattern of interactions in user preference models. Moreover, they often fail to adequately capture the multiple user behavior boundaries during model optimization. To tackle these challenges, we introduce a boundary-aware Multi-Behavioral Dynamic Graph Transformer (MB-DGT) model that dynamically refines the graph structure to reflect the evolving patterns of user behaviors and interactions. Our model involves a transformer-based dynamic graph aggregator for user preference modeling, which assimilates the changing graph structure and the sequence of user behaviors. This integration yields a more comprehensive and dynamic representation of user preferences. For model optimization, we implement a user-specific multi-behavior loss function that delineates the interest boundaries among different behaviors, thereby enriching the personalized learning of user preferences. Comprehensive experiments across three datasets indicate that our model consistently delivers remarkable recommendation performance.

• The paper introduces MB-DGT, a transformer-based dynamic graph model that combines behavior-aware sequence attention, time-aware aggregation, and personalized score boundaries for next-item recommendation.
• MB-DGT outperforms competing methods across Yelp, Taobao, and Tmall, including a 6.50% NDCG@5 gain on Yelp and a 2.03% HR@5 improvement on Taobao.
• Ablation and interpretability analyses show that behavior-change patterns and personalized boundaries are especially valuable, while higher-order graph connectivity enables competitive results with fewer neighbor sequences.

Motivation and problem setting

Multi-behavior sequential recommendation aims to predict a user's next target-behavior interaction (typically purchase) from temporally ordered sequences of heterogeneous interactions such as clicks, favorites, cart additions, and buys. The paper identifies three shortcomings of prior work. First, sequence-centric methods model each user's own interaction stream but ignore higher-order connectivity across users and items, which hurts performance under sparse data and cold-start conditions. Second, existing multi-behavior transformers largely rely on positional encodings and do not directly account for the effect of time intervals on behavior-aware collaboration, even though shorter intervals plausibly indicate stronger collaborative signals; neglecting time spans can introduce noise when user interests shift. Third, standard point-wise or pair-wise losses apply a uniform decision threshold to all users and behaviors, whereas users with different interests have distinct interest boundaries between positive and negative items.

The proposed MB-DGT model addresses these issues by combining a transformer-based dynamic graph aggregator over multi-behavior temporal interaction graphs with a personalized, boundary-aware loss function.

Model architecture

MB-DGT operates on $L$-hop multi-behavior temporal interaction graphs derived from continuous-time dynamic graphs, constructed in two variants: one centered on users (user-item-user trajectories) and one on items (item-user-item trajectories). The architecture comprises three modules stacked $L$ times.

Transformer-based Multi-behavior Correlation Modeling (TMCM): neighbor interaction sequences are encoded with behavior-conditioned self-attention. Behavior one-hot vectors modulate the query/key/value projections, behavior-pair-specific attention matrices $W^{att}_{(b_i,b_j)}$ and aggregation matrices $W^{agg}_{(b_i,b_j)}$ produce distinct attention scores per behavior pair, and behavior-specific MLPs yield the final representations. Sequential structure is injected through a multi-behavior relative positional encoding that pairs the relative distance $(j-i)$ with the behavior types at both positions. Following T5-style bucketing, short relative distances receive independent encodings while long distances share logarithmically spaced buckets, which balances the number of training examples per relative position.

Time-Aware Aggregation (TAA): temporal information is encoded via the TGAT-style random Fourier feature mapping $\Phi(t)$, whose inner product approximates a translation-invariant kernel of the time span $|t_2 - t_1|$, making it compatible with dot-product attention. Neighbor representations are concatenated with behavior-edge embeddings and time encodings, then aggregated into the target node through temporal graph attention with residual connections and layer normalization. The authors use $L=2$ hops, arguing that 1-hop graphs miss higher-order associations while 3+ hops incur exponential edge growth.

Boundary-Aware Personalized Prediction (BAPP): an auxiliary score $b_{u,k}(t)$ is computed per user, behavior, and timestamp from the user embedding, behavior-edge embeddings, and learned inter-behavioral dependency matrices. The point-wise loss penalizes positives scoring below the boundary and negatives scoring above it:

$$\mathcal{L} = - \sum_{(u,p)} \ln\sigma(s(u,p) - b_{u,k}(t)) - \alpha \sum_{(u,n)} \ln\sigma(1 - \sigma(s(u,n) - b_{u,k}(t)))$$

where $L$0 controls the balance between positive and negative score distributions. The reported time complexity is $L$1, dominated by the transformer over neighbor sequences.

Experimental results

Experiments cover three datasets — Yelp (19,800 users, four rating-derived behaviors), Taobao (147,894 users), and Tmall (31,882 users) — evaluated with HR@N and NDCG@N under leave-one-out protocol with 99 sampled negatives. MB-DGT outperforms all baselines, including strong multi-behavior sequential models MBHT and MB-STR, on every metric and dataset. Representative improvements over the best baseline include:

Dataset	Metric	Best baseline	MB-DGT	Improvement
Yelp	NDCG@5	0.569	0.606	6.50%
Yelp	NDCG@10	0.610	0.645	5.74%
Tmall	NDCG@5	0.547	0.569	4.02%
Taobao	HR@5	0.691	0.705	2.03%

Ablations show that removing TMCM degrades performance more than removing TAA on two of three datasets, suggesting behavior-change patterns within sequences matter more than time-interval modeling in most cases; removing BAPP consistently hurts, supporting the value of personalized boundaries. Auxiliary-behavior ablations indicate that dropping click/view data causes the largest degradation, consistent with its high frequency and correlation with purchase.

Two analyses support the interpretability claims. Learned boundaries $L$2 form distinct distributions per behavior type across 1,000 Taobao users, confirming that the model separates interest thresholds by behavior rather than collapsing them. Attention-score visualizations reveal plausible hierarchical dependencies — e.g., "buy" correlates strongly with page views and cart additions but weakly with favorites on both e-commerce datasets. Hyperparameter analysis shows moderate sensitivity to embedding dimension and neighbor count, and the authors report a notable efficiency claim: because higher-order graph connectivity supplies collaborative signal, MB-DGT matches or exceeds competitors' performance with as few as 10 neighbor sequences where comparable models typically require around 50, which is relevant for cold-start regimes.

Limitations and open questions

Several caveats bear directly on the results. The evaluation uses a single held-out positive per user ranked against 99 randomly sampled negatives, a protocol known to inflate absolute metrics and potentially favor models with strong score-separation properties such as the boundary-aware loss; full-ranking evaluation is not reported. The choice of exactly 2 hops is justified by computational cost rather than empirically validated against deeper alternatives. The boundary mechanism introduces hyperparameter $L$3, whose optimal value varies by dataset, and the paper does not provide guidance for selecting it beyond grid search. The claimed cold-start advantage is asserted through reduced neighbor requirements rather than a dedicated cold-start benchmark. Finally, the authors acknowledge that training efficiency limits how dynamically interests can be captured, and they leave open whether the framework generalizes to sparser datasets and whether its interpretability holds across domains beyond those tested.

Conclusion

MB-DGT unifies behavior-conditioned sequence encoding, time-span-aware dynamic graph aggregation, and a learnable per-user, per-behavior score boundary in a single multi-behavior sequential recommendation framework. It achieves consistent state-of-the-art results across three public datasets, with the largest gains in ranking quality (up to 6.50% NDCG@5 improvement on Yelp), and provides evidence that explicit interest-boundary modeling and higher-order temporal graph structure each contribute measurably to performance.