Multi-Grained Preference Enhanced Transformer for Multi-Behavior Sequential Recommendation (2411.12179v2)

Abstract: Sequential recommendation (SR) aims to predict the next purchasing item according to users' dynamic preference learned from their historical user-item interactions. To improve the performance of recommendation, learning dynamic heterogeneous cross-type behavior dependencies is indispensable for recommender system. However, there still exists some challenges in Multi-Behavior Sequential Recommendation (MBSR). On the one hand, existing methods only model heterogeneous multi-behavior dependencies at behavior-level or item-level, and modelling interaction-level dependencies is still a challenge. On the other hand, the dynamic multi-grained behavior-aware preference is hard to capture in interaction sequences, which reflects interaction-aware sequential pattern. To tackle these challenges, we propose a Multi-Grained Preference enhanced Transformer framework (M-GPT). First, M-GPT constructs a interaction-level graph of historical cross-typed interactions in a sequence. Then graph convolution is performed to derive interaction-level multi-behavior dependency representation repeatedly, in which the complex correlation between historical cross-typed interactions at specific orders can be well learned. Secondly, a novel multi-scale transformer architecture equipped with multi-grained user preference extraction is proposed to encode the interaction-aware sequential pattern enhanced by capturing temporal behavior-aware multi-grained preference . Experiments on the real-world datasets indicate that our method M-GPT consistently outperforms various state-of-the-art recommendation methods.

• The paper introduces M-GPT, a novel transformer framework that captures interaction-level dependencies and multi-grained user preferences to boost recommendation accuracy.
• It leverages an Interaction-Level Dependency Extractor and a Multifaceted Sequential Pattern Generator to model both fine-grained and aggregated user behaviors.
• Experimental results on real-world datasets demonstrate significant improvements in HR, NDCG, and MRR, confirming the model’s practical efficacy in recommendation tasks.

Analyzing Multi-Grained Preference Enhanced Transformer for Multi-Behavior Sequential Recommendation

The paper "Multi-Grained Preference Enhanced Transformer for Multi-Behavior Sequential Recommendation" introduces an innovative approach aimed at advancing sequential recommendation methodologies through an effective processing of user-item interaction data. In the field of recommendation systems, the capability to predict users' next interactions based on their historical behaviors is a vital yet complex task, fundamentally due to the heterogeneity and multi-grained nature of these interactions. This challenge has been the focal point of research within multi-behavior sequential recommendation (MBSR).

Summary of Approach

The authors present the Multi-Grained Preference Enhanced Transformer (M-GPT), a novel framework designed to address two primary challenges within MBSR: learning interaction-level multi-behavior dependencies, and effectively capturing dynamic behavior-aware multi-grained preferences. The framework incorporates two key components: the Interaction-Level Dependency Extractor (IDE) and the Multifaceted Sequential Pattern Generator (MSPG).

1. Interaction-Level Dependency Extractor (IDE):
   • Construction of Interaction Graphs: The approach constructs a comprehensive interaction graph at the interaction level, synthesizing dependencies spanning item-level and behavior-level insights. The influence of different historical interactions is captured through a parameterized adjacency matrix, which utilizes both item-specific and behavior-specific semantics.
   • Graph Convolution: The use of graph convolution allows for the extraction of complex interaction dependencies over various order levels. This process further leverages graph neural network principles to quantify these dependencies and their potential impact on subsequent recommendations.
2. Multifaceted Sequential Pattern Generator (MSPG):
   • Transformer with Linear Self-Attention: Integrating a refined transformer model, the MSPG utilizes a self-attention mechanism to efficiently capture sequential patterns. By reducing the computational complexity typically associated with full dot-product attention, it ensures scalable modeling of user sequences.
   • Multi-Grained User Preference Extraction: This component divides interaction sequences into segmented sessions with varying temporal granularity. Consequently, it extracts multifaceted user preferences using a multi-grained multi-head self-attention (MGMHSA), effectively capturing both short- and long-term user interests.

Experimental Evaluation and Implications

The researchers conducted extensive empirical studies using real-world datasets — Taobao, IJCAI, and Retailrocket. The results reveal M-GPT's superior performance when benchmarked against state-of-the-art recommendation models. Specifically, the approach consistently demonstrated improvements in Hit Rate (HR), Normalized Discounted Cumulative Gain (NDCG), and Mean Reciprocal Rank (MRR), reflecting its efficacy in sorting relevant recommendations from user interaction sequences.

Practical and Theoretical Implications

Practically, the M-GPT model's capability to capture nuanced user preferences from heterogeneous behaviors means that it can be applied widely across e-commerce platforms and social media to enhance user satisfaction and engagement. Theoretically, it underscores the necessity of modeling both interaction-level dependencies and dynamic user preferences within recommendation systems, offering further directions for research on integrating multi-behavioral data in user behavior prediction.

Future Directions

Future developments might explore incorporating even more sophisticated neural architectures, potentially integrating the nuanced temporal dynamics of interactions through techniques such as contrastive learning or reinforcement learning. Furthermore, extending the framework could involve improving interpretability to better track and explain recommendation rationale, ensuring robust real-world applicability.

In summary, this paper contributes significantly to the field of recommender systems by proposing a comprehensive solution to the dynamic and multifaceted nature of user-item interactions. Through M-GPT, the authors offer a pathway to enhancing predictive accuracy in multi-behavior sequential recommendation tasks, representing a meaningful advancement for both academic inquiry and practical application.