SPAR: Personalized Content-Based Recommendation via Long Engagement Attention (2402.10555v2)

Abstract: Leveraging users' long engagement histories is essential for personalized content recommendations. The success of pretrained LLMs (PLMs) in NLP has led to their use in encoding user histories and candidate items, framing content recommendations as textual semantic matching tasks. However, existing works still struggle with processing very long user historical text and insufficient user-item interaction. In this paper, we introduce a content-based recommendation framework, SPAR, which effectively tackles the challenges of holistic user interest extraction from the long user engagement history. It achieves so by leveraging PLM, poly-attention layers and attention sparsity mechanisms to encode user's history in a session-based manner. The user and item side features are sufficiently fused for engagement prediction while maintaining standalone representations for both sides, which is efficient for practical model deployment. Moreover, we enhance user profiling by exploiting LLM to extract global interests from user engagement history. Extensive experiments on two benchmark datasets demonstrate that our framework outperforms existing state-of-the-art (SoTA) methods.

Leveraging Long Engagement Histories for Content-based Recommendations: The SPAR Framework

Introduction to SPAR

In the burgeoning domain of digital content, personalized recommendation systems are paramount for enhancing user experience. A novel framework, SPAR (Sparse Poly-Attention for content Recommendation), has been introduced to address the challenges of extracting holistic user interests from extensive engagement histories efficiently. This framework is distinguished by its ability to fuse long user engagement histories and candidate items for prediction while maintaining independent representations for both, thereby facilitating efficient model deployment in practical settings.

Key Contributions and Methodology

SPAR leverages pretrained LLMs (PLMs), poly-attention layers, and attention sparsity mechanisms. It encodes user history in a session-based manner, allowing for efficient and comprehensive user interest extraction over long text sequences, which is a notable enhancement over existing methods.

Main Contributions:

1. SPAR Framework: Incorporation of multiple poly-attention layers and sparse attention mechanisms allows for hierarchical fusion of token-level embeddings of session-based user history texts, significantly outperforming state-of-the-art methods on benchmark datasets.
2. Independent Representations: Unlike early fusion methods, SPAR ensures standalone user and candidate item representations, facilitating lightweight retrieval and ranking stages in content-based recommendation systems.
3. LLM Utilization for User Profiling: Leveraging LLMs for extracting global interests from user engagement history enhances user profiling, further boosting the recommendation performance.

Methodology Highlights:

• Session-Based Encoding: By grouping user engagement history into subsequences, SPAR addresses the computational overhead associated with encoding long sequences.
• Poly-Attention Mechanism: The employment of poly-attention for both user history summarizing (UHS) and candidate content summarizing (CCS) layers aids in distilling comprehensive representations from long engagement histories and rich candidate item information.
• Sparse Attention Strategy: Incorporating local sliding-window attention, global attention, and random attention within the poly-attention module enables a balanced focus on both local context and overall engagement history.

Empirical Validation

SPAR's efficacy was rigorously tested against benchmark datasets like MIND (for news recommendation) and Goodreads (for book recommendation), where it demonstrably outperformed existing models across various metrics, including AUC, MRR, and nDCG. These results exemplify SPAR’s superiority in capturing user interests and predicting engagement more accurately.

Theoretical Implications and Future Directions

The introduction of SPAR to the field of content-based recommendations not only advances our understanding of engaging long user histories but also opens avenues for future research, particularly in optimizing the attention mechanisms for even longer sequences and exploring the fusion of multimodal content representations.

Concluding Thoughts

SPAR represents a significant step forward in content-based recommendation systems, especially in processing extensive user engagement histories. Its ability to maintain standalone representations for users and items while ensuring comprehensive interaction between them makes it a valuable asset for practical deployment in real-world scenarios. Moving forward, the adaptability of the SPAR framework to newer, more efficient PLMs and its application to other domains beyond news and books are promising pathways for further exploration and development in personalized content recommendations.