Contrastive Self-supervised Sequential Recommendation with Robust Augmentation (2108.06479v1)

Abstract: Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data. At their core, such approaches model transition probabilities between items in a sequence, whether through Markov chains, recurrent networks, or more recently, Transformers. However both old and new issues remain, including data-sparsity and noisy data; such issues can impair the performance, especially in complex, parameter-hungry models. In this paper, we investigate the application of contrastive Self-Supervised Learning (SSL) to the sequential recommendation, as a way to alleviate some of these issues. Contrastive SSL constructs augmentations from unlabelled instances, where agreements among positive pairs are maximized. It is challenging to devise a contrastive SSL framework for a sequential recommendation, due to its discrete nature, correlations among items, and skewness of length distributions. To this end, we propose a novel framework, Contrastive Self-supervised Learning for sequential Recommendation (CoSeRec). We introduce two informative augmentation operators leveraging item correlations to create high-quality views for contrastive learning. Experimental results on three real-world datasets demonstrate the effectiveness of the proposed method on improving model performance and the robustness against sparse and noisy data. Our implementation is available online at \url{https://github.com/YChen1993/CoSeRec}

Contrastive Self-supervised Sequential Recommendation with Robust Augmentation

The paper presents a sophisticated exploration into the domain of sequential recommendation systems, focusing on addressing prevalent issues within such models through the application of contrastive self-supervised learning (SSL). Sequential recommendation systems have continually evolved, leveraging various modeling techniques such as Markov chains, RNNs, and Transformers, with an emphasis on modeling dynamic user behavior whereby the goal is to predict future user-item interactions based on past interactions. However, these systems face chronic issues related to data sparsity and the inclusion of noisy interactions that may impair their predictive capabilities.

Proposed Framework: CoSeRec

The paper proposes a novel framework named Contrastive Self-supervised Learning for Sequential Recommendation (CoSeRec). This framework exploits contrastive SSL to alleviate certain inherent issues in sequential recommendation systems. The cornerstone of this approach is the development of new data augmentation methodologies to create high-quality sequence views for contrastive learning.

1. Augmentation Strategies:
   • Random Augmentations: Existing methods such as 'Crop', 'Mask', and 'Reorder' are briefly retained. However, these random strategies may disrupt item correlations and are particularly detrimental to short sequential data.
   • Informative Augmentations: Two new strategies are introduced – 'Substitute' and 'Insert'. These leverage inherent item relationships to maintain sequential integrity and provide robust sequence views without negating item correlations.
2. Contrastive SSL Objective:
   • The contrastive SSL objective seeks to maximize the agreement between positive pairs of augmented views derived from the same sequence. It promotes the learning of representations that best capture the signal present in the data, even when trained with unlabelled data.
3. Multi-task Training:
   • The model adopts a multi-task training paradigm, simultaneously optimizing the objectives of sequential recommendation and contrastive SSL. This synergy harnesses the SSL signal to potentially impermeate the recommendation task with enhanced predictive performance.

Experimental Findings

The proposed CoSeRec framework exhibits remarkable performance across three real-world datasets: Beauty, Sports, and Yelp. The framework's efficacy is evidenced by substantial improvement in Hit Ratio and NDCG metrics compared to traditional non-sequential models, established sequential models like SASRec and BERT4Rec, and other contemporary models including GRU4Rec and Caser.

• Data Sparsity and Noisy Interaction Robustness:

CoSeRec demonstrates robustness against sparsity by significantly outperforming baseline models even when trained with less than the full dataset. Furthermore, it maintains superior performance when exposed to added noise in validation, affirming its resilience and reliability.

• Impact of Augmentation:

The introduction of informative augmentations markedly enhances the model's ability to form meaningful positive sample pairs for contrastive learning. These augmentations outperform random methods in both effectiveness and robustness, reflecting their critical role in CoSeRec’s superior performance.

Implications and Future Work

This paper provides a strong impetus towards the adoption of contrastive SSL in enhancing sequential recommendation systems. By focusing on maintaining item correlations and intelligently managing sequence length through informed augmentations, it opens pathways for more refined and versatile applications in domains where sequence data is abundant yet variably informative.

Future research directions could include exploring dynamic item correlation in real-time, potentially integrated with reinforcement learning frameworks to actively learn and adjust correlations. Additionally, further refinement of augmentation strategies based on specific sequence characteristics could provide deeper insights into the best practices for different types of sequential data within varying contexts.

Overall, the paper contributes significantly to the theoretical foundations of modern recommendation systems and paves the way for practical enhancements that could broaden the scope and efficacy of such systems in serving diverse user needs.