Self-Supervised Learning for Recommender Systems: A Survey (2203.15876v2)

Abstract: In recent years, neural architecture-based recommender systems have achieved tremendous success, but they still fall short of expectation when dealing with highly sparse data. Self-supervised learning (SSL), as an emerging technique for learning from unlabeled data, has attracted considerable attention as a potential solution to this issue. This survey paper presents a systematic and timely review of research efforts on self-supervised recommendation (SSR). Specifically, we propose an exclusive definition of SSR, on top of which we develop a comprehensive taxonomy to divide existing SSR methods into four categories: contrastive, generative, predictive, and hybrid. For each category, we elucidate its concept and formulation, the involved methods, as well as its pros and cons. Furthermore, to facilitate empirical comparison, we release an open-source library SELFRec (https://github.com/Coder-Yu/SELFRec), which incorporates a wide range of SSR models and benchmark datasets. Through rigorous experiments using this library, we derive and report some significant findings regarding the selection of self-supervised signals for enhancing recommendation. Finally, we shed light on the limitations in the current research and outline the future research directions.

Self-Supervised Learning for Recommender Systems: A Survey

The paper "Self-Supervised Learning for Recommender Systems: A Survey" presents an exhaustive review of utilizing self-supervised learning (SSL) in recommendation systems. With the growing interest in leveraging SSL techniques, this survey provides a structured examination of key contributions, categorizing existing methods, and identifying challenges and future trajectories within the field.

Overview

Recommender systems are pivotal in modern E-commerce platforms. However, the inherent data sparsity problem hampers their performance due to the limited user-item interactions. SSL emerges as a promising paradigm by utilizing massive unlabeled data, reducing dependence on manual labeling, and mitigating data insufficiencies. SSL adapts pretext tasks to extract supervision signals from data itself, making it suitable for recommendation systems where data is sparse and often biased.

This survey categorizes SSL strategies into four distinct paradigms:

1. Contrastive Methods: Emphasizing instance discrimination, these methods create positive and negative sample pairs by contrasting augmented data. Key techniques include feature noise addition and node dropout for graph structures, known to improve uniformity and alleviate popularity bias.
2. Generative Methods: Inspired by masked LLMs like BERT, these methods reconstruct data after masking parts. Generative tasks include predicting masked items or nodes mostly seen in sequence recommender systems, leveraging architectures like Transformers.
3. Predictive Methods: Predictive methodologies utilize generated pseudo-labels, focusing on tasks like relation prediction. This avenue aligns closely with techniques such as self-training and co-training, predicting informative samples iteratively.
4. Hybrid Methods: Combine various self-supervised practices to amplify the strengths of individual tasks, typically integrating predictive or generative tasks with contrastive learning to enhance supervision signals.

Empirical Findings and Challenges

Using SELFRec, an open-source library specifically designed for benchmarking SSR methods, significant insights were drawn. In graph-based scenarios, contrastive methods exhibited superior performance, benefitting from structured augmentations like feature noise addition. However, sequential SSL methods struggled to replicate this success, suggesting potential limitations in current augmentation strategies.

Despite remarkable advancements, SSR models face several challenges. Augmentation selection remains heuristic; essential is a foundational theory to guide these choices effectively. Moreover, the black-box nature of many models necessitates efforts towards interpretability and explainability. Security concerns, particularly attacks on self-supervised models, call for the development of robust defense mechanisms. Lastly, the vision of deploying general-purpose recommendation systems, leveraging extensive multi-modal datasets, remains an enticing yet under-explored frontier.

Future Directions

Efforts should emphasize the development of principled data augmentation theories specifically tailored for recommendation systems. Explainability should be prioritized to demystify the internal mechanics that drive SSR models' successes. Attacks on SSR systems, including sophisticated data poisoning strategies, further highlight the urgent need for enhanced model robustness. Exploring on-device SSR presents opportunities for decentralized recommendation systems, aligning model performance with privacy preservation and reduced computational overhead. Lastly, creating general-purpose pre-trained models that support a wide array of recommendation tasks holds promise for tackling data sparsity in expansive modalities.

In conclusion, this survey not only delineates the landscape of self-supervised recommender systems but also sets a roadmap for addressing existing limitations and spearheading innovation within the domain.