Retrieval Augmentation via User Interest Clustering (2408.03886v1)

Abstract: Many existing industrial recommender systems are sensitive to the patterns of user-item engagement. Light users, who interact less frequently, correspond to a data sparsity problem, making it difficult for the system to accurately learn and represent their preferences. On the other hand, heavy users with rich interaction history often demonstrate a variety of niche interests that are hard to be precisely captured under the standard "user-item" similarity measurement. Moreover, implementing these systems in an industrial environment necessitates that they are resource-efficient and scalable to process web-scale data under strict latency constraints. In this paper, we address these challenges by introducing an intermediate "interest" layer between users and items. We propose a novel approach that efficiently constructs user interest and facilitates low computational cost inference by clustering engagement graphs and incorporating user-interest attention. This method enhances the understanding of light users' preferences by linking them with heavy users. By integrating user-interest attention, our approach allows a more personalized similarity metric, adept at capturing the complex dynamics of user-item interactions. The use of interest as an intermediary layer fosters a balance between scalability and expressiveness in the model. Evaluations on two public datasets reveal that our method not only achieves improved recommendation performance but also demonstrates enhanced computational efficiency compared to item-level attention models. Our approach has also been deployed in multiple products at Meta, facilitating short-form video related recommendation.

Retrieval Augmentation via User Interest Clustering

Introduction

The paper "Retrieval Augmentation via User Interest Clustering" addresses critical challenges in modern recommender systems, particularly regarding the heterogeneity in user engagement patterns. Light users and heavy users present distinct challenges: light users suffer from data sparsity, while heavy users often have complex, niche interests that are difficult to capture accurately. The authors propose introducing an intermediate "interest" layer between users and items, facilitating efficient and scalable recommendation by clustering user engagement patterns.

Methodology

The proposed methodology centers on three main stages: user interest modeling, training, and inference:

1. User Interest Modeling: The authors transform a user-item bipartite graph into an item-item co-engagement graph. Using the Louvain community detection algorithm, items are clustered into interest groups. User interests are then modeled by linking them to these clusters using personalized relevance metrics such as Personalized PageRank (PPR).
2. Training Stage: Utilizing a two-tower model, the user and item embeddings are generated. The user interest clusters derived earlier are integrated with user embeddings by either concatenation or user-interest attention mechanisms. This enriches the representation of user preferences and tunes the recommendations.
3. Inference Stage: During inference, the approach leverages KNN searches within sampled interest clusters, significantly reducing the candidate pool and improving computational efficiency.

Experimental Evaluation

The authors conduct extensive experiments on two public datasets, MovieLens-1M and Recipe, along with deploying their method in a real-world setting at Meta. The performance is assessed using several metrics, including Precision@50, Recall@50, NDCG@50, and inference time.

Key Findings

1. Accuracy and Efficiency: The proposed User Interest Clustering (UIC) method outperforms several baselines in both accuracy and computational efficiency. UIC shows significant improvements in recommendation quality metrics when compared to traditional item-level attention models, while also demonstrating a marked reduction in serving time.
2. Handling Disparate User Engagement: UIC notably enhances recommendation quality across the spectrum of engagement levels. The method mitigates the disparities observed in light and heavy user recommendations, outperforming simpler engagement-based clustering methods.
3. Scalability: The approach scales effectively with large datasets, maintaining low inference times by leveraging interest clusters to limit the number of candidate items for recommendations. This efficiency is particularly beneficial in industrial settings with web-scale data and strict latency constraints.

Practical and Theoretical Implications

Practical Implications:

• Scalability and Performance: The UIC method, implemented in Meta products, significantly improved recommendation quality for short-form videos. This indicates its practicality in handling large-scale recommendation systems with diverse user engagement patterns.
• Resource Efficiency: By reducing the candidate pool through interest clusters, UIC optimizes computational resources, making it viable for industrial applications with extensive user bases.

Theoretical Implications:

• Interest Layer Integration: The introduction of an intermediate interest layer and the use of community detection algorithms to construct user interest profiles open new avenues for enhancing the expressiveness and scalability of recommender systems.
• Mitigating Popularity Bias: UIC's ability to balance popular and niche interests presents a novel approach to addressing popularity bias, a prevalent issue in traditional recommender systems.

Future Directions

The research opens several pathways for future exploration:

• Dynamic Interest Modeling: Investigating methods to dynamically update interest clusters as user engagement patterns evolve over time would further enhance recommendation relevance.
• Personalized Attention Mechanisms: Integrating more sophisticated attention mechanisms that adapt to individual user preferences on-the-fly could yield even more personalized recommendations.
• Expanding to Other Domains: Evaluating the UIC method in other recommendation contexts (e.g., e-commerce, news) would help generalize its applicability and effectiveness.

In conclusion, the "Retrieval Augmentation via User Interest Clustering" paper presents a robust framework tackling key challenges in recommendation systems. By efficiently clustering user engagements and integrating these patterns into the recommendation pipeline, it strikes a balance between accuracy and computational efficiency, demonstrating promising results in both experimental and real-world settings.