Unifying Generative and Dense Retrieval for Sequential Recommendation (2411.18814v2)

Abstract: Sequential dense retrieval models utilize advanced sequence learning techniques to compute item and user representations, which are then used to rank relevant items for a user through inner product computation between the user and all item representations. However, this approach requires storing a unique representation for each item, resulting in significant memory requirements as the number of items grow. In contrast, the recently proposed generative retrieval paradigm offers a promising alternative by directly predicting item indices using a generative model trained on semantic IDs that encapsulate items' semantic information. Despite its potential for large-scale applications, a comprehensive comparison between generative retrieval and sequential dense retrieval under fair conditions is still lacking, leaving open questions regarding performance, and computation trade-offs. To address this, we compare these two approaches under controlled conditions on academic benchmarks and propose LIGER (LeveragIng dense retrieval for GEnerative Retrieval), a hybrid model that combines the strengths of these two widely used methods. LIGER integrates sequential dense retrieval into generative retrieval, mitigating performance differences and enhancing cold-start item recommendation in the datasets evaluated. This hybrid approach provides insights into the trade-offs between these approaches and demonstrates improvements in efficiency and effectiveness for recommendation systems in small-scale benchmarks.

• The paper proposes LIGER, a hybrid retrieval framework that unifies generative and dense methods to effectively mitigate cold-start challenges.
• It employs beam search in generative retrieval combined with dense ranking to achieve superior Recall@10 and NDCG@10 across various datasets.
• The study demonstrates that LIGER reduces computational load while maintaining high recommendation quality in dynamic, large-scale environments.

Unifying Generative and Dense Retrieval for Sequential Recommendation: A Critical Analysis

Recent advances in recommendation systems have leaned towards exploring hybrid approaches that combine different retrieval paradigms to enhance efficiency and recommendation quality. The paper "Unifying Generative and Dense Retrieval for Sequential Recommendation" investigates this by proposing a novel hybrid model, LIGER, which integrates generative retrieval with dense retrieval techniques.

The paper begins by acknowledging the merits and limitations of the existing retrieval frameworks. Sequential dense retrieval methods, renowned for their effectiveness in learning dense embeddings for items and users, face scalability issues due to computational and memory demands. These models require the storage and computation of unique representations for each item, resulting in challenges when dealing with vast corpuses of information. On the other hand, generative retrieval approaches, which have recently garnered interest, sidestep the need for dense embeddings by generating item indices directly. However, these methods often falter, especially in cold-start scenarios where new items have limited historical interaction data.

The paper positions LIGER as a fusion of these paradigms, leveraging the computational advantages and semantic richness of generative retrieval while circumventing its limitations through dense retrieval enhancements. This hybridization is theoretically and practically significant, particularly as it addresses the cold-start problem—a pervasive challenge in real-world recommendation systems—by combining the prediction-based capabilities of generative retrieval with the storage and efficiency insights of dense retrieval.

Central to the paper's contribution is the methodical comparison and analysis of these paradigms under controlled experimental conditions using various academic benchmarks. The authors identify performance disparities between generative and dense retrieval approaches and propose LIGER as a reconciliatory intermediary that not only narrows these gaps but excels in specific metrics like Recall@10 and NDCG@10 in both in-set and cold-start item predictions.

Quantitatively, the results from experiments conducted on datasets such as Beauty, Sports, Toys, and Steam underscore LIGER’s superior performance across different settings. It achieves this by adapting its rank computation based on candidate retrieval through a beam search in generative retrieval, followed by dense retrieval ranking. Importantly, these insights reveal LIGER's capabilities in reducing computational load without sacrificing recommendation quality, demonstrated through the efficiency of the algorithm in candidate retrieval using semantic IDs (SIDs).

The implications of this research are manifold. Practically, LIGER paves the way for more scalable, resource-efficient recommendation systems that maintain high accuracy in dynamic environments. Theoretically, this work invites further scrutiny into hybrid methodologies, emphasizing the potential of cross-pollinating traditionally disparate retrieval techniques. It signals a shift towards exploring retrieval models that not only address the intricacies of user-item interaction at scale but also adapt to new data with minimal retraining overheads.

Looking forward, future research could explore extending LIGER's framework with enhancements from large-scale, real-world applications, potentially incorporating more sophisticated LLMs or integrating multi-modal data sources. Additionally, exploring differential impacts across domains and fine-tuning computational optimizations could yield further benefits. As recommendation systems continue evolving, models like LIGER that blend complementary strengths of different paradigms are likely to dominate the landscape, prompting a reevaluation of how hybrid models are developed and applied.