A Prompting-Based Representation Learning Method for Recommendation with Large Language Models (2409.16674v3)

Abstract: In recent years, Recommender Systems (RS) have witnessed a transformative shift with the advent of LLMs in the field of NLP. Models such as GPT-3.5/4, Llama, have demonstrated unprecedented capabilities in understanding and generating human-like text. The extensive information pre-trained by these LLMs allows for the potential to capture a more profound semantic representation from different contextual information of users and items. While the great potential lies behind the thriving of LLMs, the challenge of leveraging user-item preferences from contextual information and its alignment with the improvement of Recommender Systems needs to be addressed. Believing that a better understanding of the user or item itself can be the key factor in improving recommendation performance, we conduct research on generating informative profiles using state-of-the-art LLMs. To boost the linguistic abilities of LLMs in Recommender Systems, we introduce the Prompting-Based Representation Learning Method for Recommendation (P4R). In our P4R framework, we utilize the LLM prompting strategy to create personalized item profiles. These profiles are then transformed into semantic representation spaces using a pre-trained BERT model for text embedding. Furthermore, we incorporate a Graph Convolution Network (GCN) for collaborative filtering representation. The P4R framework aligns these two embedding spaces in order to address the general recommendation tasks. In our evaluation, we compare P4R with state-of-the-art Recommender models and assess the quality of prompt-based profile generation.

A Prompting-Based Representation Learning Method for Recommendation with LLMs

The increasing capabilities of LLMs such as GPT-3.5 and Llama in understanding and generating natural language text have opened new avenues for improving recommender systems (RS). The paper "A Prompting-Based Representation Learning Method for Recommendation with LLMs" by Junyi Chen and Toyotaro Suzumura leverages this progress by proposing a novel framework, P4R, which enhances recommendation performance through the integration of LLMs.

Introduction and Background

Recommender systems are essential for helping users discover relevant and personalized content. Traditional collaborative filtering methods, particularly those using Graph Convolutional Networks (GCNs), have demonstrated effectiveness in learning user and item embeddings by propagating them across user-item interaction graphs. However, these methods often overlook rich textual information derived from user reviews, item descriptions, and other contextual data, which could significantly benefit recommendation tasks.

This paper addresses this gap by innovatively combining LLMs with GCN-based collaborative filtering to enhance item profile representations. The Promising-Based Representation Learning Method for Recommendation (P4R) employs a prompting strategy to generate detailed item profiles and uses a pre-trained BERT model for text embedding. These embeddings are then aligned with representations derived from a Graph Convolution Network (GCN) to improve recommendation accuracy.

Methodology

The P4R framework is designed to integrate the linguistic capabilities of LLMs with the collaborative filtering strengths of GCNs. The methodology involves several key steps:

1. Auxiliary Feature Extraction Through In-Context Learning (ICL): P4R utilizes an advanced prompting strategy inspired by In-Context Learning (ICL), which is known for its efficiency and performance. The prompts are designed to extract both intrinsic (e.g., item title, category) and extrinsic (e.g., user reviews) attributes of items. The method emphasizes the need to treat these types of information differently to generate accurate and informative item profiles.
2. Textual Embedding and Representation: The generated item profiles are transformed into semantic representation spaces using a pre-trained BERT model. This model helps in capturing the detailed and rich semantic information contained within the text.
3. Alignment with GCN-Based Recommendation: The embeddings obtained from BERT are then aligned with user and item representations learned through a GCN-based approach. The model employs an aggregation function to iteratively propagate user and item embeddings across the interaction graph, integrating the LLM-enhanced item embeddings to improve the overall recommendation quality.

Results and Evaluation

The paper presents a comprehensive evaluation of the P4R framework on two popular datasets: Yelp and Amazon Video Games. The authors compare P4R with several state-of-the-art baselines, including NGCF, LightGCN, and SGL, demonstrating significant improvements in various metrics:

• Recall@20: P4R achieved higher recall scores compared to the baselines, indicating better capability in retrieving relevant items for users.
• NDCG@20: The normalized discounted cumulative gain (NDCG) scores show that P4R provides better-ranked recommendations.
• MRR@20 and Hit@20: P4R outperforms the baselines in mean reciprocal rank (MRR) and hit rate metrics, underscoring its effectiveness in recommendation accuracy and relevance.

The paper also provides an insightful ablation paper, highlighting the impact of different design choices within the P4R framework. For example, varying the embedding dimensions and the inclusion or exclusion of LLM-enhanced item profiles were analyzed, demonstrating the robustness and adaptability of the proposed method.

Implications and Future Directions

The integration of LLMs for enhancing textual representations in recommender systems offers several practical and theoretical implications:

• Practical Implications: The P4R framework is particularly suitable for smaller organizations with limited computational resources. By leveraging pre-trained models and focusing on prompting-based generation, the method achieves high efficiency without substantial overhead.
• Theoretical Implications: The paper opens up new research possibilities in the domain of LLM-enhanced recommendation systems. Future work could explore more advanced LLMs, further optimizing the prompting strategies, and understanding the impact of different types of contextual information on recommendation outcomes.

The promising results of P4R suggest that there is considerable potential in combining LLMs with traditional collaborative filtering methods. Future developments may include exploring more sophisticated models, fine-tuning strategies, and real-time recommendation scenarios to harness the full capability of LLMs in recommender systems.