Prompt Tuning Large Language Models on Personalized Aspect Extraction for Recommendations (2306.01475v1)

Abstract: Existing aspect extraction methods mostly rely on explicit or ground truth aspect information, or using data mining or machine learning approaches to extract aspects from implicit user feedback such as user reviews. It however remains under-explored how the extracted aspects can help generate more meaningful recommendations to the users. Meanwhile, existing research on aspect-based recommendations often relies on separate aspect extraction models or assumes the aspects are given, without accounting for the fact the optimal set of aspects could be dependent on the recommendation task at hand. In this work, we propose to combine aspect extraction together with aspect-based recommendations in an end-to-end manner, achieving the two goals together in a single framework. For the aspect extraction component, we leverage the recent advances in LLMs and design a new prompt learning mechanism to generate aspects for the end recommendation task. For the aspect-based recommendation component, the extracted aspects are concatenated with the usual user and item features used by the recommendation model. The recommendation task mediates the learning of the user embeddings and item embeddings, which are used as soft prompts to generate aspects. Therefore, the extracted aspects are personalized and contextualized by the recommendation task. We showcase the effectiveness of our proposed method through extensive experiments on three industrial datasets, where our proposed framework significantly outperforms state-of-the-art baselines in both the personalized aspect extraction and aspect-based recommendation tasks. In particular, we demonstrate that it is necessary and beneficial to combine the learning of aspect extraction and aspect-based recommendation together. We also conduct extensive ablation studies to understand the contribution of each design component in our framework.

• The paper introduces a novel dual-framework that integrates prompt tuning for personalized aspect extraction and recommendation.
• It leverages fine-tuned LLMs with user and item embeddings to extract context-specific aspect terms, outperforming benchmark methods.
• The joint learning approach yields targeted recommendations and enhanced explainability, validated on three industry datasets.

Prompt Tuning LLMs on Personalized Aspect Extraction for Recommendations

Overview of the Proposed Framework

In the landscape of modern recommender systems, the integration of explainable and personalized aspects has emerged as a crucial area of interest. This paper introduces a novel framework that marries aspect extraction and aspect-based recommendations in a seamless manner by leveraging the capabilities of LLMs. Specifically, it proposes a dual-component model that utilizes prompt tuning to refine aspect extraction tailored for recommendation tasks. This approach aims to generate user-centric and contextualized aspect terms, which are subsequently utilized to enhance the recommendation process. The research validates the effectiveness of the framework through experiments on three industrial-scale datasets, demonstrating notable improvements over existing state-of-the-art methods in both personalized aspect extraction and aspect-based recommendation metrics.

Component 1: Aspect Extraction Using Prompt Tuning

The first component of the proposed framework focuses on extracting relevant aspect terms from user-generated content by employing a prompt-tuning mechanism with pre-trained LLMs, such as GPT-2. This method involves fine-tuning the LLM on domain-specific data to better align with the target task and creating soft prompts that incorporate user and item embeddings. These embeddings serve as context to guide the model in extracting aspects that are not only significant to the review content but also personalized and relevant to the users’ preferences and the items’ characteristics. The efficacy of this approach is underscored by its ability to outperform several benchmarks in aspect term extraction accuracy, highlighting the advantage of integrating user and item-specific information into the extraction process.

Component 2: Aspect-Based Recommendation

Building on the extracted aspects, the second component of the framework employs these aspects alongside traditional user and item features to generate recommendations. This part utilizes an attention mechanism to weigh the significance of different aspects based on their relevance to the user’s preferences and the item’s attributes. By synthesizing this information, the model can produce more targeted and meaningful recommendations. The formulation and optimization of this component are carefully designed to ensure that the extracted aspect terms contribute effectively to enhancing the recommendation performance, as evidenced by the experimental results.

Joint Learning and Model Effectiveness

A key innovation of the proposed framework is its end-to-end architecture, which jointly optimizes the aspect extraction and recommendation tasks. This co-training mechanism ensures that the aspect terms extracted are not only representative of the content but are also instrumental in improving recommendation quality. The framework’s superiority is demonstrated through ablation studies that compare various configurations and underscore the significance of each component and methodology, including prompt tuning, fine-tuning of LLMs, and the joint learning process. Additionally, the framework’s scalability is attested by its linear increase in training time relative to data volume, rendering it a viable solution for large-scale industrial applications.

Implications and Future Directions

The proposed model stands as a pioneering step towards integrating aspect-based explanations within recommendation systems through the adept use of LLMs and prompt tuning. Its success opens numerous avenues for future exploration, particularly in optimizing feature embeddings for aspect extraction and enhancing model adaptability to new domains or sparse data scenarios. The promising results invite further research into refining the joint learning process and exploring the application of this framework across various recommendation platforms to enrich user experience through personalized, aspect-based insights.

In conclusion, this work not only contributes a novel framework that significantly advances the field of personalized aspect extraction and aspect-based recommendation but also sets a fertile ground for future investigations into the synergistic use of LLMs and prompt tuning in recommendation systems.