Collaborative Knowledge Fusion: A Novel Approach for Multi-task Recommender Systems via LLMs (2410.20642v1)

Abstract: Owing to the impressive general intelligence of LLMs, there has been a growing trend to integrate them into recommender systems to gain a more profound insight into human interests and intentions. Existing LLMs-based recommender systems primarily leverage item attributes and user interaction histories in textual format, improving the single task like rating prediction or explainable recommendation. Nevertheless, these approaches overlook the crucial contribution of traditional collaborative signals in discerning users' profound intentions and disregard the interrelatedness among tasks. To address these limitations, we introduce a novel framework known as CKF, specifically developed to boost multi-task recommendations via personalized collaborative knowledge fusion into LLMs. Specifically, our method synergizes traditional collaborative filtering models to produce collaborative embeddings, subsequently employing the meta-network to construct personalized mapping bridges tailored for each user. Upon mapped, the embeddings are incorporated into meticulously designed prompt templates and then fed into an advanced LLM to represent user interests. To investigate the intrinsic relationship among diverse recommendation tasks, we develop Multi-Lora, a new parameter-efficient approach for multi-task optimization, adept at distinctly segregating task-shared and task-specific information. This method forges a connection between LLMs and recommendation scenarios, while simultaneously enriching the supervisory signal through mutual knowledge transfer among various tasks. Extensive experiments and in-depth robustness analyses across four common recommendation tasks on four large public data sets substantiate the effectiveness and superiority of our framework.

• The paper introduces Collaborative Knowledge Fusion, which uniquely integrates collaborative filtering techniques with LLM semantic spaces for enhanced recommendations.
• It employs a multi-module approach combining advanced embedding extraction, personalized mapping functions, and the novel Multi-Lora fine-tuning strategy.
• Experimental results on four datasets show that CKF significantly outperforms traditional models, especially in sparse data and cold-start scenarios.

Collaborative Knowledge Fusion: A Novel Approach for Multi-task Recommender Systems via LLMs

The paper by Chuang Zhao et al. introduces an innovative framework called Collaborative Knowledge Fusion (CKF) aimed at enhancing multi-task recommender systems through the integration of LLMs. The work is situated at the intersection of collaborative filtering and LLM ecosystems, seeking to leverage their combined strengths for improved recommendation accuracy and coverage.

Core Contributions

The paper presents a novel framework, CKF, which utilizes a multi-faceted approach combining collaborative filtering models with LLMs for multi-task optimization. The framework is meticulously designed to integrate traditional collaborative signals with modern LLM methodologies, categorized into three distinct modules: Collaborative Knowledge Extraction Module (CKEM), Knowledge Fusion Module (KFM), and Multi-Task Tuning Module (MTM).

1. Collaborative Knowledge Extraction Module: This module utilizes advanced collaborative filtering techniques such as matrix factorization and graph neural networks to generate embeddings which encapsulate rich collaborative knowledge.
2. Knowledge Fusion Module: In this stage, personalized mapping functions are crafted using meta-networks, transforming user-item collaborative embeddings to align seamlessly with the LLM semantic space. This alignment ensures that the unique interests of users are comprehensively represented, enhancing the recommendation fidelity.
3. Multi-Task Tuning Module: The authors develop a novel fine-tuning strategy named Multi-Lora, focusing on the explicit decoupling of task-shared and task-specific information within the LLM’s parameter space. This novel contribution facilitates a deeper insight into task interrelations, promoting mutual knowledge transfer among tasks.

Experimental Validation

The framework is rigorously evaluated on four standard datasets across multiple tasks: Rating Prediction, Click-Through Rate Estimation, Top-K Ranking, and Explainable Recommendation. CKF demonstrates significant improvements over existing methods such as traditional collaborative filtering approaches (e.g., DIN, SASRec) and LLM-based methods (e.g., CoLLM, TALLRec). The experiments validate CKF’s robustness, showing enhanced adaptation, especially in scenarios characterized by sparse data or new users.

Implications and Future Directions

CKF's approach indicates a significant stride in integrating collaborative filtering and LLMs, emphasizing the utility of collaborative signals in enhancing LLM performance for recommendations. This fusion offers a promising avenue for not only improving existing recommendation systems but also for expanding their applicability across diverse user contexts.

Theoretically, CKF opens up new research pathways addressing the integration of different learning paradigms. Practically, this work suggests improvements in user engagement through personalized and context-aware recommendations, potentially driving innovations in personalization technologies.

However, the framework also invites further exploration. Future research could delve into integrating multimodal data sources or extending the framework’s application to more diverse recommendation landscapes. There is also potential for exploring unsupervised or self-supervised learning paradigms in enhancing the semantic understanding of collaborative signals.

In conclusion, this paper contributes significantly to the recommender system domain, providing a comprehensive solution that bridges the gap between collaborative filtering and LLMs through innovative methodologies and robust validations.