QARM: Quantitative Alignment Multi-Modal Recommendation at Kuaishou (2411.11739v1)

Abstract: In recent years, with the significant evolution of multi-modal large models, many recommender researchers realized the potential of multi-modal information for user interest modeling. In industry, a wide-used modeling architecture is a cascading paradigm: (1) first pre-training a multi-modal model to provide omnipotent representations for downstream services; (2) The downstream recommendation model takes the multi-modal representation as additional input to fit real user-item behaviours. Although such paradigm achieves remarkable improvements, however, there still exist two problems that limit model performance: (1) Representation Unmatching: The pre-trained multi-modal model is always supervised by the classic NLP/CV tasks, while the recommendation models are supervised by real user-item interaction. As a result, the two fundamentally different tasks' goals were relatively separate, and there was a lack of consistent objective on their representations; (2) Representation Unlearning: The generated multi-modal representations are always stored in cache store and serve as extra fixed input of recommendation model, thus could not be updated by recommendation model gradient, further unfriendly for downstream training. Inspired by the two difficulties challenges in downstream tasks usage, we introduce a quantitative multi-modal framework to customize the specialized and trainable multi-modal information for different downstream models.

• The paper introduces QARM, a novel methodology that aligns multi-modal representations with recommendation tasks through end-to-end optimization.
• It leverages an item alignment mechanism to fine-tune pre-trained models using business-specific data and employs quantitative codes to create dynamic, trainable semantic IDs.
• Experiments on Kuaishou's platform demonstrate improved metrics like AUC and GAUC, particularly benefiting long-tail and cold-start scenarios in shopping and advertising.

Quantitative Alignment Multi-Modal Recommendation at Kuaishou: An Overview

The research paper titled "QARM: Quantitative Alignment Multi-Modal Recommendation at Kuaishou" discusses the implementation of a novel methodology for enhancing recommendation systems using multi-modal information, specifically tailored for Kuaishou, a major short-video and live-streaming platform in China. This paper tackles the challenges of aligning multi-modal representations with the practical needs of recommendation tasks and ensuring these representations are adaptable through end-to-end training.

Problem Statement and Motivation

The application of multi-modal models in recommendation systems has been recognized for its potential to improve user interest modeling significantly. The current prevalent approach in industry leverages a two-step cascading paradigm: pre-training multi-modal models on generic tasks (such as NLP or CV tasks) and subsequently using these representations as features in downstream recommendation models. Despite notable success, this approach faces two primary drawbacks:

• Representation Unmatching: Discrepancies arise due to differing objectives between the pre-trained multi-modal models and recommendation models. The representations optimized for generic tasks do not inherently align with the specific goals of user-item interaction data captured in recommendation systems.
• Representation Unlearning: The static nature of pre-trained representations, stored as fixed inputs, limits their adaptability as they cannot be updated during recommendation model training, potentially hindering performance improvements.

Proposed Methodology: QARM

The paper introduces QARM (Quantitative Alignment Multi-Modal Recommendation), designed to address the above limitations through two core mechanisms:

1. Item Alignment Mechanism: This approach involves fine-tuning pre-trained multi-modal models using data tailored to specific business tasks, enabling representations to reflect the unique user-item interactions pertinent to individual applications. This fine-tuning aligns multi-modal features more closely with the business objectives, enhancing the representation consistency.
2. Quantitative Code Mechanism: To overcome the representation’s lack of adaptability, QARM employs discrete quantitative codes (echoing hashing techniques and straight-through estimators) that convert multi-modal representations into trainable, learnable Semantic IDs, allowing for end-to-end optimization. These codes are generated using both VQ and RQ coding strategies to reflect different facets of the item representations.

Experimental Validation

The paper details extensive offline and online evaluations conducted in various Kuaishou services, including shopping and advertising domains, showing the efficacy of QARM. The offline analyses highlight QARM’s ability to improve metrics such as AUC, UAUC, and GAUC across different tasks, with enhancements particularly notable in long-tailed and cold-start items—a desirable feature for platform dynamics.

Online A/B testing further substantiates QARM's impact, demonstrating significant improvements in revenue for advertising and a noticeable increase in GMV for shopping scenarios. The results are consistent across various item categories, reinforcing the versatility of the approach.

Implications and Future Directions

QARM represents a significant advancement in incorporating multi-modal data within recommendation systems, offering a method that not only capitalizes on the rich semantic content of multi-modal inputs but also ensures these inputs are dynamically refined during the model training phase.

The implications of this work are manifold:

• Efficiency: By integrating a quantitative approach to multi-modal representations, the method enhances scalability for massive datasets.
• Improved User Experience: The end-to-end learning facilitates more accurate recommendations, likely leading to increased user engagement and satisfaction.
• Adaptability: The flexibility in adjusting representations to capture business-specific interactions paves the way for more nuanced and targeted recommendation strategies.

Future work could explore the expansion of this method to other multi-modal data sources and refine quantitative coding techniques for increased efficiency. Additionally, further investigations could involve the integration of more sophisticated alignment models or leveraging advanced neural architectures for even more refined representations.

In conclusion, the research on QARM offers a highly technical yet pragmatic solution for addressing notable challenges in the deployment of multi-modal recommendation systems, with strong implications for practical application in large-scale industrial environments. This sets a new benchmark for deploying multi-modal data in recommendation ecosystems effectively.