Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction (2411.15005v5)

Abstract: Click-through Rate (CTR) prediction is crucial for online personalization platforms. Recent advancements have shown that modeling rich user behaviors can significantly improve the performance of CTR prediction. Current long-term user behavior modeling algorithms predominantly follow two cascading stages. The first stage retrieves subsequence related to the target item from the long-term behavior sequence, while the second stage models the relationship between the subsequence and the target item. Despite significant progress, these methods have two critical flaws. First, the retrieval query typically includes only target item information, limiting the ability to capture the user's diverse interests. Second, relational information, such as sequential and interactive information within the subsequence, is frequently overlooked. Therefore, it requires to be further mined to more accurately model user interests. To this end, we propose Multi-granularity Interest Retrieval and Refinement Network (MIRRN). Specifically, we first construct queries based on behaviors observed at different time scales to obtain subsequences, each capturing users' interest at various granularities. We then introduce an noval multi-head Fourier transformer to efficiently learn sequential and interactive information within the subsequences, leading to more accurate modeling of user interests. Finally, we employ multi-head target attention to adaptively assess the impact of these multi-granularity interests on the target item. Extensive experiments have demonstrated that MIRRN significantly outperforms state-of-the-art baselines. Furthermore, an A/B test shows that MIRRN increases the average number of listening songs by 1.32% and the average time of listening songs by 0.55% on the Huawei Music App. The implementation code is publicly available at https://github.com/USTC-StarTeam/MIRRN.

• The paper demonstrates that MIRRN enhances CTR prediction by leveraging multi-granularity interest retrieval and sequence refinement, achieving up to a 1.35% AUC improvement.
• It introduces a novel multi-head Fourier Transformer to capture sequential interactions efficiently, balancing performance with computational complexity.
• Real-world tests validate MIRRN's impact, showing significant engagement gains with a 1.32% increase in song listens and a 0.55% boost in listening time.

An Analysis of Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction

This paper introduces a novel approach to Click-Through Rate (CTR) prediction by proposing the Multi-granularity Interest Retrieval and Refinement Network (MIRRN). The model addresses the limitations of current methodologies by focusing on long-term user behavior modeling to more effectively capture users' diverse interests.

Problem Statement

CTR prediction is a pivotal component for personalization in online platforms such as recommendation systems and online advertising. Current methods in long-term user behavior modeling mainly focus on identifying sequences related to a target item, followed by modeling relationships between the retrieved subsequences and the item. However, these methods often neglect two key aspects: the need to capture diverse user interests due to limited retrieval queries and the insufficient exploration of relational information within subsequences.

Proposed Methodology

The authors propose the MIRRN framework to mitigate these shortcomings through a multi-stage approach:

1. Multi-granularity Interest Retrieval: Users' diverse interests are captured by constructing different queries utilizing behaviors at various time scales. This yields subsequences reflective of multi-granular interests, categorized into target-aware, local-aware, and global-aware interests.
2. Behavior Sequence Refinement: To extract sequential and interactive information within subsequences, a novel multi-head Fourier Transformer is introduced. This approach enhances the modeling of user behaviors by efficiently capturing interaction with a time complexity comparable to self-attention mechanisms.
3. Interest Activation: MIRRN employs multi-head target attention to weigh the influence of different granular interests on the target item.

Experimental Setup and Results

The presented model underwent rigorous evaluation using three large-scale datasets: Taobao, Alipay, and Tmall. MIRRN demonstrated superior performance, achieving an AUC improvement of up to 1.35% over state-of-the-art baselines. The inclusion of multiple granularities in interest retrieval was shown to significantly enhance performance. Moreover, each component of the model, specifically the behavior sequence refinement module and the multi-head target attention mechanism, proved crucial to achieving these results.

Additionally, an online A/B test conducted on a popular music streaming application revealed that MIRRN increased the average number of songs listened to by 1.32% and the average listening time by 0.55%.

Key Contributions

• Successful Multi-granular Interest Modeling: Unlike previous models that focus solely on target-aware interests, MIRRN encapsulates the user's multi-faceted behaviors across different temporal scopes, enhancing the user interest representation.
• Efficient Interaction Modeling: Through the introduction of the multi-head Fourier transformer, sequential and interactive information is mined comprehensively and efficiently. This innovation offers a balance between model complexity and inference speed.
• Practical Application and Business Value: In addition to outperforming academic benchmarks, the model demonstrates significant practical benefits in real-world applications, as illustrated by the improvements in user engagement metrics in the online test.

Implications and Future Work

The implications of this research extend to improving the efficacy and personalization of CTR models across various platforms by more accurately modeling user behavior sequences. Future research could explore further optimizations of retrieval methods and extend the multidimensional interest representations to cater to different business scenarios. Additionally, ongoing investigation into more efficient computational techniques could reduce the time complexity further, enhancing real-time application capabilities.

The MIRRN not only pushes the boundaries of CTR prediction through advanced user modeling techniques but also sets a precedent for future innovations in personalization technologies.