Deep Interest Evolution Network for Click-Through Rate Prediction (1809.03672v5)

Abstract: Click-through rate~(CTR) prediction, whose goal is to estimate the probability of the user clicks, has become one of the core tasks in advertising systems. For CTR prediction model, it is necessary to capture the latent user interest behind the user behavior data. Besides, considering the changing of the external environment and the internal cognition, user interest evolves over time dynamically. There are several CTR prediction methods for interest modeling, while most of them regard the representation of behavior as the interest directly, and lack specially modeling for latent interest behind the concrete behavior. Moreover, few work consider the changing trend of interest. In this paper, we propose a novel model, named Deep Interest Evolution Network~(DIEN), for CTR prediction. Specifically, we design interest extractor layer to capture temporal interests from history behavior sequence. At this layer, we introduce an auxiliary loss to supervise interest extracting at each step. As user interests are diverse, especially in the e-commerce system, we propose interest evolving layer to capture interest evolving process that is relative to the target item. At interest evolving layer, attention mechanism is embedded into the sequential structure novelly, and the effects of relative interests are strengthened during interest evolution. In the experiments on both public and industrial datasets, DIEN significantly outperforms the state-of-the-art solutions. Notably, DIEN has been deployed in the display advertisement system of Taobao, and obtained 20.7\% improvement on CTR.

• The paper introduces DIEN, which innovatively captures the temporal evolution of user interests using GRU and an attentional update mechanism.
• It employs an Interest Extractor layer for latent interest extraction and an Interest Evolving layer via AUGRU to refine predictions.
• Evaluations on public and industrial datasets show DIEN significantly improves CTR performance, AUC, and eCPM over previous methods.

Deep Interest Evolution Network for Click-Through Rate Prediction

The "Deep Interest Evolution Network (DIEN) for Click-Through Rate Prediction" paper addresses the challenge of effectively modeling user interest dynamics for CTR prediction in advertising systems. The focus is on capturing latent user interests from behavior data and understanding how these interests evolve over time.

Introduction

CTR prediction is crucial in advertising systems where billing is often based on cost per click (CPC). Accurate CTR prediction impacts both revenue and user experience. Traditional models, including some deep learning approaches, do not explicitly model the evolution of user interests which can lead to suboptimal performance. The DIEN model attempts to fill this gap by focusing on the temporal evolution of user interests and their effects on CTR.

Core Contributions

Key contributions of the paper are:

1. Interest Extractor Layer: This layer utilizes GRU to model dependencies between sequential user behaviors. Each behavior in history is transformed into an interest representation. An auxiliary loss function is introduced to supervise and refine these hidden states, ensuring they effectively capture latent user interests.
2. Interest Evolving Layer: This layer models how user interests evolve over time in response to sequential behaviors and relative to the target advertisement. The authors introduce the Attentional Update Gate GRU (AUGRU) which integrates attention mechanisms with GRU. This architecture strengthens the influence of relevant interests and mitigates the effects of interest drifting.

Methodology

The DIEN architecture includes the following steps:

1. Feature Representation: User profile, behavior, ad, and context features are transformed using embedding layers.
2. Interest Extractor Layer: Sequential behaviors are processed through GRU to extract temporal interests. An auxiliary loss provides supervision, making intermediate interest states meaningful.
3. Interest Evolving Layer: Using the extracted interest sequence, AUGRU is applied to model how these interests evolve over time and how they relate to the target item.

Experimental Results

DIEN's performance was evaluated on both public datasets from Amazon (Books and Electronics) and an industrial dataset from the Taobao display advertisement system. The model significantly outperformed state-of-the-art methods:

• On public datasets, DIEN achieved an AUC improvement, particularly with an AUC of 0.7792 on the Electronics dataset and 0.8453 on the Books dataset.
• In industrial settings, DIEN achieved a 20.7% CTR improvement and a 17.1% eCPM gain, demonstrating substantial benefits over existing solutions like DIN.

Practical Implications

The deployment of DIEN in a real-world system like Taobao validates its practical utility. The model's ability to capture dynamic user interests directly translates to improved CTR predictions, enhancing both user experience and revenue.

Theoretical Implications and Future Work

The integration of attention mechanisms within GRU through AUGRU offers a novel way to handle sequential user interests evolution. This approach could inspire further research into more nuanced models that better capture user behavioral dynamics. Future work may explore more personalized interest models or extend these methods to other domains within recommendation systems.

Conclusion

DIEN represents a significant advancement in CTR prediction by dynamically modeling user interests. By using techniques like GRU and attention mechanisms, DIEN offers a robust framework for understanding and predicting user clicks more effectively, making it a valuable contribution to both academia and industry.