Learning Tree-based Deep Model for Recommender Systems (1801.02294v5)

Abstract: Model-based methods for recommender systems have been studied extensively in recent years. In systems with large corpus, however, the calculation cost for the learnt model to predict all user-item preferences is tremendous, which makes full corpus retrieval extremely difficult. To overcome the calculation barriers, models such as matrix factorization resort to inner product form (i.e., model user-item preference as the inner product of user, item latent factors) and indexes to facilitate efficient approximate k-nearest neighbor searches. However, it still remains challenging to incorporate more expressive interaction forms between user and item features, e.g., interactions through deep neural networks, because of the calculation cost. In this paper, we focus on the problem of introducing arbitrary advanced models to recommender systems with large corpus. We propose a novel tree-based method which can provide logarithmic complexity w.r.t. corpus size even with more expressive models such as deep neural networks. Our main idea is to predict user interests from coarse to fine by traversing tree nodes in a top-down fashion and making decisions for each user-node pair. We also show that the tree structure can be jointly learnt towards better compatibility with users' interest distribution and hence facilitate both training and prediction. Experimental evaluations with two large-scale real-world datasets show that the proposed method significantly outperforms traditional methods. Online A/B test results in Taobao display advertising platform also demonstrate the effectiveness of the proposed method in production environments.

• The paper presents a novel hierarchical retrieval mechanism that achieves logarithmic complexity in large-scale recommendation tasks.
• It leverages advanced deep neural network interactions by adaptively learning tree structures, optimizing both training and prediction accuracy.
• Empirical results on datasets like MovieLens-20M and Taobao demonstrate significant improvements in Precision, Recall, and real-world system performance.

Learning Tree-based Deep Model for Recommender Systems

The paper "Learning Tree-based Deep Model for Recommender Systems" by Han Zhu et al., introduces a novel methodology to enhance the performance of recommender systems through a tree-based deep learning architecture. The primary objective is to address the combinatorial explosion of computational costs associated with models that leverage advanced interaction schemes, specifically in large-scale recommendation tasks.

Overview and Contributions

The authors propose a tree-based deep model (TDM) that efficiently manages the retrieval task in recommender systems by utilizing a hierarchical approach. The method targets the challenge of incorporating complex models, like deep neural networks, into large-scale recommendation frameworks by providing a feasible and computationally efficient solution.

Key contributions of the paper include:

1. Hierarchical Retrieval Mechanism: The TDM framework relies on a tree structure to traverse and identify user interests in a coarse-to-fine manner, achieving logarithmic computational complexity with respect to the corpus size. This drastically reduces the prediction complexity compared to existing methods.
2. Advanced Model Compatibility: By bypassing computational bottlenecks, the proposed method allows the application of complex interaction models, such as those derived from deep neural networks, rather than being constrained to simpler forms like the inner product typical in matrix factorization techniques.
3. Introduction of Tree Learning: The authors propose a learning mechanism for the hierarchical tree structure, which allows the model to adaptively optimize the hierarchy of items, enhancing both the training process and the resultant prediction accuracy.
4. Empirical Validation: Stunning performance improvements are backed by experiments conducted on large-scale datasets, including MovieLens-20M and a Taobao dataset. Metrics such as Precision, Recall, and Novelty demonstrate significant gains over traditional and contemporary baselines.
5. Practical Implementation: The TDM model has been successfully deployed in Taobao’s display advertising system, confirming its utility in real-world applications through online A/B testing that recorded increased click-through rates (CTR) and revenue per mille (RPM).

Numerical Results and Implications

The proposed method reports substantial improvements across critical metrics:

• For Precision@10, TDM achieved 14.06% on the MovieLens-20M dataset, significantly outperforming traditional methods like Item-CF and even advanced models like YouTube's product-DNN.
• In terms of Recall@200, its performance on the UserBehavior dataset improved from 7.58% (by YouTube's product-DNN) to 10.81%, denoting a remarkable enhancement in capturing user preferences.

Furthermore, the methods show improved novelty without degrading accuracy, an essential aspect of user experience in recommendation systems.

Theoretical and Practical Implications

Theoretically, this approach extends the frontier of recommender system research by reconciling the trade-off between model expressiveness and computational feasibility. The hierarchical modeling technique adopted in TDM exemplifies how structure and deep learning capabilities can be harmoniously blended to solve complex retrieval problems in recommendation tasks.

Practically, the research facilitates a scalable architecture conducive to direct applicability in large-scale environments, as demonstrated by its integration into Taobao’s advertising platform. It paves the way for further exploration into adaptive hierarchical modeling and its application to other domains requiring precise large-scale predictions.

Future Directions

The scope for subsequent research includes improving tree learning algorithms for better initial tree structures and exploring alternative neural network architectures within the TDM framework to further enhance recommendation effectiveness. Additionally, fine-tuning and optimizing the model for different domains and types of recommendation tasks can expand its applicability.

In conclusion, this paper provides a comprehensive strategy to embed sophisticated models into recommendation systems efficiently, addressing a fundamental challenge in the domain and setting a new standard for computationally intensive recommendation tasks.