Graph Convolutional Neural Networks for Web-Scale Recommender Systems (1806.01973v1)

Abstract: Recent advancements in deep neural networks for graph-structured data have led to state-of-the-art performance on recommender system benchmarks. However, making these methods practical and scalable to web-scale recommendation tasks with billions of items and hundreds of millions of users remains a challenge. Here we describe a large-scale deep recommendation engine that we developed and deployed at Pinterest. We develop a data-efficient Graph Convolutional Network (GCN) algorithm PinSage, which combines efficient random walks and graph convolutions to generate embeddings of nodes (i.e., items) that incorporate both graph structure as well as node feature information. Compared to prior GCN approaches, we develop a novel method based on highly efficient random walks to structure the convolutions and design a novel training strategy that relies on harder-and-harder training examples to improve robustness and convergence of the model. We also develop an efficient MapReduce model inference algorithm to generate embeddings using a trained model. We deploy PinSage at Pinterest and train it on 7.5 billion examples on a graph with 3 billion nodes representing pins and boards, and 18 billion edges. According to offline metrics, user studies and A/B tests, PinSage generates higher-quality recommendations than comparable deep learning and graph-based alternatives. To our knowledge, this is the largest application of deep graph embeddings to date and paves the way for a new generation of web-scale recommender systems based on graph convolutional architectures.

• The paper introduces RW-GCN, a scalable graph convolutional neural network that leverages random walks for efficient recommendations on massive graphs.
• It employs a novel producer-consumer architecture and curriculum training strategy to improve hit-rate and MRR compared to baseline models.
• Empirical results on Pinterest show a 10-30% repin rate boost and significant gains in user engagement through optimized node sampling techniques.

Graph Convolutional Neural Networks for Web-Scale Recommender Systems

The paper "Graph Convolutional Neural Networks for Web-Scale Recommender Systems," presented at KDD 2018, introduces RW-GCN, an innovative and scalable approach to graph convolutional neural networks (GCNs) designed specifically for large-scale recommender systems. The research is significantly grounded in the context of Pinterest, a major content discovery platform.

Overview and Core Contributions

Much of the recent progress in recommender systems has focused on leveraging deep neural networks to enhance content-based recommendations. However, integrating both content information and relational data from user-item interaction graphs has remained challenging, particularly at a web-scale involving billions of nodes and edges. This paper effectively bridges this gap by introducing RW-GCN, a GCN variant that scales efficiently to massive datasets.

RW-GCN makes several key contributions:

1. Efficient Graph Convolutions: Unlike traditional GCN algorithms, RW-GCN utilizes dynamically constructed computation graphs from sampled neighborhoods using random walks, thus avoiding the computational infeasibility of operating on the full graph Laplacian.
2. Scalable Minibatch Construction: A novel producer-consumer architecture ensures high GPU utilization by efficiently sampling node network neighborhoods and fetching necessary features during model training.
3. Random Walk-Based Convolutions: Employing biased random walks for defining node neighborhoods improves representation quality by incorporating importance pooling, wherein neighbor nodes contribute to the embedding based on weighted random-walk similarities.
4. Curriculum Training Strategy: Introducing harder negative examples progressively during training enhances robustness and model convergence.

These innovations enable RW-GCN to be applied to the Pinterest graph, comprising 3 billion nodes and 18 billion edges, showcasing a practical deployment of GCNs at an unprecedented scale.

Numerical Results and Implementation

The empirical evaluation of RW-GCN demonstrates significant performance improvements across multiple metrics. When benchmarked against leading deep learning and graph-based recommender systems, RW-GCN achieves the following:

• Hit-Rate and MRR: In the offline evaluation, RW-GCN demonstrated a 67% hit-rate and a 0.59 Mean Reciprocal Rank (MRR), outperforming the best baseline by 150% in terms of hit-rate and by 60% in MRR.
• User Studies: Human evaluations in head-to-head comparisons consistently favored RW-GCN's recommendations over those from other systems, with an overall preference rate of about 60%.
• A/B Testing: Production A/B tests for homefeed recommendations on Pinterest indicated a 10-30% improvement in repin rate compared to baselines using visual and annotation embeddings.

Technical Implications and Future Directions

The theoretical implications of this work are substantial. RW-GCN addresses the limitations of previous GCN approaches by ensuring that the model complexity remains independent of the graph size, a critical factor for scalability. The integration of random-walk-based convolutions and importance pooling contributes to more informative embeddings, achieving a nuanced balance between feature aggregation and computational efficiency.

On a practical level, the deployment of RW-GCN sets a new benchmark for recommender systems capable of handling dynamic, large-scale graphs. The MapReduce inference pipeline further optimizes model application by avoiding redundant computations, thereby facilitating swift deployment of embeddings across billions of nodes.

Looking forward, RW-GCN opens several avenues for future research:

1. Generalization across domains: Expanding the application of RW-GCN to other problem domains such as knowledge graphs, biological networks, and social networks to evaluate its generalizability and effectiveness.
2. Incorporating Temporal Dynamics: Integrating temporal changes in graph structures and node features could enhance the adaptability of RW-GCN in real-time recommendation scenarios.
3. Enhanced Sampling Techniques: Exploring advanced sampling techniques might further improve the quality and efficiency of the node embeddings, especially in more complex graph topologies.

Conclusion

RW-GCN represents a significant step forward in the evolution of graph convolutional neural networks for recommender systems, demonstrating scalable, high-performance capabilities in a real-world production environment. The introduction of effective sampling strategies, importance-based neighborhoods, and a rigorous training schema provides a robust framework for future developments in web-scale graph-based learning systems.