Vertical Federated Graph Neural Network for Recommender System (2303.05786v3)

Abstract: Conventional recommender systems are required to train the recommendation model using a centralized database. However, due to data privacy concerns, this is often impractical when multi-parties are involved in recommender system training. Federated learning appears as an excellent solution to the data isolation and privacy problem. Recently, Graph neural network (GNN) is becoming a promising approach for federated recommender systems. However, a key challenge is to conduct embedding propagation while preserving the privacy of the graph structure. Few studies have been conducted on the federated GNN-based recommender system. Our study proposes the first vertical federated GNN-based recommender system, called VerFedGNN. We design a framework to transmit: (i) the summation of neighbor embeddings using random projection, and (ii) gradients of public parameter perturbed by ternary quantization mechanism. Empirical studies show that VerFedGNN has competitive prediction accuracy with existing privacy preserving GNN frameworks while enhanced privacy protection for users' interaction information.

• The paper introduces VerFedGNN, the first application of vertical federated learning to GNN-based recommender systems to enhance privacy in data sharing.
• It leverages random projection and ternary quantization to securely aggregate graph embeddings and perturb gradients, safeguarding user interactions.
• Empirical and theoretical analyses demonstrate that VerFedGNN achieves competitive prediction accuracy while significantly reducing data leakage risks.

Vertical Federated Graph Neural Network for Recommender System Enhances Privacy Protection

Introduction

The paper introduces a novel framework named VerFedGNN, which pioneers the application of Vertical Federated Learning (VFL) to Graph Neural Networks (GNNs) in the context of recommender systems. Federated Learning (FL) emerges as a promising solution for handling data privacy and isolation issues in machine learning. Specifically, FL enables multiple entities to collaboratively train models without sharing local data, thereby preserving user privacy. A crucial challenge in federated GNN-based recommender systems is maintaining the privacy of graph structure during embedding propagation, as this process traditionally requires sharing user-item interaction data, which is sensitive. VerFedGNN overcomes this by transmitting aggregated embeddings and perturbed gradients, leveraging the benefits of GNNs in capturing high-order connectivities while enhancing user interaction privacy.

Key Contributions

The paper presents several notable contributions:

• Introduction of VFL to GNN-based Recommender Systems: This is the first work to apply vertical federated learning to GNN-based recommender systems. It addresses the challenge of embedding propagation across different graph segments belonging to various parties without compromising user privacy.
• Privacy-enhanced Framework: The framework combines random projection and ternary quantization to secure the transmission of neighborhood embeddings and gradients, effectively protecting user interaction data against inference attacks.
• Theoretical and Empirical Validation: The paper includes a rigorous theoretical analysis of privacy and communication costs. Empirical results demonstrate that VerFedGNN maintains competitive prediction accuracy compared with existing privacy-preserving GNN frameworks while significantly improving privacy protection.

Problem Statement and Solution

In a vertical federated learning setting, multiple parties possess different sets of attributes for the same set of users. For GNN-based recommender systems, this translates to different item interactions by the same users across various parties. The key problem is how to aggregate the interactions from all parties to enhance recommendation performance while preserving the privacy of user-item interactions.

VerFedGNN addresses this by:

• Utilizing random projection to reduce the dimensionality of neighbor embeddings, which are then securely aggregated across parties.
• Employing a ternary quantization mechanism to perturb gradients of public parameters, ensuring differential privacy in the model training process.

Empirical Results

The empirical evaluation shows that VerFedGNN achieves commendable prediction accuracy close to that of centralized models and superior to other federated approaches. Moreover, a simulated de-anonymization attack demonstrates that VerFedGNN significantly reduces the risk of user interaction data leakage, asserting its prowess in privacy preservation.

Implications and Future Work

VerFedGNN's ability to securely leverage cross-party user-item interactions for recommendation heralds a significant advancement in the creation of privacy-preserving, federated, GNN-based recommender systems. This opens avenues for future research, particularly in exploring the scalability of such frameworks and extending them to other domains beyond recommendation systems. Further investigation into optimizing the trade-offs between privacy, accuracy, and communication costs will also be crucial in making federated GNNs more practical for real-world applications.

In summary, VerFedGNN presents a breakthrough in the intersection of GNNs, federated learning, and recommender systems, paving the way for more secure and efficient collaborative machine learning efforts in privacy-sensitive environments.