Biscotti: A Ledger for Private and Secure Peer-to-Peer Machine Learning (1811.09904v4)

Abstract: Federated Learning is the current state of the art in supporting secure multi-party ML: data is maintained on the owner's device and the updates to the model are aggregated through a secure protocol. However, this process assumes a trusted centralized infrastructure for coordination, and clients must trust that the central service does not use the byproducts of client data. In addition to this, a group of malicious clients could also harm the performance of the model by carrying out a poisoning attack. As a response, we propose Biscotti: a fully decentralized peer to peer (P2P) approach to multi-party ML, which uses blockchain and cryptographic primitives to coordinate a privacy-preserving ML process between peering clients. Our evaluation demonstrates that Biscotti is scalable, fault tolerant, and defends against known attacks. For example, Biscotti is able to protect the privacy of an individual client's update and the performance of the global model at scale when 30% of adversaries are trying to poison the model. The implementation can be found at: https://github.com/DistributedML/Biscotti

• The paper introduces Biscotti, a decentralized system integrating blockchain and cryptography for secure peer-to-peer machine learning.
• The paper employs secure aggregation, differential privacy, and Multi-KRUM to mitigate adversarial poisoning while maintaining model accuracy.
• The paper demonstrates Biscotti’s scalability and fault tolerance, achieving robust convergence even with up to 30% malicious nodes.

Overview of Biscotti: A Ledger for Private and Secure Peer-to-Peer Machine Learning

The paper presents Biscotti, an innovative decentralized system for secure multi-party ML via a peer-to-peer (P2P) architecture. It leverages blockchain and cryptographic methodologies to address privacy and security challenges inherent in federated learning environments.

Motivation

Traditional federated learning consolidates data updates into a central model without exchanging raw data. However, issues such as central authority trust, vulnerability to model poisoning, and data leakage persist. Existing approaches center on anomaly detection, differential privacy with centralized orchestration, or strong consistency protocols unsuitable for ML. Biscotti is proposed as a decentralization strategy using blockchain, ensuring both privacy and resilience against poisoning while eliminating reliance on a centralized authority.

Technical Contributions

1. Decentralized ML Coordination: Biscotti integrates blockchain technology to orchestrate ML tasks securely in a distributed setting. By avoiding a central authority, it mitigates risks linked to trust and potential misuse of aggregated data.
2. Privacy Solutions: Using cryptographic techniques, Biscotti ensures privacy in gradient updates via differential privacy and secure aggregation mechanisms. It integrates these with VRFs to prevent deterministic selection of malicious nodes.
3. Poisoning Resistance: Protection against adversarial attacks is established through the Multi-KRUM algorithm. This allows the system to identify and filter out updates that diverge significantly from others, thus securing model integrity even when a significant adversary subset (up to 30%) is present.
4. Scalability and Fault Tolerance: Evaluations on distributed infrastructures demonstrate Biscotti’s scalability and robustness to node churn, maintaining model convergence despite dynamic peer participation.
5. Use of Proof of Stake: Consensus is achieved without intensive computational proofs (unlike Proof of Work), assigning roles based on a peer's stake, which correlates to their contribution to the model training.

Experimental Results

Extensive evaluation on Azure indicates Biscotti’s resilience to poisoning and privacy attacks while maintaining model accuracy comparable to traditional federated learning. Under rigorous adversarial scenarios, Biscotti's test error approximates that of an unpoisoned federated learning model, demonstrating its robust adversarial resistance.

• Model Performance: Despite higher resource costs compared to standard federated learning, Biscotti achieves the same model accuracy across iterations.
• Attack Robustness: Against a label-flipping adversary representation of 30%, Biscotti effectively neutralizes poisoning attempts.
• Privacy Protection: Secure aggregation prevents significant information leakage from aggregated updates, safeguarding individual data privacy.

Implications and Future Directions

Biscotti signifies a meaningful advance in decentralized ML, highlighting the potential of blockchain to enhance privacy and security in distributed environments. Its design could inspire further exploration into synergy between blockchain technologies and ML applications.

Future research may focus on extending Biscotti to support larger models and optimizing communication efficiency to lower operational costs. Further investigation into handling more complex threat models, such as adaptive attacks and robust stake mechanisms, would also strengthen its applicability in diverse real-world scenarios.