FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learning (2004.02229v2)

Abstract: We propose Falcon, an end-to-end 3-party protocol for efficient private training and inference of large machine learning models. Falcon presents four main advantages - (i) It is highly expressive with support for high capacity networks such as VGG16 (ii) it supports batch normalization which is important for training complex networks such as AlexNet (iii) Falcon guarantees security with abort against malicious adversaries, assuming an honest majority (iv) Lastly, Falcon presents new theoretical insights for protocol design that make it highly efficient and allow it to outperform existing secure deep learning solutions. Compared to prior art for private inference, we are about 8x faster than SecureNN (PETS'19) on average and comparable to ABY3 (CCS'18). We are about 16-200x more communication efficient than either of these. For private training, we are about 6x faster than SecureNN, 4.4x faster than ABY3 and about 2-60x more communication efficient. Our experiments in the WAN setting show that over large networks and datasets, compute operations dominate the overall latency of MPC, as opposed to the communication.

• The paper introduces a maliciously secure three-party protocol that enables private deep learning with 6x to 200x improved communication efficiency over previous methods.
• It employs novel arithmetic techniques for non-linear computations, offering full support for batch normalization during training and inference.
• The end-to-end implementation is evaluated on complex networks like VGG16 and AlexNet, demonstrating practical efficacy for real-world secure computation.

Honest-Majority Maliciously Secure Framework for Private Deep Learning

The research paper presents a comprehensive framework for achieving private training and inference of complex machine learning models using a maliciously secure, three-party protocol. The authors introduce a novel system designed with several key advantages including expressiveness, support for batch normalization, and efficiency in computation and communication resources.

The cornerstone of the framework is a secure multi-party computation (MPC) protocol tailored for high-capacity neural networks like VGG16 and AlexNet. Fundamentally, the system ensures security under a model where a majority among the three computing parties are assumed to be honest, which is prevalent in contemporary MPC research to enhance protocol efficiency and practicality in real-world deployments. It significantly outperforms previous methods regarding computational speed and communication efficiency. The framework, when compared to preceding solutions such as SecureNN and ABY$^3$, provides a 6x to 200x improvement in communication efficiency and is also markedly faster.

Core Contributions and Innovations

The authors have made several critical contributions to the secure computation landscape in the context of machine learning:

1. Malicious Security: The framework achieves robust security guarantees, withstanding malicious adversaries who may deviate from protocol while ensuring correctness with abort. This contrasts with earlier methods that often only provided semi-honest security, vulnerable to attacks where adversaries do not follow protocols.
2. Enhanced Protocol Efficiency: Integrating techniques from existing MPC frameworks, the authors devise novel protocols that significantly reduce overhead. Notably, the introduced techniques for non-linear computations like ReLU and derivative of ReLU boast twice the efficiency due to enhanced arithmetic methods, a marked improvement over existing practices.
3. Expressiveness: Critical to machine learning training, batch normalization is fully supported by the framework, marking the first such implementation for both forward and backward passes in a fully private setting. The framework allows the training and inference of large-scale networks, thus demonstrating its wide-ranging applicability and expressiveness.
4. End-to-End Implementation: The paper describes a fully implemented solution evaluated on several datasets and architectures, showcasing the practical utility of the approach. The system is tested on six diverse networks including the challenging VGG16 and AlexNet architectures.

Theoretical and Practical Implications

The implications of this framework are notable both in theory and in practice:

• Performance Optimizations: The optimized protocols that reduce both round complexity and data exchange mark substantial advancements in MPC that are applicable beyond just machine learning, suggesting potential improvements for secure computations in other domains.
• Deployment Scenarios: The focus on both LAN and WAN settings, as well as considerations for real-world adversarial models, makes the solution viable for distributed computation across varied network environments, a crucial step for broader deployment in sensitive applications like health data aggregation and social media moderation.
• Future Directions: The work sets the stage for further research into optimizing compute operations within secure computation, given the identified dominance of computation over communication in resource demand as models scale. This insight directs future explorations into hardware accelerations like GPUs or improved computational paradigms.

By providing a robust, efficient, and expressive framework, the authors significantly advance the capabilities of private deep learning. Future studies will likely build upon these findings, potentially leading to even more efficient cryptographic protocols and broader applications in secure, privacy-preserving computations.