Low-Latency Privacy-Preserving Deep Learning Design via Secure MPC

Abstract: Secure multi-party computation (MPC) facilitates privacy-preserving computation between multiple parties without leaking private information. While most secure deep learning techniques utilize MPC operations to achieve feasible privacy-preserving machine learning on downstream tasks, the overhead of the computation and communication still hampers their practical application. This work proposes a low-latency secret-sharing-based MPC design that reduces unnecessary communication rounds during the execution of MPC protocols. We also present a method for improving the computation of commonly used nonlinear functions in deep learning by integrating multivariate multiplication and coalescing different packets into one to maximize network utilization. Our experimental results indicate that our method is effective in a variety of settings, with a speedup in communication latency of $10\sim20\%$.

• The paper proposes a new secret-sharing method that minimizes communication rounds in secure MPC, boosting latency efficiency in deep learning.
• It optimizes nonlinear function computations, using extended n-ary Beaver triples to achieve a 10-20% reduction in communication latency.
• Experimental results with models like LeNet and ResNet-18 validate the practical trade-off between increased data size and lower communication rounds.

Low-Latency Privacy-Preserving Deep Learning Design via Secure MPC

The paper "Low-Latency Privacy-Preserving Deep Learning Design via Secure MPC" addresses a critical issue in secure multi-party computation (MPC): reducing communication latency while preserving data privacy during deep learning operations. Secure MPC allows multiple parties to compute functions over their private inputs without revealing those inputs, which is particularly valuable in privacy-sensitive domains, such as finance and healthcare.

Contributions and Methodological Advancements

The primary contributions of this work can be categorized into three technical advancements:

1. Low-Latency Secret-Sharing-Based Method: The authors propose an efficient method of reducing communication rounds during the MPC protocol execution. This is achieved by employing an optimized multivariate multiplication technique that integrates arithmetic secret sharing with extended n-ary Beaver triples. By doing so, the number of required communication rounds is minimized, exploiting a single round communication for processes that would traditionally require multiple rounds.
2. Enhanced Nonlinear Function Computation: The paper describes a novel approach to handling nonlinear functions, which often present computational challenges in secure MPC settings. They leverage the proposed multivariate multiplication to streamline computations involving commonly used nonlinearities in deep learning, such as exponential and logarithmic functions. This enhancement maximizes network utilization and contributes to a communication latency reduction of 10% to 20%.
3. Experimental Validation and Network Optimization: Experiments were conducted using various neural models, including LinearSVC, LeNet, ResNet-18, and Transformers across multiple datasets. These tests revealed an overall improvement in communication latency, supporting the authors' claims of efficiency and practicality in different network conditions and model complexities.

Technical Insights and Evaluation

The approach implements a practical trade-off between computational complexity and communication efficiency. By extending the traditional Beaver triples method into an n-ary setting, the paper demonstrates a decrease in computational rounds but at the cost of an increase in the size of communication data. Notably, the methodology achieves a balance between these parameters, contributing to overall efficiency.

The paper also tests the boundaries of their approach by conducting experiments under different network conditions (low, medium, and high latency) and investigating the impact of the number of parties involved. These experiments show that while the data size increases, the decrease in communication rounds significantly enhances practical deployment, particularly in high-latency environments.

Moreover, the proposed improvements result in comparable classification accuracy to benchmark implementations but highlight certain inefficiencies that could arise due to precision constraints inherent in multivariate multiplications within finite fields.

Implications and Prospective Development

The implications of this research extend to accelerating the deployment of privacy-preserving deep learning applications in real-world settings. By demonstrating a tangible reduction in communication latency, the proposed method makes secure MPC more feasible for intricate and computationally intensive deep learning tasks.

Looking forward, this foundational work suggests avenues for further refinement of privacy-preserving techniques that could integrate seamlessly with other cryptographic protocols such as Homomorphic Encryption and Garbled Circuits. Future work may explore hybrid models that blend secret-sharing with other secure computation methods to optimize both privacy and performance.

In summary, this research provides meaningful insights into optimizing the latency and throughput of secure MPC executions in deep learning applications. By addressing the communication bottlenecks and improving nonlinear computation efficiency, the paper contributes significantly to the evolving landscape of privacy-preserving artificial intelligence and secure multi-party computations.