CrypTen: Secure Multi-Party Computation Meets Machine Learning (2109.00984v2)

Abstract: Secure multi-party computation (MPC) allows parties to perform computations on data while keeping that data private. This capability has great potential for machine-learning applications: it facilitates training of machine-learning models on private data sets owned by different parties, evaluation of one party's private model using another party's private data, etc. Although a range of studies implement machine-learning models via secure MPC, such implementations are not yet mainstream. Adoption of secure MPC is hampered by the absence of flexible software frameworks that "speak the language" of machine-learning researchers and engineers. To foster adoption of secure MPC in machine learning, we present CrypTen: a software framework that exposes popular secure MPC primitives via abstractions that are common in modern machine-learning frameworks, such as tensor computations, automatic differentiation, and modular neural networks. This paper describes the design of CrypTen and measure its performance on state-of-the-art models for text classification, speech recognition, and image classification. Our benchmarks show that CrypTen's GPU support and high-performance communication between (an arbitrary number of) parties allows it to perform efficient private evaluation of modern machine-learning models under a semi-honest threat model. For example, two parties using CrypTen can securely predict phonemes in speech recordings using Wav2Letter faster than real-time. We hope that CrypTen will spur adoption of secure MPC in the machine-learning community.

• The paper introduces CrypTen, a framework that integrates secure multi-party computation with machine learning for privacy-preserving operations.
• It employs arithmetic and binary secret sharing techniques with secure conversion protocols to support complex ML tasks without data leakage.
• CrypTen leverages methods like Beaver triples and polynomial approximations to ensure computational accuracy and robust security.

Overview of "CrypTen: Secure Multi-Party Computation Meets Machine Learning"

The paper presents CrypTen, a framework dedicated to integrating secure multi-party computation (MPC) with machine learning tasks. The CrypTen system orchestrates a suite of techniques aimed at enabling secure computations on machine learning models, ensuring data confidentiality across multiple parties without compromising the functionality of the underlying models.

Core Contributions and Techniques

The framework employs various secure computations, primarily capitalizing on two secret-sharing methodologies: arithmetic and binary secret sharing. These parallel mechanisms provide a foundation for CrypTen to efficiently run complex machine learning operations securely.

Arithmetic Secret Sharing

In arithmetic secret sharing, scalar values are divided into shares across parties in such a way that only the sum of these shares reconstructs the original value. A significant utility of this method within CrypTen is its support for operations like addition, multiplication using Beaver triples, square computation, and truncation. Importantly, techniques such as this emphasize security by preventing information leakage, even during intermediate computations. The truncation procedure is particularly noteworthy for handling division operations by accommodating the number of wraps, thereby preserving arithmetic correctness in secure computations.

Binary Secret Sharing

The framework further highlights binary secret sharing, operating in the field of binary operations where computations like XOR and AND are employed. This form of secret sharing is tightly integrated with logic-based operations, offering implementations for comparison functions and multiplexing operations widely used in decision-making tasks within ML models. CrypTen leverages this for operations like argmax and bitwise manipulations, integral for facilitating complex neural network evaluations.

Conversion Between Share Types

Given the dual secret-sharing approach, CrypTen embeds mechanisms to convert between arithmetic and binary shares seamlessly. Conversions utilize established protocols such as those found in \cite{demmler2015aby}, to bridge arithmetic and binary share types while maintaining computational integrity and security.

Mathematical Approximations

CrypTen introduces mathematical approximation techniques targeting functions otherwise computationally intensive within secret-sharing schemes. These include polynomial approximations for exponential functions and Newton-Raphson iterations for computing roots and reciprocals. The approximations are optimized for different domains and are integral to the performance scaling of CrypTen for real-world ML tasks.

Functionality and Implementation

An extensive overview of implemented functions within CrypTen underscores its broad applicability in ML environments. Functions range from basic arithmetic operations to complex functions like convolutions, dropout layers, and batch normalization crucial for neural networks. Each function is meticulously designed to accommodate CrypTen's secure computation model, providing seamless integration with secure algorithms without the need for plaintext data exposure.

Security Discussion

CrypTen’s security guarantees are founded on a rigorous set of security proofs, ensuring the confidentiality of operations. For example, arithmetic operations rely on the security of ring computations, while binary operations leverage the established GMW protocol. The conversion protocols between share types are proved to maintain secrecy, thus allowing the construction of complex secure operations via compositional security.

Implications and Future Directions

The integration of secure computation methodologies with ML frameworks in CrypTen holds substantial promise for privacy-preserving AI applications. It facilitates the deployment of ML models across sensitive domains like healthcare and finance, where data confidentiality is paramount. Future expansions could focus on enhancing scalability and efficiency, particularly for high-dimensional data and deep networks. Moreover, exploring broader algorithmic support under the MPC paradigm may widen CrypTen's applicability and streamline secure AI model deployment in various real-world scenarios.