nGraph-HE2: A High-Throughput Framework for Neural Network Inference on Encrypted Data (1908.04172v2)

Abstract: In previous work, Boemer et al. introduced nGraph-HE, an extension to the Intel nGraph deep learning (DL) compiler, that enables data scientists to deploy models with popular frameworks such as TensorFlow and PyTorch with minimal code changes. However, the class of supported models was limited to relatively shallow networks with polynomial activations. Here, we introduce nGraph-HE2, which extends nGraph-HE to enable privacy-preserving inference on standard, pre-trained models using their native activation functions and number fields (typically real numbers). The proposed framework leverages the CKKS scheme, whose support for real numbers is friendly to data science, and a client-aided model using a two-party approach to compute activation functions. We first present CKKS-specific optimizations, enabling a 3x-88x runtime speedup for scalar encoding, and doubling the throughput through a novel use of CKKS plaintext packing into complex numbers. Second, we optimize ciphertext-plaintext addition and multiplication, yielding 2.6x-4.2x runtime speedup. Third, we exploit two graph-level optimizations: lazy rescaling and depth-aware encoding, which allow us to significantly improve performance. Together, these optimizations enable state-of-the-art throughput of 1,998 images/s on the CryptoNets network. Using the client-aided model, we also present homomorphic evaluation of (to our knowledge) the largest network to date, namely, pre-trained MobileNetV2 models on the ImageNet dataset, with 60.4\percent/82.7\percent\ top-1/top-5 accuracy and an amortized runtime of 381 ms/image.

• The paper introduces nGraph-HE2, a framework that enhances encrypted neural network inference with CKKS, achieving runtime improvements from 3x to 88x.
• Key optimizations such as CKKS plaintext packing, lazy rescaling, and depth-aware encoding reduce computational overhead and memory usage.
• The framework supports standard pre-trained models like CryptoNets and MobileNetV2, demonstrating practical impact in privacy-preserving machine learning.

A High-Throughput Framework for Neural Network Inference on Encrypted Data

This paper introduces nGraph-HE2, a framework designed to facilitate neural network inference on encrypted data while preserving data privacy. The authors extend their previous work, nGraph-HE, which enabled inference using shallow neural networks with polynomial activations. The novel framework, nGraph-HE2, utilizes the CKKS homomorphic encryption scheme, focusing on enhancing compatibility with standard pre-trained models that use native activation functions and real number fields, addressing limitations in earlier renditions of homomorphic encryption models.

Key contributions of the paper include several optimizations crafted to enhance the performance of encrypted computations. The authors present CKKS-specific optimizations, showcasing a significant runtime improvement by a factor of 3x to 88x for scalar encoding. These optimizations also double throughput using a novel approach of CKKS plaintext packing into complex numbers. Furthermore, improvements in ciphertext-plaintext addition and multiplication increase runtime efficiency by factors ranging from 2.6x to 4.2x.

Another notable advancement is the introduction of graph-level optimizations, namely lazy rescaling and depth-aware encoding, which further improve performance. Specifically, lazy rescaling significantly reduces computation time by postponing rescaling operations to minimize their frequency, and depth-aware encoding strategically encodes values at appropriate levels to reduce memory use.

The implications of this research are extensive, particularly for fields requiring enhanced data privacy during machine learning inference tasks. The authors demonstrate state-of-the-art throughput with the CryptoNets network, achieving a throughput of 1998 images per second. Moreover, the framework achieves a remarkable first: enabling homomorphic evaluation of the MobileNetV2 network on the ImageNet dataset. The network delivers 60.4% top-1 and 82.7% top-5 accuracy with an under-400ms amortized runtime per image.

The practical implementations of this work highlight the utility of homomorphic encryption in real-world scenarios where privacy is paramount. Moreover, nGraph-HE2 significantly lowers the computational barrier for deploying deep learning models on encrypted data by supporting standard data science workflows without specialized cryptographic knowledge. This paper serves as a promising catalyst for advancing privacy-preserving machine learning methodologies.

Future research should address the challenges related to the secure computation of non-linear network activations. While the current framework employs a placeholder client-aided model, methods leveraging cryptographic primitives, like garbled circuits or trusted execution environments, could enhance security assurances for secure protocol implementation. Additionally, the work opens potential for further refining the scalar encoding optimization for complex numbers and leveraging advanced computational instructions like Intel AVX to optimize plaintext-ciphertext operations. With these enhancements, the framework’s applicability and efficiency in various domains could be notably broadened.