Not Just Privacy: Improving Performance of Private Deep Learning in Mobile Cloud (1809.03428v3)

Abstract: The increasing demand for on-device deep learning services calls for a highly efficient manner to deploy deep neural networks (DNNs) on mobile devices with limited capacity. The cloud-based solution is a promising approach to enabling deep learning applications on mobile devices where the large portions of a DNN are offloaded to the cloud. However, revealing data to the cloud leads to potential privacy risk. To benefit from the cloud data center without the privacy risk, we design, evaluate, and implement a cloud-based framework ARDEN which partitions the DNN across mobile devices and cloud data centers. A simple data transformation is performed on the mobile device, while the resource-hungry training and the complex inference rely on the cloud data center. To protect the sensitive information, a lightweight privacy-preserving mechanism consisting of arbitrary data nullification and random noise addition is introduced, which provides strong privacy guarantee. A rigorous privacy budget analysis is given. Nonetheless, the private perturbation to the original data inevitably has a negative impact on the performance of further inference on the cloud side. To mitigate this influence, we propose a noisy training method to enhance the cloud-side network robustness to perturbed data. Through the sophisticated design, ARDEN can not only preserve privacy but also improve the inference performance. To validate the proposed ARDEN, a series of experiments based on three image datasets and a real mobile application are conducted. The experimental results demonstrate the effectiveness of ARDEN. Finally, we implement ARDEN on a demo system to verify its practicality.

• The paper introduces a framework that partitions deep neural networks between mobile devices and cloud servers to balance privacy with computational constraints.
• It implements differential privacy on mobile devices with noise injection and noisy training on the cloud to mitigate inference quality loss.
• Experiments on image datasets and a live mobile application demonstrate substantial improvements in privacy preservation and energy efficiency.

Improving Performance of Private Deep Learning in Mobile Cloud

The paper conducted by Wang et al. investigates the challenge of deploying deep neural network (DNN) models for private inference on mobile devices while leveraging the computational capabilities of cloud data centers. The authors present a framework, namely \system, which partitions a DNN between a mobile device and a cloud server, aiming to improve privacy without sacrificing performance. This partitioning approach address the computational limits of mobile devices and reduces associated energy consumption.

The paper outlines a two-tiered strategy. At the mobile level, a shallow segment of a pre-trained DNN is maintained. This segment performs initial data transformation and feature extraction. To safeguard user data privacy, a differential privacy mechanism, utilizing data nullification and the injection of random noise, is implemented. The framework then rigorously assesses the privacy budget, providing a mathematical guarantee of privacy under the differential privacy paradigm, commonly characterized by the parameter $\varepsilon$.

Despite preserving privacy, the random perturbations of data adversely affect inference quality on the cloud end. Wang et al. counter this through a specially designed noisy training scheme for the deep portion of the DNN hosted in the cloud. This approach involves training the network with varying degrees of noise embedded in input data, fostering robustness to perturbed input data encountered during live inference.

The researchers validated \system\ across standard image datasets and a live mobile application, where the experimental results demonstrate substantial effectiveness in preserving privacy while minimizing performance loss. The findings carry significant theoretical and practical implications, primarily by illustrating how DNN inference can be efficiently deployed in mobile cloud settings without relinquishing data privacy.

This work presents several avenues for future research. Adjusting the model to cater more effectively to diverse datasets or tasks, refining differential privacy constraints, and further optimizing for mobile hardware capabilities are potential areas of exploration. As the field advances, incorporating advanced architectures within the \system\ framework could prospectively achieve improved performance metrics under resource and privacy constraints.