Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10.12 (1709.02753v2)

Abstract: In June 2016, Apple announced that it will deploy differential privacy for some user data collection in order to ensure privacy of user data, even from Apple. The details of Apple's approach remained sparse. Although several patents have since appeared hinting at the algorithms that may be used to achieve differential privacy, they did not include a precise explanation of the approach taken to privacy parameter choice. Such choice and the overall approach to privacy budget use and management are key questions for understanding the privacy protections provided by any deployment of differential privacy. In this work, through a combination of experiments, static and dynamic code analysis of macOS Sierra (Version 10.12) implementation, we shed light on the choices Apple made for privacy budget management. We discover and describe Apple's set-up for differentially private data processing, including the overall data pipeline, the parameters used for differentially private perturbation of each piece of data, and the frequency with which such data is sent to Apple's servers. We find that although Apple's deployment ensures that the (differential) privacy loss per each datum submitted to its servers is $1$ or $2$, the overall privacy loss permitted by the system is significantly higher, as high as $16$ per day for the four initially announced applications of Emojis, New words, Deeplinks and Lookup Hints. Furthermore, Apple renews the privacy budget available every day, which leads to a possible privacy loss of 16 times the number of days since user opt-in to differentially private data collection for those four applications. We advocate that in order to claim the full benefits of differentially private data collection, Apple must give full transparency of its implementation, enable user choice in areas related to privacy loss, and set meaningful defaults on the privacy loss permitted.

• The paper analyzes Apple's DP implementation using experiments, code reviews, and configuration file examinations.
• It reveals that while per-datum privacy loss is modest (about 1–2 units), the daily cumulative loss may reach up to 16 units.
• The study advocates for greater transparency and user control to bridge the gap between theoretical DP and its practical deployment.

Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10.12

The paper "Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10.12" presents an in-depth analysis of Apple's deployment of differential privacy (DP) in macOS Sierra (Version 10.12). Apple, a leading technology enterprise, introduced differential privacy to enhance data privacy for its users. While the concept of differential privacy is theoretically sound, this paper critically examines the nuances and trade-offs Apple made in practice and the potential risks associated with their implementation.

The paper uses a combination of experimental methods, code analysis, and configuration file reviews to unveil the parameters and mechanisms behind Apple's DP strategy. It reveals that, although Apple's implementation ensured that the differential privacy loss per datum remains modest (approximately 1 to 2 per submission), the cumulative privacy loss permitted by Apple's system per day is substantially higher, potentially reaching 16 units for primary applications like Emojis, New Words, Deeplinks, and Lookup Hints. This daily renewal of privacy budgets is noteworthy as it can result in substantial privacy loss over extended periods of user engagement.

A keen observation from the paper is the lack of transparency, both in the algorithms' specifics and in privacy parameter choices by Apple. Such transparency is critical for stakeholders, including consumers, researchers, and privacy advocates, to evaluate the genuine privacy benefits provided by Apple's system against theoretical DP guarantees. The authors call for Apple to disclose its privacy implementations and offer choices to users regarding their data's privacy parameters, thus allowing users to consent to their data's usage more informedly.

From a practical and theoretical perspective, this paper accentuates the existing gap between the theoretical definitions of differential privacy and the real-world implementations by corporations. The findings press the importance of transparent communication from data collectors regarding privacy metrics and asserting user control over personal data privacy. Moreover, it highlights potential areas for future improvement and research, such as the design of systems that can better balance privacy and utility, enhancement of user interfaces for comprehensible privacy metric presentation, and development of strategies to mitigate cumulative privacy losses over time.

In summary, the paper offers crucial insights for practitioners and theoreticians keen on understanding the practical implications of deploying differential privacy in commercial systems. While Apple's efforts are commendable for pushing privacy boundaries in industry settings, clear documentation and user autonomy in their privacy practices remain essential to claim the full benefits of differential privacy. Future developments could involve refinement of privacy budgeting frameworks and increased collaboration between academia and industry to enhance the security, transparency, and effectiveness of privacy-preserving technologies.