The Discrete Gaussian for Differential Privacy (2004.00010v6)

Abstract: A key tool for building differentially private systems is adding Gaussian noise to the output of a function evaluated on a sensitive dataset. Unfortunately, using a continuous distribution presents several practical challenges. First and foremost, finite computers cannot exactly represent samples from continuous distributions, and previous work has demonstrated that seemingly innocuous numerical errors can entirely destroy privacy. Moreover, when the underlying data is itself discrete (e.g., population counts), adding continuous noise makes the result less interpretable. With these shortcomings in mind, we introduce and analyze the discrete Gaussian in the context of differential privacy. Specifically, we theoretically and experimentally show that adding discrete Gaussian noise provides essentially the same privacy and accuracy guarantees as the addition of continuous Gaussian noise. We also present an simple and efficient algorithm for exact sampling from this distribution. This demonstrates its applicability for privately answering counting queries, or more generally, low-sensitivity integer-valued queries.

• The paper establishes a rigorous theoretical foundation for using the discrete Gaussian to achieve concentrated differential privacy with high computational accuracy.
• It provides a detailed privacy analysis demonstrating that discrete Gaussian noise delivers comparable (ε,δ)-guarantees to continuous methods while maintaining exact digital representation.
• It presents an efficient rejection sampling algorithm that improves practical implementations and utility, especially for low sensitivity queries.

Essay on "The Discrete Gaussian for Differential Privacy"

The necessity of implementing differential privacy using noise perturbation of output data has been a topic of significant interest. Commonly, noise drawn from continuous distributions such as Gaussian or Laplace distributions is added to ensure privacy. However, these continuous distributions present challenges, notably due to the inherent limitations of computers in representing real numbers and potential privacy breaches through numerical inaccuracies. In the paper titled "The Discrete Gaussian for Differential Privacy," the authors propose and investigate the use of the discrete Gaussian distribution as a tool for achieving differential privacy, offering a compelling alternative that alleviates these issues while delivering competitive privacy and utility guarantees.

Main Contributions

1. Theoretical Framework: The discrete Gaussian distribution is proposed as the discrete analogue of the continuous Gaussian. The authors provide definitions and establish a rigorous theoretical foundation for its application in differential privacy contexts. The discretized form is shown to retain similar privacy guarantees, specifically concentrated differential privacy (CDP), as its continuous counterpart, while being inherently suitable for use with discrete-valued data.
2. Privacy Analysis: Detailed analysis demonstrates that adding noise from a discrete Gaussian $0{\sigma^2}$ ensures $\frac12 \varepsilon^2$-concentrated differential privacy under certain scaling conditions. The analysis is extended to approximate differential privacy (ADP), providing tight bounds on achievable $(\varepsilon,\delta)$ guarantees. The results confirm that the discrete Gaussian provides nearly equivalent privacy protection as the continuous Gaussian, with the added benefit of exact representation on digital hardware.
3. Sampling Methodology: The authors introduce a practical and efficient algorithm for exact sampling from the discrete Gaussian distribution, leveraging a rejection sampling-based method. This algorithm circumvents the inaccuracies associated with floating-point approximations, making it particularly suitable for privacy-preserving computations on finite-precision systems.
4. Utility Assessment: The utility of using the discrete Gaussian is quantitatively backed by variance and tail-bound analyses. It is shown that the variance and sub-Gaussian tail properties of the discrete Gaussian are comparable to the continuous Gaussian. Notably, for low sensitivity queries, such as counting queries, the discrete Gaussian often exhibits superior utility than the rounded Gaussian alternative.
5. Comparison with Discrete Laplace: The work draws a comparative analysis between the discrete Gaussian and the discrete Laplace distributions. While both satisfy discrete privacy models, the Gaussian is aligned with concentrated differential privacy and demonstrates more favorable behavior in high composition settings where the additive noise is iterated multiple times.

Implications and Future Directions

The introduction of the discrete Gaussian distribution into the arsenal of differential privacy mechanisms is noteworthy due to its operational advantages over continuous distributions, primarily indicated by its suitability for integer-based computations and its strong alignment with digital data representations. This work has potential implications for the broader deployment of privacy-preserving measures in contexts where computational accuracy and security are paramount, such as in sensitive data management in census operations and real-time data analytics.

Moving forward, the discrete Gaussian provides a foundation for numerous applications and extensions. Potential avenues include further optimization of sampling algorithms to minimize computational overhead or deeper investigation into mixed data types, particularly blending continuous models with discrete distributions to handle more complex queries. Moreover, exploring alternative privacy guarantees, algorithmic compositions, or deployment strategies in multi-user environments could further broaden the applicability of these foundational results.

In conclusion, while meeting contemporary needs for privacy, the discrete Gaussian distribution provides an efficient and theoretically robust mechanism for differential privacy that promises to enhance the accuracy and reliability of privacy-preserving systems in practice. The work positions the discrete Gaussian as a central component of differential privacy, worthy of attention for both theoretical exploration and practical implementation.