Differentially Private Testing of Identity and Closeness of Discrete Distributions

Abstract: We study the fundamental problems of identity testing (goodness of fit), and closeness testing (two sample test) of distributions over $k$ elements, under differential privacy. While the problems have a long history in statistics, finite sample bounds for these problems have only been established recently. In this work, we derive upper and lower bounds on the sample complexity of both the problems under $(\varepsilon, \delta)$-differential privacy. We provide optimal sample complexity algorithms for identity testing problem for all parameter ranges, and the first results for closeness testing. Our closeness testing bounds are optimal in the sparse regime where the number of samples is at most $k$. Our upper bounds are obtained by privatizing non-private estimators for these problems. The non-private estimators are chosen to have small sensitivity. We propose a general framework to establish lower bounds on the sample complexity of statistical tasks under differential privacy. We show a bound on differentially private algorithms in terms of a coupling between the two hypothesis classes we aim to test. By constructing carefully chosen priors over the hypothesis classes, and using Le Cam's two point theorem we provide a general mechanism for proving lower bounds. We believe that the framework can be used to obtain strong lower bounds for other statistical tasks under privacy.

• The paper establishes optimal sample complexity algorithms for identity testing and introduces first-time bounds for closeness testing under differential privacy.
• It employs a low-sensitivity privatization approach using a novel coupling framework and Le Cam’s two-point method to derive tight lower bounds.
• The results offer practical strategies for privacy-preserving data analysis in sensitive domains, enhancing statistical testing and survey methodologies.

Differentially Private Testing of Identity and Closeness of Discrete Distributions

The paper on "Differentially Private Testing of Identity and Closeness of Discrete Distributions" focuses on establishing the sample complexity for two fundamental problems in statistical hypothesis testing under the constraints of differential privacy. The first problem is identity testing, where the goal is to determine if a given discrete distribution is equal to a known distribution, or if it lies at a distance greater than a specified threshold. The second problem, closeness testing, involves determining if two unknown discrete distributions are identical or if they differ by a specified threshold in total variation distance. These problems bear significance in the statistical community both for theoretical development and in practical applications where privacy concerns are paramount, such as surveys involving sensitive information.

Sample Complexity Bounds

The authors rigorously derive the sample complexity bounds for both identity and closeness testing, providing precise upper and lower bounds. Their contribution reveals optimal sample complexity algorithms for identity testing across all parameter ranges and presents first-time results for closeness testing. A noteworthy result concerning closeness testing is the optimal bound established in the sparse regime, where the number of samples is at most the domain size $k$.

Their approach successfully privatizes non-private estimators by ensuring these estimators have low sensitivity, which is crucial for maintaining differential privacy. Furthermore, for establishing lower bounds under differential privacy, the authors introduce a general framework utilizing a coupling technique and Le Cam’s two-point theorem—a powerful method in statistical hypothesis testing. By constructing carefully chosen priors over the hypothesis classes, this framework serves as a tool to potentially prove strong lower bounds for other statistical tasks under privacy constraints.

Numerical Results

The paper provides sample complexity bounds that are tight up to constant factors across various privacy regimes. For identity testing, the results reveal that the complexity varies depending on the relationship between sample size and domain size. Specifically, the complexity ranges from $$\Theta(\frac{\sqrt{k}}{\alpha^2} + \frac{k^{1/2}}{\alpha(+\delta)^{1/2}})$$ in sparse settings to $$\Theta(\frac{\sqrt{k}}{\alpha^2} + \frac{1}{\alpha(+\delta)})$$ when sample size is large.

For closeness testing, the paper delineates the sample complexity as $$\Theta( \frac{k^{2/3}}{\alpha^{4/3}} + \frac{\sqrt{k}}{\alpha(+\delta)^{1/2}})$$ when sufficient sample size can be maintained relative to domain size constraints. These results are crucial for statisticians designing differentially private protocols that balance data utility with privacy requirements.

Implications and Speculation on Future Developments

The implications of these findings are profound, impacting multiple spheres where differential privacy is relevant. Practical implications include enhanced privacy-preserving methodologies for data release and statistical analysis in sensitive domains. The theoretical implications extend to potentially refining privacy-preserving algorithms for a wide array of statistical tasks beyond hypothesis testing.

The framework proposed by the paper is versatile and can be generalized for proving lower bounds for other types of estimation and hypothesis testing problems under differential privacy. Such advancements in understanding the trade-offs between privacy and data utility are crucial as the demand for privacy-centric solutions intensifies across various sectors.

Future development in AI could see the integration of these rigorous statistical methodologies into machine learning environments, particularly those relying on large datasets where privacy preservation is critical. Furthermore, research could build upon these foundational results to explore adaptive privacy mechanisms that dynamically adjust privacy levels based on the sensitivity of the statistical task at hand, further enriching the landscape of privacy-preserving data analysis.

In conclusion, this paper sets a significant milestone in adopting differential privacy in testing discrete distributions by providing a combination of optimal algorithms, sample bounds, and a framework for continuous improvement in privacy analysis methodologies.