Simple Binary Hypothesis Testing under Local Differential Privacy and Communication Constraints (2301.03566v2)
Abstract: We study simple binary hypothesis testing under both local differential privacy (LDP) and communication constraints. We qualify our results as either minimax optimal or instance optimal: the former hold for the set of distribution pairs with prescribed Hellinger divergence and total variation distance, whereas the latter hold for specific distribution pairs. For the sample complexity of simple hypothesis testing under pure LDP constraints, we establish instance-optimal bounds for distributions with binary support; minimax-optimal bounds for general distributions; and (approximately) instance-optimal, computationally efficient algorithms for general distributions. When both privacy and communication constraints are present, we develop instance-optimal, computationally efficient algorithms that achieve the minimum possible sample complexity (up to universal constants). Our results on instance-optimal algorithms hinge on identifying the extreme points of the joint range set $\mathcal A$ of two distributions $p$ and $q$, defined as $\mathcal A := {(\mathbf T p, \mathbf T q) | \mathbf T \in \mathcal C}$, where $\mathcal C$ is the set of channels characterizing the constraints.
- “Hypothesis testing with communication constraints” In IEEE Transactions on Information Theory 32.4, 1986, pp. 533–542
- “Inference under information constraints III: Local privacy constraints” In IEEE Journal on Selected Areas in Information Theory 2.1 IEEE, 2021, pp. 253–267
- “Interactive Inference Under Information Constraints” In IEEE Transactions on Information Theory 68.1, 2022, pp. 502–516
- J. Acharya, C.L. Canonne and H. Tyagi “Inference Under Information Constraints I: Lower Bounds From Chi-Square Contraction” In IEEE Transactions on Information Theory 66.12, 2020 DOI: 10.1109/TIT.2020.3028440
- J. Acharya, C.L. Canonne and H. Tyagi “Inference Under Information Constraints II: Communication Constraints and Shared Randomness” In IEEE Transactions on Information Theory 66.12, 2020 DOI: 10.1109/TIT.2020.3028439
- H. Asi, V. Feldman and K. Talwar “Optimal Algorithms for Mean Estimation under Local Differential Privacy” In Proc. 39th International Conference on Machine Learning (ICML), 2022
- “Statistical inference under multiterminal data compression” In IEEE Transactions on Information Theory 44.6, 1998, pp. 2300–2324
- “Contraction of Locally Differentially Private Mechanisms” In CoRR abs/2210.13386, 2022
- L.P. Barnes, W-N. Chen and A. Özgür “Fisher information under local differential privacy” In IEEE Journal on Selected Areas in Information Theory 1.3 IEEE, 2020, pp. 645–659
- “Prochlo: Strong Privacy for Analytics in the Crowd” In Proc. of the 26th Symposium on Operating Systems Principles, 2017
- T. Berger “Decentralized estimation and decision theory” In IEEE Seven Springs Workshop on Information Theory, 1979
- “Communication Lower Bounds for Statistical Estimation Problems via a Distributed Data Processing Inequality” In Proc. 49th Annual ACM Symposium on Theory of Computing (STOC), 2016 DOI: 10.1145/2897518.2897582
- M. Braverman, S. Garg and O. Zamir “Tight space complexity of the coin problem” In Proc. 62nd IEEE Symposium on Foundations of Computer Science (FOCS), 2022
- L.P. Barnes, Y. Han and A. Özgür “Lower bounds for learning distributions under communication constraints via Fisher information” In Journal of Machine Learning Research 21.1, 2020, pp. 9583–9612
- “Private Hypothesis Selection” In Advances in Neural Information Processing Systems 32 (NeurIPS), 2019
- “Information-Distilling Quantizers” In IEEE Transactions on Information Theory 67.4, 2021, pp. 2472–2487
- T. Berg, O. Ordentlich and O. Shayevitz “Binary Hypothesis Testing with Deterministic Finite-Memory Decision Rules” In Proc. 2020 IEEE International Symposium on Information Theory, 2020 DOI: 10.1109/ISIT44484.2020.9174505
- “Concentrated differential privacy: Simplifications, extensions, and lower bounds” In Theory of Cryptography Conference, 2016
- “Introduction to linear optimization” Athena Scientific, 1997
- L.L. Cam “Asymptotic Methods in Statistical Decision Theory”, Springer Series in Statistics New York, NY: Springer New York, 1986
- F. Carpi, S. Garg and E. Erkip “Single-shot compression for hypothesis testing” In 2021 IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), 2021, pp. 176–180 IEEE
- “The Structure of Optimal Private Tests for Simple Hypotheses” In Proc. 51st Annual ACM Symposium on Theory of Computing (STOC), 2019 DOI: 10.1145/3313276.3316336
- W.-N. Chen, P. Kairouz and A. Ozgur “Pointwise Bounds for Distribution Estimation under Communication Constraints” In Advances in Neural Information Processing Systems 34 (NeurIPS), 2021
- T.M. Cover “Hypothesis Testing with Finite Statistics” In The Annals of Mathematical Statistics 40.3, 1969, pp. 828–835 DOI: 10.1214/aoms/1177697590
- “Distributed Differential Privacy via Shuffling” In Advances in Cryptology – EUROCRYPT 2019, 2019
- “Differential privacy as a mutual information constraint” In Proc. 2016 ACM SIGSAC Conference on Computer and Communications Security, 2016, pp. 43–54
- J.C. Duchi, M.I. Jordan and M.J. Wainwright “Minimax Optimal Procedures for Locally Private Estimation” In Journal of the American Statistical Association 113.521, 2018, pp. 182–201 DOI: 10.1080/01621459.2017.1389735
- “Optimality Guarantees for Distributed Statistical Estimation” In CoRR abs/1405.0782, 2014
- “Streaming Algorithms for High-Dimensional Robust Statistics” In Proc. 39th International Conference on Machine Learning (ICML), 2022
- “The Algorithmic Foundations of Differential Privacy” In Foundations and Trends® in Theoretical Computer Science 9.3-4, 2013, pp. 211–407 DOI: 10.1561/0400000042
- “Lower Bounds for Locally Private Estimation via Communication Complexity” In Proc. 32nd Annual Conference on Learning Theory (COLT), 2019
- “Rényi Divergence and Kullback-Leibler Divergence” In IEEE Transactions on Information Theory 60.7, 2014, pp. 3797–3820
- “Extreme points of the (0,δ)0𝛿(0,\delta)( 0 , italic_δ )-differential privacy polytope” In Working paper, 2023
- V. Feldman, A. McMillan and K. Talwar “Hiding Among the Clones: A Simple and Nearly Optimal Analysis of Privacy Amplification by Shuffling” In Proc. 62nd IEEE Symposium on Foundations of Computer Science (FOCS), 2021
- “Deep Learning with Label Differential Privacy” In Advances in Neural Information Processing Systems 34 (NeurIPS), 2021
- “Locally Private Hypothesis Selection” In Proc. 33rd Annual Conference on Learning Theory (COLT), 2020
- “On memory saved by randomization” In The Annals of Mathematical Statistics JSTOR, 1971, pp. 1075–1078
- “A Review of Recent Results on Learning with Finite Memory” In International Symposium on Information Theory (ISIT), 1973, pp. 289–294
- M. Hellman “Finite-memory algorithms for estimating the mean of a Gaussian distribution” In IEEE Transactions on Information Theory 20.3, 1974, pp. 382–384
- N. Holohan, D.J. Leith and O. Mason “Extreme Points of the Local Differential Privacy Polytope” In Linear Algebra and its Applications 534, 2017, pp. 78–96
- “The Role of Interactivity in Local Differential Privacy” In Proc. 60th IEEE Symposium on Foundations of Computer Science (FOCS), 2019
- P. Kairouz, S. Oh and P. Viswanath “Extremal Mechanisms for Local Differential Privacy” In Journal of Machine Learning Research 17, 2016, pp. 17:1–17:51
- “Estimating a probability using finite memory” In IEEE Transactions on Information Theory 32.6, 1986, pp. 733–742 DOI: 10.1109/TIT.1986.1057250
- “Hypothesis testing under maximal leakage privacy constraints” In Proc. 2017 IEEE International Symposium on Information Theory, 2017
- “Hypothesis testing under mutual information privacy constraints in the high privacy regime” In IEEE Transactions on Information Forensics and Security 13.4 IEEE, 2017
- I. Mironov “Rényi differential privacy” In Proc. 2017 IEEE 30th Computer Security Foundations Symposium (CSF), 2017 IEEE
- “On the Problem of the Most Efficient Tests of Statistical Hypotheses” In Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character 231, 1933, pp. 289–337
- “Simple binary hypothesis testing under local differential privacy and communication constraints” In Proc. 36th Annual Conference on Learning Theory (COLT), 2023
- A. Pensia, V. Jog and P. Loh “Communication-constrained hypothesis testing: Optimality, robustness, and reverse data processing inequalities” In IEEE Transactions on Information Theory To appear, 2023, pp. 1–1 DOI: 10.1109/TIT.2023.3334024
- A. Pensia, P. Loh and V. Jog “Simple Binary Hypothesis Testing under Communication Constraints” In Proc. 2022 IEEE International Symposium on Information Theory, 2022
- “Estimation with finite memory” In IEEE Transactions on Information Theory 16.6, 1970, pp. 685–691
- Or Sheffet “Locally Private Hypothesis Testing” In Proc. 35th International Conference on Machine Learning (ICML), 2018
- J.N. Tsitsiklis “Decentralized Detection by a Large Number of Sensors” In Mathematics of Control, Signals, and Systems 1.2, 1988, pp. 167–182 DOI: 10.1007/BF02551407
- J.N. Tsitsiklis “Decentralized Detection” In Advances in Statistical Signal Processing, 1993, pp. 297–344
- A.B. Tsybakov “Introduction to Nonparametric Estimation”, Springer Series in Statistics Springer New York, 2009
- A. Wald “Sequential Tests of Statistical Hypotheses” In The Annals of Mathematical Statistics 16.2, 1945, pp. 117–186 DOI: 10.1214/aoms/1177731118
- S.L. Warner “Randomized Response: A Survey Technique for Eliminating Evasive Answer Bias” In Journal of the American Statistical Association 60.309, 1965, pp. 63–69