Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Achieving the Exactly Optimal Privacy-Utility Trade-Off with Low Communication Cost via Shared Randomness (2307.03962v1)

Published 8 Jul 2023 in cs.IT, cs.CR, and math.IT

Abstract: We consider a discrete distribution estimation problem under a local differential privacy (LDP) constraint in the presence of shared randomness. By exploiting the shared randomness, we suggest a new method for constructing LDP schemes which achieve the exactly optimal privacy-utility trade-off (PUT) with the communication cost of less than or equal to the input data size for any privacy regime. The main idea is to decompose a block design scheme by Park et al. (2023), based on the combinatorial concept called resolution. The LDP scheme decomposed from a block design scheme is called a resolution of the block design scheme, and it achieves the same PUT as the original block design scheme while requiring a less communication cost. We provide two resolutions of an exactly PUT-optimal block design scheme, called the Baranyai's resolution and the cyclic shift resolution, both requiring the communication cost of less than or equal to the input data size. In particular, we show that the Baranyai's resolution achieves the minimum communication cost among all the PUT-optimal resolutions of block design schemes. One drawback of the Baranyai's resolution is that it can be obtained through a recursive algorithm in general. In contrast, the cyclic shift resolution has an explicit structure, but its communication cost can be larger than that of Baranyai's resolution. To complement this, we also suggest resolutions of other block design schemes achieving the optimal PUT for some privacy budgets, which require the minimum communication cost as the Baranyai's resolution and have explicit structures as the cyclic shift resolution.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (25)
  1. S. P. Kasiviswanathan, H. K. Lee, K. Nissim, S. Raskhodnikova, and A. Smith, “What can we learn privately?” SIAM Journal on Computing, vol. 40, no. 3, pp. 793–826, 2011.
  2. J. Duchi, M. J. Wainwright, and M. I. Jordan, “Local privacy and minimax bounds: Sharp rates for probability estimation,” in Advances in Neural Information Processing Systems, vol. 26, 2013, pp. 1529–1537.
  3. I. Issa, A. B. Wagner, and S. Kamath, “An operational approach to information leakage,” IEEE Transactions on Information Theory, vol. 66, no. 3, pp. 1625–1657, 2019.
  4. M. Ye and A. Barg, “Optimal schemes for discrete distribution estimation under locally differential privacy,” IEEE Transactions on Information Theory, vol. 64, no. 8, pp. 5662–5676, 2018.
  5. J. Acharya, Z. Sun, and H. Zhang, “Hadamard response: Estimating distributions privately, efficiently, and with little communication,” in Proc. 22nd Int. Conf. Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, vol. 89.   PMLR, 16–18 Apr 2019, pp. 1120–1129.
  6. L. P. Barnes, W.-N. Chen, and A. Özgür, “Fisher information under local differential privacy,” IEEE Journal on Selected Areas in Information Theory, vol. 1, no. 3, pp. 645–659, 2020.
  7. V. Feldman, J. Nelson, H. Nguyen, and K. Talwar, “Private frequency estimation via projective geometry,” in Proc. 39th Int. Conf. Machine Learning, ser. Proceedings of Machine Learning Research, vol. 162.   PMLR, 17–23 Jul 2022, pp. 6418–6433.
  8. N. Wang, X. Xiao, Y. Yang, J. Zhao, S. C. Hui, H. Shin, J. Shin, and G. Yu, “Collecting and analyzing multidimensional data with local differential privacy,” in 2019 IEEE 35th International Conference on Data Engineering (ICDE).   IEEE, 2019, pp. 638–649.
  9. H. Asi, V. Feldman, and K. Talwar, “Optimal algorithms for mean estimation under local differential privacy,” in International Conference on Machine Learning.   PMLR, 2022, pp. 1046–1056.
  10. A. Bhowmick, J. Duchi, J. Freudiger, G. Kapoor, and R. Rogers, “Protection against reconstruction and its applications in private federated learning,” 2018. [Online]. Available: arXiv:1812.00984
  11. S. Truex, L. Liu, K.-H. Chow, M. E. Gursoy, and W. Wei, “LDP-Fed: Federated learning with local differential privacy,” in Proceedings of the Third ACM International Workshop on Edge Systems, Analytics and Networking, 2020, pp. 61–66.
  12. Y. Zhao, J. Zhao, M. Yang, T. Wang, N. Wang, L. Lyu, D. Niyato, and K.-Y. Lam, “Local differential privacy-based federated learning for internet of things,” IEEE Internet of Things Journal, vol. 8, no. 11, pp. 8836–8853, 2020.
  13. M. Seif, R. Tandon, and M. Li, “Wireless federated learning with local differential privacy,” in 2020 IEEE International Symposium on Information Theory (ISIT).   IEEE, 2020, pp. 2604–2609.
  14. M. Ye and A. Barg, “Optimal locally private estimation under ℓpsubscriptℓ𝑝\ell_{p}roman_ℓ start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT loss for 1≤p≤21𝑝21\leq p\leq 21 ≤ italic_p ≤ 2,” Electronic Journal of Statistics, vol. 13, no. 2, pp. 4102 – 4120, 2019.
  15. H.-Y. Park, S.-H. Nam, and S.-H. Lee, “Block design-based local differential privacy mechanisms,” 2023. [Online]. Available: arXiv:2305.01261
  16. W.-N. Chen, P. Kairouz, and A. Ozgur, “Breaking the communication-privacy-accuracy trilemma,” Advances in Neural Information Processing Systems, vol. 33, pp. 3312–3324, 2020.
  17. A. Shah, W.-N. Chen, J. Balle, P. Kairouz, and L. Theis, “Optimal compression of locally differentially private mechanisms,” in International Conference on Artificial Intelligence and Statistics.   PMLR, 2022, pp. 7680–7723.
  18. R. Bassily and A. Smith, “Local, private, efficient protocols for succinct histograms,” in Proceedings of the forty-seventh annual ACM symposium on Theory of computing, 2015, pp. 127–135.
  19. T. Wang, J. Blocki, N. Li, and S. Jha, “Locally differentially private protocols for frequency estimation,” in 26th USENIX Security Symposium (USENIX Security 17), 2017, pp. 729–745.
  20. S. L. Warner, “Randomized response: A survey technique for eliminating evasive answer bias,” Journal of the American Statistical Association, vol. 60, no. 309, pp. 63–69, 1965.
  21. J. Acharya and Z. Sun, “Communication complexity in locally private distribution estimation and heavy hitters,” in International Conference on Machine Learning.   PMLR, 2019, pp. 51–60.
  22. Ú. Erlingsson, V. Pihur, and A. Korolova, “RAPPOR: Randomized aggregatable privacy-preserving ordinal response,” in Proceedings of the 2014 ACM SIGSAC conference on computer and communications security, 2014, pp. 1054–1067.
  23. Z. Baranyai, “On the factorization of the complete uniform hypergraphs,” Infinite and finite sets, pp. 91–108, 1974.
  24. Y. M. Chee, T. Etzion, H. M. Kiah, A. Vardy, and C. Wang, “Explicit Baranyai partitions for quadruples, part I: Quadrupling constructions,” Journal of Combinatorial Designs, vol. 29, no. 7, pp. 447–481, 2021.
  25. A. Hedayat and W. D. Wallis, “Hadamard matrices and their applications,” The Annals of Statistics, vol. 6, no. 6, pp. 1184 – 1238, 1978.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now