Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
114 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
35 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
3 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
2000 character limit reached

Total Variation Meets Differential Privacy (2311.01553v2)

Published 2 Nov 2023 in cs.IT and math.IT

Abstract: The framework of approximate differential privacy is considered, and augmented by leveraging the notion of ``the total variation of a (privacy-preserving) mechanism'' (denoted by $\eta$-TV). With this refinement, an exact composition result is derived, and shown to be significantly tighter than the optimal bounds for differential privacy (which do not consider the total variation). Furthermore, it is shown that $(\varepsilon,\delta)$-DP with $\eta$-TV is closed under subsampling. The induced total variation of commonly used mechanisms are computed. Moreover, the notion of total variation of a mechanism is studied in the local privacy setting and privacy-utility tradeoffs are investigated. In particular, total variation distance and KL divergence are considered as utility functions and studied through the lens of contraction coefficients. Finally, the results are compared and connected to the locally differentially private setting.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” in Theory of Cryptography: 3rd Theory of Cryptography Conference, New York, NY, USA, Mar. 4-7, 2006.   Springer, 2006, pp. 265–284.
  2. C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor, “Our data, ourselves: Privacy via distributed noise generation,” in Annual International Conference on the Theory and Applications of Cryptographic Techniques.   Springer, 2006, pp. 486–503.
  3. I. Mironov, “Rényi differential privacy,” in IEEE 30th Computer Security Foundations Symposium.   IEEE, 2017, pp. 263–275.
  4. C. Dwork and G. N. Rothblum, “Concentrated differential privacy,” arXiv preprint arXiv:1603.01887, 2016.
  5. J. Dong, A. Roth, and W. J. Su, “Gaussian differential privacy,” 2019. [Online]. Available: https://arxiv.org/abs/1905.02383
  6. L. Wasserman and S. Zhou, “A statistical framework for differential privacy,” Journal of the American Statistical Association, vol. 105, no. 489, pp. 375–389, 2010.
  7. P. Kairouz, S. Oh, and P. Viswanath, “The composition theorem for differential privacy,” in International conference on machine learning.   PMLR, 2015, pp. 1376–1385.
  8. Q. Geng, P. Kairouz, S. Oh, and P. Viswanath, “The staircase mechanism in differential privacy,” IEEE Journal of Selected Topics in Signal Processing, vol. 9, no. 7, pp. 1176–1184, 2015.
  9. J. C. Duchi, M. I. Jordan, and M. J. Wainwright, “Local privacy, data processing inequalities, and statistical minimax rates,” 2014.
  10. P. Kairouz, S. Oh, and P. Viswanath, “Extremal mechanisms for local differential privacy,” vol. 17, no. 17, 2016, pp. 1–51.
  11. S. Asoodeh and H. Zhang, “Contraction of locally differentially private mechanisms,” arXiv preprint arXiv:2210.13386, 2022.
  12. K. Chatzikokolakis, G. Cherubin, C. Palamidessi, and C. Troncoso, “Bayes security: A not so average metric,” in IEEE 36th Computer Security Foundations Symposium, 2023, pp. 388–406.
  13. R. F. Barber and J. C. Duchi, “Privacy and statistical risk: Formalisms and minimax bounds,” 2014.
  14. Q. Geng, W. Ding, R. Guo, and S. Kumar, “Optimal noise-adding mechanism in additive differential privacy,” in The 22nd International Conference on Artificial Intelligence and Statistics.   PMLR, 2019, pp. 11–20.
  15. J. Jia, C. Tan, Z. Liu, X. Li, Z. Liu, S. Lv, and C. Dong, “Total variation distance privacy: Accurately measuring inference attacks and improving utility,” Information Sciences, vol. 626, pp. 537–558, 2023.
  16. C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel, “Fairness through awareness,” in Proceedings of the 3rd Innovations in Theoretical Computer Science Conference.   New York, NY, USA: Association for Computing Machinery, 2012, p. 214–226.
  17. R. Bassily, K. Nissim, A. Smith, T. Steinke, U. Stemmer, and J. Ullman, “Algorithmic stability for adaptive data analysis,” in Proceedings of the Forty-Eighth Annual ACM Symposium on Theory of Computing.   New York, NY, USA: Association for Computing Machinery, 2016, p. 1046–1059.
  18. M. Raginsky, A. Rakhlin, M. Tsao, Y. Wu, and A. Xu, “Information-theoretic analysis of stability and bias of learning algorithms,” in IEEE Information Theory Workshop, 2016.
  19. M. Yaghini, B. Kulynych, and C. Troncoso, “Disparate vulnerability: on the unfairness of privacy attacks against machine learning,” CoRR, vol. abs/1906.00389, 2019.
  20. B. Kulynych, Y.-Y. Yang, Y. Yu, J. Błasiok, and P. Nakkiran, “What you see is what you get: Principled deep learning via distributional generalization,” vol. 35, 2022, pp. 2168–2183.
  21. D. Blackwell, “Equivalent comparisons of experiments,” The annals of mathematical statistics, pp. 265–272, 1953.
  22. T. M. Cover and J. A. Thomas, “Elements of information theory 2nd edition,” Willey-Interscience: NJ, 2006.
  23. S. P. Kasiviswanathan, H. K. Lee, K. Nissim, S. Raskhodnikova, and A. Smith, “What can we learn privately?” SIAM Journal on Computing, vol. 40, no. 3, pp. 793–826, 2011.
  24. A. Smith, “Differential privacy and the secrecy of the sample,” Sep 2009. [Online]. Available: https://adamdsmith.wordpress.com/2009/09/02/sample-secrecy/
  25. K. Chaudhuri and N. Mishra, “When random sampling preserves privacy,” in Annual International Cryptology Conference.   Springer, 2006, pp. 198–213.
  26. B. Balle, G. Barthe, and M. Gaboardi, “Privacy amplification by subsampling: Tight analyses via couplings and divergences,” Advances in neural information processing systems, vol. 31, 2018.
  27. T. Steinke, “Composition of differential privacy & privacy amplification by subsampling,” arXiv preprint arXiv:2210.00597, 2022.
  28. A. Koskela, J. Jälkö, L. Prediger, and A. Honkela, “Tight differential privacy for discrete-valued mechanisms and for the subsampled gaussian mechanism using FFT,” in International Conference on Artificial Intelligence and Statistics.   PMLR, 2021, pp. 3358–3366.
  29. S. Gopi, Y. T. Lee, and L. Wutschitz, “Numerical composition of differential privacy,” vol. 34, 2021, pp. 11 631–11 642.
  30. Y. Zhu, J. Dong, and Y.-X. Wang, “Optimal accounting of differential privacy via characteristic function,” in International Conference on Artificial Intelligence and Statistics.   PMLR, 2022, pp. 4782–4817.
  31. B. Balle and Y.-X. Wang, “Improving the gaussian mechanism for differential privacy: Analytical calibration and optimal denoising,” in International Conference on Machine Learning.   PMLR, 2018, pp. 394–403.
  32. Q. Geng and P. Viswanath, “The optimal mechanism in differential privacy,” in 2014 IEEE International Symposium on Information Theory, 2014, pp. 2371–2375.
  33. M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, 2016, pp. 308–318.
  34. R. L. Dobrushin, “Central limit theorem for nonstationary markov chains. i,” Theory of Probability & Its Applications, vol. 1, no. 1, pp. 65–80, 1956.
  35. S. Asoodeh, M. Aliakbarpour, and F. P. Calmon, “Local differential privacy is equivalent to contraction of an f𝑓fitalic_f-divergence,” in IEEE International Symposium on Information Theory.   IEEE, 2021, pp. 545–550.
  36. B. Zamanlooy and S. Asoodeh, “Strong data processing inequalities for locally differentially private mechanisms,” in IEEE International Symposium on Information Theory.   IEEE, 2023, pp. 1794–1799.
  37. M.-D. Choi, M. B. Ruskai, and E. Seneta, “Equivalence of certain entropy contraction coefficients,” Linear algebra and its applications, vol. 208, pp. 29–36, 1994.
  38. O. Ordentlich and Y. Polyanskiy, “Strong data processing constant is achieved by binary inputs,” IEEE Transactions on Information Theory, vol. 68, no. 3, pp. 1480–1481, 2021.
  39. Y. Polyanskiy and Y. Wu, “Strong data-processing inequalities for channels and bayesian networks,” in Convexity and Concentration.   Springer, 2017, pp. 211–249.
  40. L. Györfi and I. Vajda, “A class of modified pearson and neyman statistics,” Statistics & Risk Modeling, vol. 19, no. 3, pp. 239–252, 2001.
  41. M. Raginsky, “Strong data processing inequalities and ΦΦ\Phiroman_Φ-sobolev inequalities for discrete channels,” IEEE Transactions on Information Theory, vol. 62, no. 6, pp. 3355–3389, 2016.
  42. I. Sason and S. Verdú, “f𝑓fitalic_f-divergence inequalities,” IEEE Transactions on Information Theory, vol. 62, no. 11, pp. 5973–6006, 2016.
  43. J. C. Duchi and F. Ruan, “The right complexity measure in locally private estimation: It is not the fisher information,” arXiv preprint arXiv:1806.05756, 2018.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets