Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Synthesizing Tight Privacy and Accuracy Bounds via Weighted Model Counting (2402.16982v3)

Published 26 Feb 2024 in cs.CR and cs.PL

Abstract: Programmatically generating tight differential privacy (DP) bounds is a hard problem. Two core challenges are (1) finding expressive, compact, and efficient encodings of the distributions of DP algorithms, and (2) state space explosion stemming from the multiple quantifiers and relational properties of the DP definition. We address the first challenge by developing a method for tight privacy and accuracy bound synthesis using weighted model counting on binary decision diagrams, a state-of-the-art technique from the artificial intelligence and automated reasoning communities for exactly computing probability distributions. We address the second challenge by developing a framework for leveraging inherent symmetries in DP algorithms. Our solution benefits from ongoing research in probabilistic programming languages, allowing us to succinctly and expressively represent different DP algorithms with approachable language syntax that can be used by non-experts. We provide a detailed case study of our solution on the binary randomized response algorithm. We also evaluate an implementation of our solution using the Dice probabilistic programming language for the randomized response and truncated geometric above threshold algorithms. We compare to prior work on exact DP verification using Markov chain probabilistic model checking and the decision procedure DiPC. Very few existing works consider mechanized analysis of accuracy guarantees for DP algorithms. We additionally provide a detailed analysis using our technique for finding tight accuracy bounds for DP algorithms.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” Journal of Privacy and Confidentiality, vol. 7, no. 3, 2016.
  2. I. Mironov, “On significance of the least significant bits for differential privacy,” in Proceedings of the 2012 ACM conference on Computer and communications security, 2012, pp. 650–661.
  3. F. Tramer, A. Terzis, T. Steinke, S. Song, M. Jagielski, and N. Carlini, “Debugging differential privacy: A case study for privacy auditing,” arXiv preprint arXiv:2202.12219, 2022.
  4. T. Stevens, I. C. Ngong, D. Darais, C. Hirsch, D. Slater, and J. P. Near, “Backpropagation clipping for deep learning with differential privacy,” arXiv preprint arXiv:2202.05089, 2022.
  5. G. Barthe, G. Danezis, B. Grégoire, C. Kunz, and S. Zanella-Beguelin, “Verified computational differential privacy with applications to smart metering,” in 2013 IEEE 26th Computer Security Foundations Symposium.   IEEE, 2013, pp. 287–301.
  6. G. Barthe, B. Köpf, F. Olmedo, and S. Zanella Beguelin, “Probabilistic relational reasoning for differential privacy,” in Proceedings of the 39th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages, 2012, pp. 97–110.
  7. G. Barthe, M. Gaboardi, E. J. G. Arias, J. Hsu, C. Kunz, and P.-Y. Strub, “Proving differential privacy in hoare logic,” in 2014 IEEE 27th Computer Security Foundations Symposium.   IEEE, 2014, pp. 411–424.
  8. G. Barthe, M. Gaboardi, B. Grégoire, J. Hsu, and P.-Y. Strub, “Proving differential privacy via probabilistic couplings,” in Proceedings of the 31st Annual ACM/IEEE Symposium on Logic in Computer Science, 2016, pp. 749–758.
  9. Z. Ding, Y. Wang, G. Wang, D. Zhang, and D. Kifer, “Detecting violations of differential privacy,” in Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, 2018, pp. 475–489.
  10. Y. Wang, Z. Ding, D. Kifer, and D. Zhang, “Checkdp: An automated and integrated approach for proving differential privacy or finding precise counterexamples,” in Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, 2020, pp. 919–938.
  11. B. Bichsel, T. Gehr, D. Drachsler-Cohen, P. Tsankov, and M. Vechev, “Dp-finder: Finding differential privacy violations by sampling and optimization,” in Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, 2018, pp. 508–524.
  12. H. Zhang, E. Roth, A. Haeberlen, B. C. Pierce, and A. Roth, “Testing differential privacy with dual interpreters,” Proceedings of the ACM on Programming Languages, vol. 4, no. OOPSLA, pp. 1–26, 2020.
  13. M. Jagielski, J. Ullman, and A. Oprea, “Auditing differentially private machine learning: How private is private SGD?” in Proceedings of Advances in Neural Information Processing Systems, ser. NeurIPS, vol. 33, 2020, pp. 22 205–22 216. [Online]. Available: https://proceedings.neurips.cc/paper/2020/file/fc4ddc15f9f4b4b06ef7844d6bb53abf-Paper.pdf
  14. M. Nasr, S. Song, A. Thakurta, N. Papernot, and N. Carlini, “Adversary instantiation: Lower bounds for differentially private machine learning,” in 42nd IEEE Symposium on Security and Privacy, SP 2021, San Francisco, CA, USA, 24-27 May 2021.   IEEE, 2021, pp. 866–882. [Online]. Available: https://doi.org/10.1109/SP40001.2021.00069
  15. G. Andrew, P. Kairouz, S. Oh, A. Oprea, H. B. McMahan, and V. Suriyakumar, “One-shot empirical privacy estimation for federated learning,” CoRR, vol. abs/2302.03098, 2023.
  16. M. Nasr, J. Hayes, T. Steinke, B. Balle, F. Tramèr, M. Jagielski, N. Carlini, and A. Terzis, “Tight auditing of differentially private machine learning,” in Proceedings of the 32nd USENIX Conference on Security Symposium, ser. SEC ’23.   USA: USENIX Association, 2023.
  17. K. Pillutla, G. Andrew, P. Kairouz, H. B. McMahan, A. Oprea, and S. Oh, “Unleashing the power of randomization in auditing differentially private ML,” in Thirty-seventh Conference on Neural Information Processing Systems, 2023. [Online]. Available: https://openreview.net/forum?id=mlbes5TAAg
  18. M. J. Thomas Steinke, Milad Nasr, “Privacy auditing with one (1) training run,” in Thirty-seventh Conference on Neural Information Processing Systems, 2023. [Online]. Available: https://openreview.net/forum?id=mlbes5TAAg
  19. S. Roy, J. Hsu, and A. Albarghouthi, “Learning differentially private mechanisms,” in 2021 IEEE Symposium on Security and Privacy (SP).   IEEE, 2021, pp. 852–865.
  20. D. Liu, B.-Y. Wang, and L. Zhang, “Model checking differentially private properties,” in Asian Symposium on Programming Languages and Systems.   Springer, 2018, pp. 394–414.
  21. ——, “Verifying pufferfish privacy in hidden markov models,” in International Conference on Verification, Model Checking, and Abstract Interpretation.   Springer, 2022, pp. 174–196.
  22. M. Chavira and A. Darwiche, “On probabilistic inference by weighted model counting,” Artificial Intelligence, vol. 172, no. 6-7, pp. 772–799, 2008.
  23. S. Holtzen, G. Van den Broeck, and T. Millstein, “Scaling exact inference for discrete probabilistic programs,” Proceedings of the ACM on Programming Languages, vol. 4, no. OOPSLA, pp. 1–31, 2020.
  24. S. Holtzen, T. Millstein, and G. Van den Broeck, “Generating and sampling orbits for lifted probabilistic inference,” in Uncertainty in Artificial Intelligence.   PMLR, 2020, pp. 985–994.
  25. C. Dwork, A. Roth et al., “The algorithmic foundations of differential privacy,” Foundations and Trends® in Theoretical Computer Science, vol. 9, no. 3–4, pp. 211–407, 2014.
  26. S. Holtzen, S. Junges, M. Vazquez-Chanlatte, T. Millstein, S. A. Seshia, and G. Van den Broeck, “Model checking finite-horizon markov chains with probabilistic inference,” in Proceedings of the 33rd International Conference on Computer-Aided Verification (CAV), July 2021.
  27. C. Hensel, S. Junges, J.-P. Katoen, T. Quatmann, and M. Volk, “The probabilistic model checker storm,” International Journal on Software Tools for Technology Transfer, pp. 1–22, 2021.
  28. J. Reed and B. C. Pierce, “Distance makes the types grow stronger: a calculus for differential privacy,” in Proceedings of the 15th ACM SIGPLAN international conference on Functional programming, 2010, pp. 157–168.
  29. M. Gaboardi, A. Haeberlen, J. Hsu, A. Narayan, and B. C. Pierce, “Linear dependent types for differential privacy,” in Proceedings of the 40th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages, 2013, pp. 357–370.
  30. G. Barthe, M. Gaboardi, E. J. Gallego Arias, J. Hsu, A. Roth, and P.-Y. Strub, “Higher-order approximate relational refinement types for mechanism design and differential privacy,” ACM SIGPLAN Notices, vol. 50, no. 1, pp. 55–68, 2015.
  31. D. Zhang and D. Kifer, “Lightdp: Towards automating differential privacy proofs,” in Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages, 2017, pp. 888–901.
  32. J. P. Near, D. Darais, C. Abuah, T. Stevens, P. Gaddamadugu, L. Wang, N. Somani, M. Zhang, N. Sharma, A. Shan et al., “Duet: an expressive higher-order language and linear type system for statically enforcing differential privacy,” Proceedings of the ACM on Programming Languages, vol. 3, no. OOPSLA, pp. 1–30, 2019.
  33. M. Fredrikson and S. Jha, “Satisfiability modulo counting: A new approach for analyzing privacy properties,” in Proceedings of the Joint Meeting of the Twenty-Third EACSL Annual Conference on Computer Science Logic (CSL) and the Twenty-Ninth Annual ACM/IEEE Symposium on Logic in Computer Science (LICS), 2014, pp. 1–10.
  34. A. Albarghouthi and J. Hsu, “Synthesizing coupling proofs of differential privacy,” Proceedings of the ACM on Programming Languages, vol. 2, no. POPL, pp. 1–30, 2017.
  35. G. Barthe, G. P. Farina, M. Gaboardi, E. J. G. Arias, A. Gordon, J. Hsu, and P.-Y. Strub, “Differentially private bayesian programming,” in Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, 2016, pp. 68–79.
  36. C. Smith and A. Albarghouthi, “Synthesizing differentially private programs,” Proceedings of the ACM on Programming Languages, vol. 3, no. ICFP, pp. 1–29, 2019.
  37. Y. Wang, Z. Ding, Y. Xiao, D. Kifer, and D. Zhang, “Dpgen: Automated program synthesis for differential privacy,” in Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, 2021, pp. 393–411.
  38. M. Gaboardi, K. Nissim, and D. Purser, “The complexity of verifying loop-free programs as differentially private,” arXiv preprint arXiv:1911.03272, 2019.
  39. M. Bun, M. Gaboardi, and L. Glinskih, “The complexity of verifying boolean programs as differentially private,” in 2022 IEEE 35th Computer Security Foundations Symposium (CSF).   IEEE, 2022, pp. 396–411.
  40. S. Zanella-Beguelin, L. Wutschitz, S. Tople, A. Salem, V. Rühle, A. Paverd, M. Naseri, B. Köpf, and D. Jones, “Bayesian estimation of differential privacy,” in Proceedings of the 40th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett, Eds., vol. 202.   PMLR, 23–29 Jul 2023, pp. 40 624–40 636. [Online]. Available: https://proceedings.mlr.press/v202/zanella-beguelin23a.html
  41. G. Barthe, R. Chadha, P. Krogmeier, A. P. Sistla, and M. Viswanathan, “Deciding accuracy of differential privacy schemes,” Proceedings of the ACM on Programming Languages, vol. 5, no. POPL, pp. 1–30, 2021.
  42. E. Lobo-Vesga, A. Russo, and M. Gaboardi, “A programming framework for differential privacy with accuracy concentration bounds,” in 2020 IEEE Symposium on Security and Privacy (SP).   IEEE, 2020, pp. 411–428.
  43. A. Cheu, A. Smith, J. Ullman, D. Zeber, and M. Zhilyaev, “Distributed differential privacy via shuffling,” in Advances in Cryptology–EUROCRYPT 2019: 38th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Darmstadt, Germany, May 19–23, 2019, Proceedings, Part I 38.   Springer, 2019, pp. 375–403.
  44. V. Balcer and S. Vadhan, “Differential privacy on finite computers,” arXiv preprint arXiv:1709.05396, 2017.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now