Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Private Prediction Sets (2102.06202v3)

Published 11 Feb 2021 in cs.LG, cs.AI, cs.CR, stat.ME, and stat.ML

Abstract: In real-world settings involving consequential decision-making, the deployment of machine learning systems generally requires both reliable uncertainty quantification and protection of individuals' privacy. We present a framework that treats these two desiderata jointly. Our framework is based on conformal prediction, a methodology that augments predictive models to return prediction sets that provide uncertainty quantification -- they provably cover the true response with a user-specified probability, such as 90%. One might hope that when used with privately-trained models, conformal prediction would yield privacy guarantees for the resulting prediction sets; unfortunately, this is not the case. To remedy this key problem, we develop a method that takes any pre-trained predictive model and outputs differentially private prediction sets. Our method follows the general approach of split conformal prediction; we use holdout data to calibrate the size of the prediction sets but preserve privacy by using a privatized quantile subroutine. This subroutine compensates for the noise introduced to preserve privacy in order to guarantee correct coverage. We evaluate the method on large-scale computer vision datasets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. J. C. Perez, C. de Blas Perez, F. L. Alvarez, and J. M. C. Contreras, “Databiology Lab CORONAHACK: Collection of public COVID-19 data,” bioRxiv, 2020.
  2. H. Papadopoulos, K. Proedrou, V. Vovk, and A. Gammerman, “Inductive confidence machines for regression,” in Machine Learning: European Conference on Machine Learning, pp. 345–356, 2002.
  3. Springer, 2005.
  4. J. Lei, M. G’Sell, A. Rinaldo, R. J. Tibshirani, and L. Wasserman, “Distribution-free predictive inference for regression,” Journal of the American Statistical Association, vol. 113, no. 523, pp. 1094–1111, 2018.
  5. C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” in Theory of Cryptography Conference, pp. 265–284, Springer, 2006.
  6. Ú. Erlingsson, V. Pihur, and A. Korolova, “Rappor: Randomized aggregatable privacy-preserving ordinal response,” in Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, pp. 1054–1067, 2014.
  7. A. Bittau, Ú. Erlingsson, P. Maniatis, I. Mironov, A. Raghunathan, D. Lie, M. Rudominer, U. Kode, J. Tinnes, and B. Seefeld, “Prochlo: Strong privacy for analytics in the crowd,” in Proceedings of the 26th Symposium on Operating Systems Principles, pp. 441–459, 2017.
  8. Differential Privacy Team Apple, “Learning with privacy at scale,” in Apple Machine Learning Research, 2017.
  9. B. Ding, J. Kulkarni, and S. Yekhanin, “Collecting telemetry data privately,” in Advances in Neural Information Processing Systems, pp. 3571–3580, 2017.
  10. J. M. Abowd, “The US census bureau adopts differential privacy,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2867–2867, 2018.
  11. C. Dwork, “Differential privacy and the US census,” in Proceedings of the 38th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, pp. 1–1, 2019.
  12. K. Chaudhuri, C. Monteleoni, and A. D. Sarwate, “Differentially private empirical risk minimization.,” Journal of Machine Learning Research, vol. 12, no. 3, 2011.
  13. R. Bassily, A. Smith, and A. Thakurta, “Private empirical risk minimization: Efficient algorithms and tight error bounds,” in 2014 IEEE 55th Annual Symposium on Foundations of Computer Science, pp. 464–473, IEEE, 2014.
  14. M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308–318, 2016.
  15. S. Neel, A. Roth, G. Vietri, and S. Wu, “Oracle efficient private non-convex optimization,” in International Conference on Machine Learning, pp. 7243–7252, PMLR, 2020.
  16. J. Xu, Z. Zhang, X. Xiao, Y. Yang, G. Yu, and M. Winslett, “Differentially private histogram publication,” The VLDB Journal, vol. 22, no. 6, pp. 797–822, 2013.
  17. J. Lei, “Differentially private m-estimators,” Advances in Neural Information Processing Systems, vol. 24, pp. 361–369, 2011.
  18. A. Smith, “Privacy-preserving statistical estimation with optimal convergence rates,” in Proceedings of the forty-third annual ACM symposium on Theory of computing, pp. 813–822, 2011.
  19. V. Feldman and T. Steinke, “Generalization for adaptively-chosen estimators via stable median,” in Conference on Learning Theory, pp. 728–757, PMLR, 2017.
  20. V. Karwa and S. Vadhan, “Finite sample differentially private confidence intervals,” arXiv preprint arXiv:1711.03908, 2017.
  21. O. Sheffet, “Differentially private ordinary least squares,” in International Conference on Machine Learning, pp. 3105–3114, PMLR, 2017.
  22. M. Gaboardi, R. Rogers, and O. Sheffet, “Locally private mean estimation: z𝑧zitalic_z-test and tight confidence intervals,” in The 22nd International Conference on Artificial Intelligence and Statistics, pp. 2545–2554, PMLR, 2019.
  23. Y. Wang, D. Kifer, and J. Lee, “Differentially private confidence intervals for empirical risk minimization,” Journal of Privacy and Confidentiality, vol. 9, no. 1, 2019.
  24. S. S. Wilks, “Determination of sample sizes for setting tolerance limits,” Annals of Mathematical Statistics, vol. 12, no. 1, pp. 91–96, 1941.
  25. S. S. Wilks, “Statistical prediction with special reference to the problem of tolerance limits,” Annals of Mathematical Statistics, vol. 13, no. 4, pp. 400–409, 1942.
  26. A. Wald, “An extension of Wilks’ method for setting tolerance limits,” Annals of Mathematical Statistics, vol. 14, no. 1, pp. 45–55, 1943.
  27. J. W. Tukey, “Non-parametric estimation II. statistically equivalent blocks and tolerance regions—the continuous case,” Annals of Mathematical Statistics, vol. 18, no. 4, pp. 529–539, 1947.
  28. Wiley, 2009.
  29. S. Park, O. Bastani, N. Matni, and I. Lee, “PAC confidence sets for deep neural networks via calibrated prediction,” in International Conference on Learning Representations, 2020.
  30. V. Vovk, A. Gammerman, and C. Saunders, “Machine-learning applications of algorithmic randomness,” in International Conference on Machine Learning, pp. 444–453, 1999.
  31. G. Shafer and V. Vovk, “A tutorial on conformal prediction,” Journal of Machine Learning Research, vol. 9, no. Mar, pp. 371–421, 2008.
  32. J. Lei, A. Rinaldo, and L. Wasserman, “A conformal prediction approach to explore functional data,” Annals of Mathematics and Artificial Intelligence, vol. 74, pp. 29–43, 2015.
  33. V. Vovk, “Cross-conformal predictors,” Annals of Mathematics and Artificial Intelligence, vol. 74, no. 1-2, pp. 9–28, 2015.
  34. R. F. Barber, E. J. Candes, A. Ramdas, R. J. Tibshirani, et al., “Predictive inference with the jackknife+,” Annals of Statistics, vol. 49, no. 1, pp. 486–507, 2021.
  35. M. Sadinle, J. Lei, and L. Wasserman, “Least ambiguous set-valued classifiers with bounded error levels,” Journal of the American Statistical Association, vol. 114, pp. 223 – 234, 2019.
  36. A. N. Angelopoulos, S. Bates, J. Malik, and M. I. Jordan, “Uncertainty sets for image classifiers using conformal prediction,” arXiv:2009.14193, 2020.
  37. V. Vovk, “Conditional validity of inductive conformal predictors,” in Proceedings of the Asian Conference on Machine Learning, vol. 25, pp. 475–490, 2012.
  38. R. Foygel Barber, E. J. Candès, A. Ramdas, and R. J. Tibshirani, “The limits of distribution-free conditional predictive inference,” Information and Inference: A Journal of the IMA, 2019.
  39. Y. Romano, E. Patterson, and E. Candès, “Conformalized quantile regression,” in Advances in Neural Information Processing Systems, vol. 32, pp. 3543–3553, 2019.
  40. R. Izbicki, G. T. Shimizu, and R. B. Stern, “Flexible distribution-free conditional predictive bands using density estimators,” arXiv:1910.05575, 2019.
  41. Y. Romano, M. Sesia, and E. J. Candès, “Classification with valid and adaptive coverage,” arXiv:2006.02544, 2020.
  42. L. Guan, “Conformal prediction with localization,” arXiv:1908.08558, 2020.
  43. M. Cauchois, S. Gupta, and J. Duchi, “Knowing what you know: valid and validated confidence sets in multiclass and multilabel prediction,” arXiv:2004.10181, 2020.
  44. J. Lei, “Classification with confidence,” Biometrika, vol. 101, pp. 755–769, 10 2014.
  45. Y. Hechtlinger, B. Poczos, and L. Wasserman, “Cautious deep learning,” arXiv:1805.09460, 2018.
  46. L. Guan and R. Tibshirani, “Prediction and outlier detection in classification problems,” arXiv:1905.04396, 2019.
  47. V. Vovk, J. Shen, V. Manokhin, and M.-g. Xie, “Nonparametric predictive distributions based on conformal prediction,” Machine Learning, pp. 1–30, 2017.
  48. V. Vovk, I. Petej, P. Toccaceli, A. Gammerman, E. Ahlberg, and L. Carlsson, “Conformal calibrators,” in Conformal and Probabilistic Prediction and Applications, pp. 84–99, PMLR, 2020.
  49. R. J. Tibshirani, R. Foygel Barber, E. Candes, and A. Ramdas, “Conformal prediction under covariate shift,” in Advances in Neural Information Processing Systems 32, pp. 2530–2540, 2019.
  50. M. Cauchois, S. Gupta, A. Ali, and J. C. Duchi, “Robust validation: Confident predictions even when distributions shift,” arXiv:2008.04267, 2020.
  51. X. Hu and J. Lei, “A distribution-free test of covariate shift using conformal prediction,” arXiv:2010.07147, 2020.
  52. L. Lei and E. J. Candès, “Conformal inference of counterfactuals and individual treatment effects,” arXiv:2006.06138, 2020.
  53. S. Bates, A. Angelopoulos, L. Lei, J. Malik, and M. I. Jordan, “Distribution-free, risk-controlling prediction sets,” arXiv:2101.02703, 2021.
  54. A. N. Angelopoulos and S. Bates, “A gentle introduction to conformal prediction and distribution-free uncertainty quantification,” arXiv preprint arXiv:2107.07511, 2021.
  55. C. Jung, C. Lee, M. M. Pai, A. Roth, and R. Vohra, “Moment multicalibration for uncertainty estimation,” arXiv:2008.08037, 2020.
  56. V. Gupta, C. Jung, G. Noarov, M. M. Pai, and A. Roth, “Online multivalid learning: Means, moments, and prediction intervals,” arXiv:2101.01739, 2021.
  57. E. Grycko, “Classification with set-valued decision functions,” in Information and Classification, pp. 218–224, 1993.
  58. J. J. del Coz, J. Díez, and A. Bahamonde, “Learning nondeterministic classifiers,” Journal of Machine Learning Research, vol. 10, no. 79, pp. 2273–2293, 2009.
  59. T. Mortier, M. Wydmuch, K. Dembczyński, E. Hüllermeier, and W. Waegeman, “Efficient set-valued prediction in multi-class classification,” arXiv:1906.08129, 2020.
  60. F. McSherry and K. Talwar, “Mechanism design via differential privacy,” in 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS’07), pp. 94–103, IEEE, 2007.
  61. A. Krizhevsky, G. Hinton, et al., “Learning multiple layers of features from tiny images,” 2009.
  62. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255, 2009.
  63. C. Dwork and A. Roth, “The algorithmic foundations of differential privacy.,” Foundations and Trends in Theoretical Computer Science, vol. 9, no. 3-4, pp. 211–407, 2014.
Citations (10)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now