2000 character limit reached
An Expectation-Maximization Relaxed Method for Privacy Funnel
Published 1 May 2024 in cs.IT and math.IT | (2405.00616v1)
Abstract: The privacy funnel (PF) gives a framework of privacy-preserving data release, where the goal is to release useful data while also limiting the exposure of associated sensitive information. This framework has garnered significant interest due to its broad applications in characterization of the privacy-utility tradeoff. Hence, there is a strong motivation to develop numerical methods with high precision and theoretical convergence guarantees. In this paper, we propose a novel relaxation variant based on Jensen's inequality of the objective function for the computation of the PF problem. This model is proved to be equivalent to the original in terms of optimal solutions and optimal values. Based on our proposed model, we develop an accurate algorithm which only involves closed-form iterations. The convergence of our algorithm is theoretically guaranteed through descent estimation and Pinsker's inequality. Numerical results demonstrate the effectiveness of our proposed algorithm.
- F. du Pin Calmon and N. Fawaz, “Privacy Against Statistical Inference,” in 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2012, pp. 1401–1408.
- A. Makhdoumi, S. Salamatian, N. Fawaz, and M. Médard, “From the Information Bottleneck to the Privacy Funnel,” in 2014 IEEE Information Theory Workshop (ITW), 2014, pp. 501–505.
- F. P. Calmon, A. Makhdoumi, and M. Médard, “Fundamental Limits of Perfect Privacy,” in 2015 IEEE International Symposium on Information Theory (ISIT), 2015, pp. 1796–1800.
- F. du Pin Calmon, A. Makhdoumi, M. Médard, M. Varia, M. Christiansen, and K. R. Duffy, “Principal Inertia Components and Applications,” IEEE Transactions on Information Theory, vol. 63, no. 8, pp. 5011–5038, 2017.
- S. Asoodeh, M. Diaz, F. Alajaji, and T. Linder, “Estimation Efficiency Under Privacy Constraints,” IEEE Transactions on Information Theory, vol. 65, no. 3, pp. 1512–1534, 2019.
- M. Romanelli, C. Palamidessi, and K. Chatzikokolakis, “Generating Optimal Privacy-Protection Mechanisms via Machine Learning,” arXiv preprint arXiv:1904.01059, 2019.
- A. Tripathy, Y. Wang, and P. Ishwar, “Privacy-Preserving Adversarial Networks,” in 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2019, pp. 495–505.
- M. Bertran, N. Martinez, A. Papadaki, Q. Qiu, M. Rodrigues, G. Reeves, and G. Sapiro, “Adversarially Learned Representations for Information Obfuscation and Inference,” in 36th International Conference on Machine Learning (ICML), vol. 97, 2019, pp. 614–623.
- B. Rodríguez-Gálvez, R. Thobaben, and M. Skoglund, “A Variational Approach to Privacy and Fairness,” in 2021 IEEE Information Theory Workshop (ITW), 2021, pp. 1–6.
- Y. Bu, T. Wang, and G. W. Wornell, “SDP Methods for Sensitivity-Constrained Privacy Funnel and Information Bottleneck Problems,” in 2021 IEEE International Symposium on Information Theory (ISIT), 2021, pp. 49–54.
- R. Blahut, “Computation of Channel Capacity and Rate-Distortion Functions,” IEEE Transactions on Information Theory, vol. 18, no. 4, pp. 460–473, 1972.
- N. Tishby, F. C. Pereira, and W. Bialek, “The Information Bottleneck Method,” in 37th Annual Allerton Conference on Communications, Control and Computing (Allerton), 1999, pp. 368–377.
- L. Chen, S. Wu, J. Ye, H. Wu, W. Zhang, and H. Wu, “Efficient and Provably Convergent Computation of Information Bottleneck: A Semi-Relaxed Approach,” IEEE International Conference on Communications (ICC), 2024, accepted.
- N. Ding and P. Sadeghi, “A Submodularity-Based Agglomerative Clustering Algorithm for the Privacy Funnel,” arXiv preprint arXiv:1901.06629, 2019.
- N. Slonim and N. Tishby, “Agglomerative Information Bottleneck,” in Advances in Neural Information Processing Systems (NIPS), vol. 12, 1999.
- B. Razeghi, F. P. Calmon, D. Gunduz, and S. Voloshynovskiy, “Bottlenecks CLUB: Unifying Information-Theoretic Trade-offs Among Complexity, Leakage, and Utility,” IEEE Transactions on Information Forensics and Security, vol. 18, pp. 2060–2075, 2023.
- B. Razeghi, P. Rahimi, and S. Marcel, “Deep Variational Privacy Funnel: General Modeling with Applications in Face Recognition,” arXiv preprint arXiv:2401.14792, 2024.
- T.-H. Huang, A. El Gamal, and H. El Gamal, “A Linearly Convergent Douglas-Rachford Splitting Solver for Markovian Information-Theoretic Optimization Problems,” IEEE Transactions on Information Theory, vol. 69, no. 5, pp. 3372–3399, 2022.
- T.-H. Huang and H. E. Gamal, “An Efficient Difference-of-Convex Solver for Privacy Funnel,” arXiv preprint arXiv:2403.04778, 2024.
- M. R. Gupta and Y. Chen, “Theory And Use of the EM Algorithm,” Foundations and Trends® in Signal Processing, vol. 4, no. 3, pp. 223–296, 2011.
- A. Kolchinsky, B. D. Tracey, and D. H. Wolpert, “Nonlinear Information Bottleneck,” Entropy, vol. 21, no. 12, p. 1181, 2019.
- W. Ye, H. Wu, S. Wu, Y. Wang, W. Zhang, H. Wu, and B. Bai, “An Optimal Transport Approach to the Computation of the LM Rate,” IEEE Global Communications Conference (GLOBECOM), pp. 239–244, 2022.
- S. Wu, W. Ye, H. Wu, H. Wu, W. Zhang, and B. Bai, “A Communication Optimal Transport Approach to the Computation of Rate Distortion Functions,” IEEE Information Theory Workshop (ITW), 2023.
- C. L. Blake, “UCI Repository of Machine Learning Databases,” https://archive.ics.uci.edu, 1998.
Paper Prompts
Sign up for free to create and run prompts on this paper.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
