Adaptive Differentially Quantized Subspace Perturbation (ADQSP): A Unified Framework for Privacy-Preserving Distributed Average Consensus (2312.07947v2)
Abstract: Privacy-preserving distributed average consensus has received significant attention recently due to its wide applicability. Based on the achieved performances, existing approaches can be broadly classified into perfect accuracy-prioritized approaches such as secure multiparty computation (SMPC), and worst-case privacy-prioritized approaches such as differential privacy (DP). Methods of the first class achieve perfect output accuracy but reveal some private information, while methods from the second class provide privacy against the strongest adversary at the cost of a loss of accuracy. In this paper, we propose a general approach named adaptive differentially quantized subspace perturbation (ADQSP) which combines quantization schemes with so-called subspace perturbation. Although not relying on cryptographic primitives, the proposed approach enjoys the benefits of both accuracy-prioritized and privacy-prioritized methods and is able to unify them. More specifically, we show that by varying a single quantization parameter the proposed method can vary between SMPC-type performances and DP-type performances. Our results show the potential of exploiting traditional distributed signal processing tools for providing cryptographic guarantees. In addition to a comprehensive theoretical analysis, numerical validations are conducted to substantiate our results.
- J. Poushter and others, “Smartphone ownership and internet usage continues to climb in emerging economies,” Pew Research Center, vol. 22, pp. 1–44, 2016.
- “Consensus and cooperation in networked multi-agent systems,” IEEEE Proc., vol. 95, no. 1, pp. 215-233, 2007.
- J. N. Tsitsiklis, “Problems in decentralized decision making and computation.,” Tech. Rep., Tech. Rep., Ph.D. dissertation, Massachusetts Institute of Technology, Cambridge, 1984.
- “Federated learning: Challenges, methods, and future directions,” IEEE Signal Process. Magazine, vol. 37, no. 3, pp. 50-60,, 2020.
- “Privacy-preserving distributed speech enhancement for wireless sensor networks by processing in the encrypted domain,” in Proc. Int. Conf. Acoust., Speech, Signal Process., pp. 7005–7009, 2013.
- “Secure and privacy-preserving average consensus,” in Proc. Workshop Cyber-Phys. Syst. Secur. Privacy, pp. 123-129, 2017.
- “Learning with privacy in consensus +++ obfuscation,” IEEE signal process. Lett., vol. 23, no. 9, pp. 1174–1178, 2016.
- M. T. Hale, M. Egerstedt, “Differentially private cloud-based multi-agent optimization with constraints,” in Proc. American Control Conf., pp. 1235-1240, 2015.
- M. T. Hale, M. Egerstedt, “Cloud-enabled differentially private multiagent optimization with constraints,” IEEE Trans. Control Netw. Syst., vol. 5, no. 4, pp. 1693–1706, 2018.
- “Differentially private average consensus: Obstructions, trade-offs, and optimal algorithm design,” Automatica, vol. 81, pp. 221–231, 2017.
- “Secure consensus averaging in sensor networks using random offsets,” in Proc. of the IEEE Int. Conf. on Telec., and Malaysia Int. Conf. on Commun., pp. 556–560, 2007.
- “Differentially private iterative synchronous consensus,” in ACM workshop Privacy electron. Soc., pp. 81–90, 2012.
- “Practical collaborative learning for crowdsensing in the internet of things with differential privacy,” in in Proc. IEEE Conf. Commun. Netw. Secur. (CNS), pp.1–9, 2018.
- “Privacy in distributed average consensus,” IFAC-PapersOnLine, vol. 50, no. 1, pp. 9515-9520, 2017.
- N. Gupta, J. Kat and N. Chopra, “Statistical privacy in distributed average consensus on bounded real inputs,” in ACC, pp 1836-1841, 2019.
- “Privacy-preserving distributed average consensus based on additive secret sharing,” in Proc. Eur. Signal Process. Conf., pp. 1-5, 2019.
- Q. Li and M. G. Christensen, “A privacy-preserving asynchronous averaging algorithm based on shamir’s secret sharing,” in Proc. Eur. Signal Process. Conf., pp. 1-5, 2019.
- Q. Li, R. Heusdens and M. G. Christensen, “Convex optimisation-based privacy-preserving distributed average consensus in wireless sensor networks,” in Proc. Int. Conf. Acoust., Speech, Signal Process., pp. 5895-5899, 2020.
- Q. Li, R. Heusdens and M. G. Christensen, “Privacy-preserving distributed optimization via subspace perturbation: A general framework,” in IEEE Trans. Signal Process., vol. 68, pp. 5983 - 5996, 2020.
- N. E. Manitara and C. N. Hadjicostis, “Privacy-preserving asymptotic average consensus,” in ECC, pp. 760–765, 2013.
- Y. Mo and R. M. Murray, “Privacy preserving average consensus,” IEEE Trans. Automat Contr., vol. 62, no. 2, pp. 753–765, 2017.
- “Privacy-preserving average consensus: privacy analysis and algorithm design,” IEEE Trans. Signal Inf. Process. Netw., vol. 5, no. 1, pp. 127–138, 2019.
- Y. Xiong and Z. Li, “Privacy-preserved average consensus algorithms with edge-based additive perturbations,” Automatica, vol. 140, pp. 110223, 2022.
- Y. Zhou and S. Pu, “Private and accurate decentralized optimization via encrypted and structured functional perturbation,” IEEE Control Syst. Lett., vol. 7, pp. 1339–1344, 2022.
- “Privacy-preserving distributed processing: Metrics, bounds, and algorithms,” in IEEE Trans. Inf. Forensics Security. vol. 16, pp. 2090–2103, 2021.
- C. Dwork, “Differential privacy,” in ICALP, pp. 1–12, 2006.
- “Calibrating noise to sensitivity in private data analysis,” in Proc. Theory of Cryptography Conf. , pp. 265-284, 2006.
- “Two for the price of one: communication efficient and privacy-preserving distributed average consensus using quantization,” in Proc. Eur. Signal Process. Conf., 2022.
- T. M. Cover and J. A. Tomas, Elements of information theory, John Wiley & Sons, 2012.
- “Tightening mutual information-based bounds on generalization error,” IEEE J. Sel. Areas Info. Theory. vol. 1, no. 1, pp. 121–130, 2020.
- “Differential privacy as a mutual information constraint,” in Proc. 23rd ACM SIGSAC Conf. Comput. Commun. Secur., pp 43–54, 2016.
- G. Barthe and B. Kopf, “Information-theoretic bounds for differentially private mechanisms,” in IEEE 24th Computer Security Foundations Symposium, 2011, pp. 191–204.
- “Publishing search logs—a comparative study of privacy guarantees,” IEEE Trans. Knowl. Data. Eng. vol. 24, pp. 520 - 532, 2011.
- Amos Beimel, “On private computation in incomplete networks,” in Structural Information and Communication Complexity. 2005, pp. 18–33, Springer Berlin Heidelberg.
- Q. Li, R. Heusdens and M. G. Christensen, “Communication efficient privacy-preserving distributed optimization using adaptive differential quantization,” Signal Process., 2022.
- “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Foundations and Trends in Machine learning, vol. 3, no. 1, pp. 1–122, 2011.
- “Distributed optimization using the primal-dual method of multipliers,” IEEE Trans. Signal Process., vol. 4, no. 1, pp. 173–187, 2018.
- “Derivation and analysis of the primal-dual method of multipliers based on monotone operator theory,” IEEE Trans. Signal Inf. Process. Netw., vol. 5, no. 2, pp 334-347, 2018.
- E. Ryu, S. P. Boyd, “Primer on monotone operator methods,” Appl. Comput. Math., vol. 15, no. 1, pp. 3-43,, 2016.
- “Quantisation effects in PDMM: A first study for synchronous distributed averaging,” in Proc. Int. Conf. Acoust., Speech, Signal Process., pp. 4237–4241, 2017.
- “Quantisation effects in distributed optimisation,” in Proc. Int. Conf. Acoust., Speech, Signal Process., pp. 3649–3653, 2018.
- L. Schuchman, “Dither signals and their effect on quantization noise,” IEEE Transactions on Communication Technology, vol. 12, no. 4, pp. 162–165, 1964.
- “Perfectly secure message transmission,” J. Assoc. Comput. Mach., vol. 40, no. 1, pp. 17-47,, 1993.
- “Secure multiparty computation and secret sharing,” 2015.
- “What can we learn privately?,” SIAM Journal on Computing, vol. 40, no. 3, pp. 793–826, 2011.
- “Random geometric graphs,” Physical review E, vol. 66, no. 1, pp. 016121, 2002.
- G. Ver Steeg, “Non-parametric entropy estimation toolbox (npeet),” https://github.com/gregversteeg/NPEET, 2000.