Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Distributed Nonlinear Conic Optimisation with partially separable Structure (2405.09490v1)

Published 15 May 2024 in cs.DC

Abstract: In this paper we consider the problem of distributed nonlinear optimisation of a separable convex cost function over a graph subject to cone constraints. We show how to generalise, using convex analysis, monotone operator theory and fixed-point theory, the primal-dual method of multipliers (PDMM), originally designed for equality constraint optimisation and recently extended to include linear inequality constraints, to accommodate for cone constraints. The resulting algorithm can be used to implement a variety of optimisation problems, including the important class of semidefinite programs with partially separable structure, in a fully distributed fashion. We derive update equations by applying the Peaceman-Rachford splitting algorithm to the monotonic inclusion related to the lifted dual problem. The cone constraints are implemented by a reflection method in the lifted dual domain where auxiliary variables are reflected with respect to the intersection of the polar cone and a subspace relating the dual and lifted dual domain. Convergence results for both synchronous and stochastic update schemes are provided and an application of the proposed algorithm is demonstrated to implement an approximate algorithm for maximum cut problems based on semidefinite programming in a fully distributed fashion.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Distributed optimization and statistical learning via the alternating direction method of multipliers. In Foundations and Trends in Machine Learning, 3(1):1–122, 2011.
  2. Gossip Algorithms for Distributed Signal Processing. Proceedings of the IEEE, 98(11):1847–1864, 2010.
  3. Randomized gossip algorithms. IEEE Trans. Information Theory, 52(6):2508–2530, 2006.
  4. Greedy gossip with eavesdropping. IEEE Trans. on Signal Processing, 58(7):3765–3776, July 2010.
  5. Order-optimal consensus through randomized path averaging. IEEE Trans. on Information Theory, 56(10):5150–5167, October 2010.
  6. Analysis of Sum-Weight-Like Algorithms for Averaging in Wireless Sensor Networks. IEEE Trans. Signal Processing, 61(11):2802–2814, 2013.
  7. MAP estimation via agreement on trees: Message-passing and linear programming. IEEE Trans. Information Theory, 51(11):3697–3717, November 2005.
  8. L. Schenato and F. Fiorentin. Average timesynch: A consensus-based protocol for clock synchronization in wireless sensor networks. Automatica, 47(9):1878–1886, September 2011.
  9. Distributed autoregressive moving average graph filters. IEEE Signal Process. Letters, 22(11):1931–1935, November 2015.
  10. Convex optimisation-based privacy-preserving distributed average consensus in wireless sensor networks. In Proc. of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), May 2020.
  11. Privacy-preserving distributed processing: Metrics, bounds and algorithms. IEEE Transactions on Information Forensics and Security, 16:2090–2103, 2021.
  12. Privacy-preserving distributed optimization via subspace perturbation: A general framework. IEEE Trans. on Signal Processing, 68:5983 – 5996, October 2020.
  13. Adaptive differentially quantized subspace perturbation (adqsp): A unified framework for privacy-preserving distributed average consensus. IEEE Transactions on Information Forensics and Security, 19:1780–1793, 2024.
  14. Z. Luo and W. Yu. An introduction to convex optimization for communications and signal processing. IEEE Journal on Selected Areas in Communications, 24(8):1426–1438, August 2006.
  15. G. Zhang and R. Heusdens. Linear coordinate-descent message-passing for quadratic optimization. Neural Computation, 24(12):3340–3370, december 2012.
  16. Derivation and analysis of the primal-dual method of multipliers based on monotone operator theory. IEEE Transactions on Signal and Information Processing over Networks, 5(2):334–347, 2019.
  17. Convergence of stochastic pdmm. In Proceedings IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2023.
  18. Revisiting the Primal-Dual Method of Multipliers for Optimisation Over Centralised Networks. IEEE Trans. Signal and Information Processing over Networks, 8:228–243, 2022.
  19. SCAFFOLD: Stochastic controlled averaging for federated learning. In Hal Daum III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 5132–5143. PMLR, 13–18 Jul 2020.
  20. Fedsplit: an algorithmic framework for fast federated optimization. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 7057–7066. Curran Associates, Inc., 2020.
  21. R. Heusdens and G. Zhang. Distributed optimisation with linear equality and inequality constraints using pdmm. IEEE trans. on Signal and Information Processing over Networks, 2024.
  22. Semidefinite programming approaches for sensor network localization with noisy distance measurements. IEEE Transactions on Automation Science and Engineering, 3(4):360–371, 2006.
  23. A. M. C. So and Y. Ye. Theory of semidefinite programming for sensor network localization. Mathematical Programming, 109:367–384, 2007.
  24. Distributed maximum likelihood sensor network localization. IEEE Transactions on Signal Processing, 62:1424–1437, 2013.
  25. The application of semidefinite programming for detection in cdma. IEEE Journal on Selected Areas in Communications, 19(8):1442–1449, 2001.
  26. Quasi-maximum-likelihood multiuser detection using semi-definite relaxation with application to synchronous cdma. IEEE Transactions on Signal Processing, 50(4):912–922, 2002.
  27. A near-maximum-likelihood decoding algorithm for mimo systems based on semi-definite programming. IEEE Transactions on Information Theory, 53(11):3869–3886, 2007.
  28. Distributed optimal power flow for smart microgrids. IEEE Transactions on Smart Grid, 4(3):1464–1475, 2013.
  29. Rank-one solutions for sdp relaxation of qcqps in power systems. IEEE Transactions on Smart Grid, 10(1):5–15, 2019.
  30. Exploiting sparsity in semidefinite programming via matrix completion i: General framework. SIAM Journal on Optimization, 11(3):647–674, 2001.
  31. Distributed semidefinite programming with application to large-scale system analysis. IEEE Transactions on Automatic Control, 63(4):1045–1058, 2018.
  32. E.K. Ryu and S. Boyd. A primer on monotone operator methods. Applied Computational mathematics, 15(1):3 – 43, 2016.
  33. Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, second edition, 2017. CMS Books in Mathematics.
  34. J. J. Moreau. Deécomposition orthogonale dÕun espace hilbertien selon deux coônes mutuellement polaires. Comptes rendus hebdomadaires des seéances de lÕAcadeémie des sciences, (255):238–240, 1962. hal-01867187.
  35. J. Douglas and H.H. Rachford. On the numerical solution of heat conduction problems in two and three space variables. Transactions of the American Mathematical Society, 82:421–439, 1956.
  36. P.L. Lions and B. Mercier. Splitting algorithms for the sum of two nonlinear operators. SIAM Journal on Numerical Analysis, 16(6):964–979, 1979.
  37. A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Computers & Mathematics with Applications, 2(1):17–40, 1976.
  38. J. Eckstein and M. Fukushima. Some Reformulations and Applications of the Alternating Direction Method of Multipliers. Large Scale Optimization: State of the Art, pages 119–138, 1993.
  39. L.M. Bregman. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Computational Mathematics and Mathematical Physics, 7(3):200–217, 1967.
  40. Function splitting and quadratic approximation of the primal-dual method of multipliers for distributed optimization over graphs. IEEE Trans. on Signal and Information Processing over Networks, 4(4):656–666, December 2018.
  41. V. Berinde. Picard iteration converges faster than Mann iteration for a class of quasi-contractive operators. Fixed Point Theory and Applications, 2004(2):97–105, 2004.
  42. Asynchronous distributed optimizationusing a randomized alternating direction method of multipliers. 52nd IEEE Conference on Decision and Control, pages 3671–3676, December 2013. Firenze, Italy.
  43. A coordinate descent primal-dual algorithmand application to distributed asynchronous optimization. IEEE Trans. on Automatic Control, 61(10):2947–2957, October 2016.
  44. J. Jacod and P. Protter. Probability Essentials. Springer, second edition, 2004.
  45. Random geometric graphs. Phys. Rev. E, 66:016121, Jul 2002.
  46. Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM, 42:1115–1145, 1995.
  47. Approximate graph coloring by semidefinite programming. Technical report, Department of Computer Science, Stanford University, 1994.
  48. A semidefinite programming method for integer convex quadratic minimization. Optimization Letters, 12(3):499–518, 2018.
  49. Quantization effects in distributed optimisation. ICASSP, pages 3649–3653, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets