Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Distributed Difference of Convex Optimization (2407.16728v1)

Published 23 Jul 2024 in math.OC, cs.AI, cs.DC, cs.SY, and eess.SY

Abstract: In this article, we focus on solving a class of distributed optimization problems involving $n$ agents with the local objective function at every agent $i$ given by the difference of two convex functions $f_i$ and $g_i$ (difference-of-convex (DC) form), where $f_i$ and $g_i$ are potentially nonsmooth. The agents communicate via a directed graph containing $n$ nodes. We create smooth approximations of the functions $f_i$ and $g_i$ and develop a distributed algorithm utilizing the gradients of the smooth surrogates and a finite-time approximate consensus protocol. We term this algorithm as DDC-Consensus. The developed DDC-Consensus algorithm allows for non-symmetric directed graph topologies and can be synthesized distributively. We establish that the DDC-Consensus algorithm converges to a stationary point of the nonconvex distributed optimization problem. The performance of the DDC-Consensus algorithm is evaluated via a simulation study to solve a nonconvex DC-regularized distributed least squares problem. The numerical results corroborate the efficacy of the proposed algorithm.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. V. Khatana and M. V. Salapaka, “D-distadmm: A o(1/k) distributed admm for distributed optimization in directed graph topologies,” in 2020 59th IEEE Conference on Decision and Control (CDC), 2020, pp. 2992–2997.
  2. V. Khatana, G. Saraswat, S. Patel, and M. V. Salapaka, “Gradient-consensus method for distributed optimization in directed multi-agent networks,” in 2020 American Control Conference (ACC), 2020, pp. 4689–4694.
  3. P. Hartman, “On functions representable as a difference of convex functions.” Pacific Journal of Mathematics, vol. 9, no. 3, pp. 707–713, 1959.
  4. M. Nouiehed, J.-S. Pang, and M. Razaviyayn, “On the pervasiveness of difference-convexity in optimization and statistics,” Mathematical Programming, vol. 174, no. 1, pp. 195–222, 2019.
  5. S. Merkli, A. Domahidi, J. L. Jerez, M. Morari, and R. S. Smith, “Fast ac power flow optimization using difference of convex functions programming,” IEEE Transactions on Power Systems, vol. 33, no. 1, pp. 363–372, 2017.
  6. H. An and L. Thi, “Solving large scale molecular distance geometry problems by a smoothing technique via the gaussian transform and dc programming,” Journal of Global Optimization, vol. 27, no. 4, pp. 375–397, 2003.
  7. T. H. L. An and P. D. Tao, “Dc optimization approaches via markov models for restoration of signal (1-d) and (2-d),” in Advances in Convex Analysis and Global Optimization.   Springer, 2001, pp. 303–317.
  8. L. T. Hoai An and P. D. Tao, “Dc programming approach for multicommodity network optimization problems with step increasing cost functions,” Journal of Global Optimization, vol. 22, no. 1, pp. 205–232, 2002.
  9. H. A. Le Thi and T. P. Dinh, “A continuous approch for globally solving linearly constrained quadratic,” Optimization, vol. 50, no. 1-2, pp. 93–120, 2001.
  10. J. N. Tsitsiklis, “Problems in decentralized decision making and computation.” Massachusetts Inst of Tech Cambridge Lab for Information and Decision Systems, Tech. Rep., 1984.
  11. S. Pu, W. Shi, J. Xu, and A. Nedic, “Push-pull gradient methods for distributed optimization in networks,” IEEE Transactions on Automatic Control, 2020.
  12. V. Khatana and M. V. Salapaka, “Dc-distadmm: Admm algorithm for constrained optimization over directed graphs,” IEEE Transactions on Automatic Control, vol. 68, no. 9, pp. 5365–5380, 2023.
  13. V. Khatana, G. Saraswat, S. Patel, and M. V. Salapaka, “Gradconsensus: Linearly convergent algorithm for reducing disagreement in multi-agent optimization,” IEEE Transactions on Network Science and Engineering, vol. 11, no. 1, pp. 1251–1264, 2024.
  14. J. Zeng and W. Yin, “On nonconvex decentralized gradient descent,” IEEE Transactions on Signal Processing, vol. 66, no. 11, pp. 2834–2848, 2018.
  15. R. Xin, U. A. Khan, and S. Kar, “A fast randomized incremental gradient method for decentralized nonconvex optimization,” IEEE Transactions on Automatic Control, vol. 67, pp. 5150–5165, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:226282498
  16. Y. Yan, C. Niu, Y. Ding, Z. Zheng, F. Wu, G. Chen, S. Tang, and Z. Wu, “Distributed non-convex optimization with sublinear speedup under intermittent client availability,” INFORMS J. Comput., vol. 36, pp. 185–202, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:211146497
  17. G. Scutari and Y. Sun, “Distributed nonconvex constrained optimization over time-varying digraphs,” Mathematical Programming, vol. 176, pp. 497–544, 2019.
  18. E. Gorbunov, K. P. Burlachenko, Z. Li, and P. Richtárik, “Marina: Faster non-convex distributed learning with compression,” in International Conference on Machine Learning.   PMLR, 2021, pp. 3788–3798.
  19. X. Jiang, X. Zeng, J. Sun, and J. Chen, “Distributed proximal gradient algorithm for nonconvex optimization over time-varying networks,” IEEE Transactions on Control of Network Systems, vol. 10, no. 2, pp. 1005–1017, 2023.
  20. P. D. Lorenzo and G. Scutari, “Next: In-network nonconvex optimization,” IEEE Transactions on Signal and Information Processing over Networks, vol. 2, pp. 120–136, 2016. [Online]. Available: https://api.semanticscholar.org/CorpusID:3261082
  21. S. Chen, A. García, and S. Shahrampour, “On distributed nonconvex optimization: Projected subgradient method for weakly convex problems in networks,” IEEE Transactions on Automatic Control, vol. 67, pp. 662–675, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:234356225
  22. A. Alvarado, G. Scutari, and J.-S. Pang, “A new decomposition method for multiuser dc-programming and its applications,” IEEE Transactions on Signal Processing, vol. 62, no. 11, pp. 2984–2998, 2014.
  23. H. A. Le Thi, T. P. Dinh, H. M. Le, and X. T. Vo, “Dc approximation approaches for sparse optimization,” European Journal of Operational Research, vol. 244, no. 1, pp. 26–46, 2015.
  24. P. D. Tao and L. H. An, “Convex analysis approach to dc programming: theory, algorithms and applications,” Acta mathematica vietnamica, vol. 22, no. 1, pp. 289–355, 1997.
  25. L. T. H. An and P. D. Tao, “The dc (difference of convex functions) programming and dca revisited with dc models of real world nonconvex optimization problems,” Annals of operations research, vol. 133, pp. 23–46, 2005.
  26. M. Ahn, J.-S. Pang, and J. Xin, “Difference-of-convex learning: directional stationarity, optimality, and sparsity,” SIAM Journal on Optimization, vol. 27, no. 3, pp. 1637–1665, 2017.
  27. P. D. Nhat, H. M. Le, and H. A. Le Thi, “Accelerated difference of convex functions algorithm and its application to sparse binary logistic regression.” in IJCAI, 2018, pp. 1369–1375.
  28. T. Sun, P. Yin, L. Cheng, and H. Jiang, “Alternating direction method of multipliers with difference of convex functions,” Advances in Computational Mathematics, vol. 44, pp. 723–744, 2018.
  29. K. Sun and X. A. Sun, “Algorithms for difference-of-convex programs based on difference-of-moreau-envelopes smoothing,” INFORMS Journal on Optimization, vol. 5, no. 4, pp. 321–339, 2023.
  30. B. Wen, X. Chen, and T. K. Pong, “A proximal difference-of-convex algorithm with extrapolation,” Computational optimization and applications, vol. 69, pp. 297–324, 2018.
  31. V. Yadav and M. V. Salapaka, “Distributed protocol for determining when averaging consensus is reached,” in 45th Annual Allerton Conf, 2007, pp. 715–720.
  32. M. Prakash, S. Talukdar, S. Attree, S. Patel, and M. V. Salapaka, “Distributed stopping criterion for ratio consensus,” in 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2018, pp. 131–135.
  33. M. Prakash, S. Talukdar, S. Attree, V. Yadav, and M. V. Salapaka, “Distributed stopping criterion for consensus in the presence of delays,” IEEE Transactions on Control of Network Systems, vol. 7, no. 1, pp. 85–95, 2020.
  34. J. Melbourne, G. Saraswat, V. Khatana, S. Patel, and M. V. Salapaka, “On the geometry of consensus algorithms with application to distributed termination in higher dimension,” in the proceedings of International Federation of Automatic Control (IFAC), 2020. [Online]. Available: http://box5779.temp.domains/~jamesmel/wp-content/uploads/2019/11/Vector_Consensus__IFAC_-1.pdf
  35. ——, “Convex decreasing algorithms: Distributed synthesis and finite-time termination in higher dimension,” IEEE Transactions on Automatic Control, vol. 69, no. 6, pp. 3960–3967, 2024.
  36. D. Kempe, A. Dobra, and J. Gehrke, “Gossip-based computation of aggregate information,” in 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings.   IEEE, 2003, pp. 482–491.
  37. G. Saraswat, V. Khatana, S. Patel, and M. V. Salapaka, “Distributed finite-time termination for consensus algorithm in switching topologies,” IEEE Transactions on Network Science and Engineering, vol. 10, no. 1, pp. 489–499, 2023.
  38. P. Erdős and A. Rényi, “On the evolution of random graphs,” Publ. Math. Inst. Hung. Acad. Sci, vol. 5, no. 1, pp. 17–60, 1960.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now