Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
131 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning-Augmented Decentralized Online Convex Optimization in Networks (2306.10158v3)

Published 16 Jun 2023 in cs.LG and math.OC

Abstract: This paper studies decentralized online convex optimization in a networked multi-agent system and proposes a novel algorithm, Learning-Augmented Decentralized Online optimization (LADO), for individual agents to select actions only based on local online information. LADO leverages a baseline policy to safeguard online actions for worst-case robustness guarantees, while staying close to the ML policy for average performance improvement. In stark contrast with the existing learning-augmented online algorithms that focus on centralized settings, LADO achieves strong robustness guarantees in a decentralized setting. We also prove the average cost bound for LADO, revealing the tradeoff between average performance and worst-case robustness and demonstrating the advantage of training the ML policy by explicitly considering the robustness requirement.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. Decentralized voltage control of power systems using multi-agent systems. Journal of Modern Power Systems and Clean Energy, 8(2):249–259, 2020.
  2. Stability constrained reinforcement learning for real-time voltage control. arXiv preprint arXiv:2109.14854, 2021.
  3. Online distributed convex optimization on dynamic networks. IEEE Transactions on Automatic Control, 61(11):3545–3550, 2016.
  4. The computational sprinting game. In ASPLOS, 2016.
  5. Single and multi-agent deep reinforcement learning for ai-enabled wireless networks: A tutorial. IEEE Communications Surveys & Tutorials, 23(2):1226–1252, 2021.
  6. Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks. IEEE Journal on Selected Areas in Communications, 37(10):2239–2250, 2019.
  7. A collaborative multi-agent reinforcement learning anti-jamming algorithm in wireless networks. IEEE wireless communications letters, 8(4):1024–1027, 2019.
  8. Clearance pricing optimization for a fast-fashion retailer. Operations research, 60(6):1404–1422, 2012.
  9. Optimal pricing in networks with externalities. Operations Research, 60(4):883–905, 2012.
  10. A saddle point algorithm for networked online convex optimization. IEEE Transactions on Signal Processing, 63(19):5149–5164, 2015.
  11. Decentralized online convex optimization based on signs of relative states. Automatica, 129:109676, 2021.
  12. Collaborative deep reinforcement learning. arXiv preprint arXiv:1702.05796, 2017.
  13. Decentralized online convex optimization with feedback delays. IEEE Transactions on Automatic Control, 67(6):2889–2904, 2021.
  14. Trading regret for efficiency: Online convex optimization with long term constraints. J. Mach. Learn. Res., 13(1):2503–2528, sep 2012.
  15. An online algorithm for smoothed online convex optimization. SIGMETRICS Perform. Eval. Rev., 47(2):6–8, December 2019.
  16. Beyond online balanced descent: An optimal algorithm for smoothed online optimization. In NeurIPS, volume 32, 2019.
  17. Revisiting smoothed online learning. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
  18. Online optimization with memory and competitive control. Advances in Neural Information Processing Systems, 33:20636–20647, 2020.
  19. Online optimization with feedback delay and nonlinear switching cost. Proc. ACM Meas. Anal. Comput. Syst., 6(1), feb 2022.
  20. Smoothed online convex optimization in high dimensions via online balanced descent. In COLT, 2018.
  21. Distributed online convex optimization with an aggregative variable. IEEE Transactions on Control of Network Systems, 2021.
  22. A new dog learns old tricks: RL finds classic optimization algorithms. In ICLR, 2019.
  23. Exploratory combinatorial optimization with reinforcement learning. In AAAI, 2020.
  24. Reles: A neural adaptive multipath scheduler based on deep reinforcement learning. In INFOCOM, 2019.
  25. Learning for robust combinatorial optimization: Algorithm and application. In INFOCOM, 2022.
  26. Fully decentralized multi-agent reinforcement learning with networked agents. In International Conference on Machine Learning, pages 5872–5881. PMLR, 2018.
  27. Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, pages 321–384, 2021.
  28. A review of cooperative multi-agent deep reinforcement learning. Applied Intelligence, pages 1–46, 2022.
  29. Robustified learning for online optimization with memory costs. In INFOCOM, 2023.
  30. Chasing convex bodies and functions with black-box advice. In COLT, 2022.
  31. Online algorithms with advice: A survey. SIGACT News, 47(3):93–129, August 2016.
  32. Optimal robustness-consistency trade-offs for learning-augmented online algorithms. In NeurIPS, 2020.
  33. The primal-dual method for learning augmented algorithms. Advances in Neural Information Processing Systems, 33:20083–20094, 2020.
  34. Competitive caching with machine learned advice. J. ACM, 68(4), July 2021.
  35. Distributed online convex optimization with compressed communication. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 34492–34504. Curran Associates, Inc., 2022.
  36. Decentralized online convex optimization in networked systems. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 13356–13393. PMLR, 17–23 Jul 2022.
  37. Conservative bandits. In International Conference on Machine Learning, pages 1254–1262. PMLR, 2016.
  38. A reduction-based framework for conservative bandits and reinforcement learning. In International Conference on Learning Representations, 2021.
  39. Conservative exploration in reinforcement learning. In Silvia Chiappa and Roberto Calandra, editors, Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pages 1431–1441. PMLR, 26–28 Aug 2020.
  40. Dynamic right-sizing for power-proportional data centers. In INFOCOM, 2011.
  41. Exploiting spatio-temporal diversity for water saving in geo-distributed data centers. IEEE Transactions on Cloud Computing, 6(3):734–746, 2018.
  42. Meta. Sustainability report, 2021, https://sustainability.fb.com/.
  43. Beyond online balanced descent: An optimal algorithm for smoothed online optimization. Advances in Neural Information Processing Systems, 32, 2019.
  44. Control regularization for reduced variance reinforcement learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1141–1150. PMLR, 09–15 Jun 2019.
  45. Smooth imitation learning for online sequence prediction. In ICML, 2016.
  46. A reduction-based framework for conservative bandits and reinforcement learning. In International Conference on Learning Representations, 2022.
  47. Robust learning-augmented caching: An experimental study. In ICML, 2021.
  48. A regression approach to learning-augmented online algorithms. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
  49. Improving online algorithms via ml predictions. In NeurIPS, 2018.
  50. Learning robust algorithms for online allocation problems using adversarial training. In https://arxiv.org/abs/2010.08418, 2020.
  51. Smoothed online optimization with unreliable predictions. Proc. ACM Meas. Anal. Comput. Syst., 7(1), mar 2023.
  52. Online metric algorithms with untrusted predictions. In ICML, 2020.
  53. Bounded-regret MPC via perturbation analysis: Prediction error, constraints, and nonlinearity. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
  54. Differentiable convex optimization layers. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
  55. Carbon-aware computing for datacenters. IEEE Transactions on Power Systems, 38(2):1270–1280, 2023.
  56. U.S. Department of Engergy. What is environmental justice?, https://www.energy.gov/lm/what-environmental-justice.
  57. Balanced control strategies for interconnected heterogeneous battery systems. IEEE Transactions on Sustainable Energy, 7(1):189–199, 2016.
  58. Resource central: Understanding and predicting workloads for improved resource management in large cloud platforms. In Proceedings of the 26th Symposium on Operating Systems Principles, pages 153–167, 2017.
  59. The national solar radiation data base (nsrdb). Renewable and sustainable energy reviews, 89:51–60, 2018.
  60. Photovoltaic and solar power forecasting for smart grid energy management. CSEE Journal of Power and Energy Systems, 1(4):38–46, 2015.
  61. Wind turbine blade efficiency and power calculation with electrical analogy. International Journal of Scientific and Research Publications, 2(2):1–5, 2012.
  62. Convex optimization and approximation. https://ee227c.github.io/notes/ee227c-notes.pdf, 2018.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now