Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
112 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Online Allocation with Replenishable Budgets: Worst Case and Beyond (2401.04340v1)

Published 9 Jan 2024 in cs.GT and cs.PF

Abstract: This paper studies online resource allocation with replenishable budgets, where budgets can be replenished on top of the initial budget and an agent sequentially chooses online allocation decisions without violating the available budget constraint at each round. We propose a novel online algorithm, called OACP (Opportunistic Allocation with Conservative Pricing), that conservatively adjusts dual variables while opportunistically utilizing available resources. OACP achieves a bounded asymptotic competitive ratio in adversarial settings as the number of decision rounds T gets large. Importantly, the asymptotic competitive ratio of OACP is optimal in the absence of additional assumptions on budget replenishment. To further improve the competitive ratio, we make a mild assumption that there is budget replenishment every T* >= 1 decision rounds and propose OACP+ to dynamically adjust the total budget assignment for online allocation. Next, we move beyond the worst-case and propose LA-OACP (Learning-Augmented OACP/OACP+), a novel learning-augmented algorithm for online allocation with replenishable budgets. We prove that LA-OACP can improve the average utility compared to OACP/OACP+ when the ML predictor is properly trained, while still offering worst-case utility guarantees when the ML predictions are arbitrarily wrong. Finally, we run simulation studies of sustainable AI inference powered by renewables, validating our analysis and demonstrating the empirical benefits of LA-OACP.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. Shipra Agrawal and Nikhil R Devanur. 2014. Fast algorithms for online stochastic convex programming. In Proceedings of the twenty-sixth annual ACM-SIAM symposium on Discrete algorithms. 1405–1424.
  2. Deep Policies for Online Bipartite Matching: A Reinforcement Learning Approach. Transactions on Machine Learning Research (2022). https://openreview.net/forum?id=mbwm7NdkpO
  3. Differentiable MPC for End-to-end Planning and Control. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2018/file/ba6d843eb4251a4526ce65d1807a9309-Paper.pdf
  4. Brandon Amos and J. Zico Kolter. 2017. OptNet: Differentiable Optimization as a Layer in Neural Networks. In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 70). PMLR, 136–145.
  5. Online Metric Algorithms with Untrusted Predictions. In ICML.
  6. The Multiplicative Weights Update Method: a Meta-Algorithm and Applications. Theory of Computing 8, 6 (2012), 121–164. https://doi.org/10.4086/toc.2012.v008a006
  7. Kamiar Asgari and Michael J. Neely. 2020. Bregman-Style Online Convex Optimization with Energy Harvesting Constraints. Proc. ACM Meas. Anal. Comput. Syst. 4, 3, Article 52 (nov 2020), 25 pages. https://doi.org/10.1145/3428337
  8. Online Resource Allocation under Horizon Uncertainty. In Abstract Proceedings of the 2023 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (Orlando, Florida, United States) (SIGMETRICS ’23). Association for Computing Machinery, New York, NY, USA, 63–64. https://doi.org/10.1145/3578338.3593559
  9. Dual mirror descent for online allocation problems. In International Conference on Machine Learning. PMLR, 613–628.
  10. Santiago R Balseiro and Yonatan Gur. 2019. Learning in repeated auctions with budgets: Regret minimization and equilibrium. Management Science 65, 9 (2019), 3952–3968.
  11. The Best of Many Worlds: Dual Mirror Descent for Online Allocation Problems. Operations Research 0, 0 (May 2022), null. https://doi.org/10.1287/opre.2021.2242
  12. Exploratory combinatorial optimization with reinforcement learning. In AAAI.
  13. Dimitri P Bertsekas. 2014. Constrained optimization and Lagrange multiplier methods. Academic press.
  14. Allan Borodin and Ran El-Yaniv. 2005. Online computation and competitive analysis. cambridge university press.
  15. Online Algorithms with Advice: A Survey. SIGACT News 47, 3 (Aug. 2016), 93–129.
  16. Subgradient methods. lecture notes of EE392o, Stanford University, Autumn Quarter 2004, 01 (2003).
  17. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 1877–1901. https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
  18. Learning to Optimize: A Primer and A Benchmark. arXiv:2103.12828 [math.OC]
  19. Chasing Convex Bodies and Functions with Black-Box Advice. In COLT.
  20. Nikhil R Devanur and Thomas P Hayes. 2009. The adwords problem: online keyword matching with budgeted bidders under random permutations. In Proceedings of the 10th ACM conference on Electronic commerce. 71–78.
  21. Near optimal online algorithms and fast approximation algorithms for resource allocation problems. Journal of the ACM (JACM) 66, 1 (2019), 1–41.
  22. Natural policy gradient primal-dual method for constrained markov decision processes. Advances in Neural Information Processing Systems 33 (2020), 8378–8390.
  23. Adversarial Deep Learning for Online Resource Allocation. ACM Trans. Model. Perform. Eval. Comput. Syst. 6, 4, Article 13 (feb 2022), 25 pages. https://doi.org/10.1145/3494526
  24. Learning Resource Allocation and Pricing for Cloud Profit Maximization. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence (Honolulu, Hawaii, USA) (AAAI’19/IAAI’19/EAAI’19). AAAI Press, Article 929, 8 pages. https://doi.org/10.1609/aaai.v33i01.33017570
  25. Exploration-exploitation in constrained mdps. arXiv preprint arXiv:2003.02189 (2020).
  26. Online stochastic packing applied to display ad allocation. In European Symposium on Algorithms. 182–194.
  27. Jean-Louis Goffin. 1977. On convergence rates of subgradient optimization methods. Mathematical programming 13 (1977), 329–347.
  28. L. Huang. 2020. Fast-Convergent Learning-Aided Control in Energy Harvesting Networks. IEEE Transactions on Mobile Computing 19, 12 (dec 2020), 2793–2803. https://doi.org/10.1109/TMC.2019.2936344
  29. The Power of Online Learning in Stochastic Network Optimization. SIGMETRICS Perform. Eval. Rev. 42, 1 (June 2014), 153–165.
  30. Longbo Huang and Michael J. Neely. 2011. Utility Optimal Scheduling in Energy Harvesting Networks. In MobiHoc.
  31. Stefanus Jasin and Sunil Kumar. 2012. A re-solving heuristic with bounded revenue loss for network revenue management with customer choice. Mathematics of Operations Research 37, 2 (2012), 313–345.
  32. Online Stochastic Optimization with Wasserstein Based Non-stationarity. arXiv preprint arXiv:2012.06961 (2020).
  33. Conservative Contextual Linear Bandits. In Advances in Neural Information Processing Systems, I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2017/file/bdc4626aa1d1df8e14d80d345b2a442d-Paper.pdf
  34. Deep Implicit Layers.
  35. A New Dog Learns Old Tricks: RL Finds Classic Optimization Algorithms. In ICLR. https://openreview.net/forum?id=rkluJ2R9KQ
  36. Ke Li and Jitendra Malik. 2017. Learning to Optimize. In ICLR.
  37. Expert-Calibrated Learning for Online Optimization with Switching Costs. Proc. ACM Meas. Anal. Comput. Syst. 6, 2, Article 28 (Jun 2022), 35 pages.
  38. Robustified Learning for Online Optimization with Memory Costs. In INFOCOM.
  39. Robustness and Consistency in Linear Quadratic Control with Untrusted Predictions. Proc. ACM Meas. Anal. Comput. Syst. 6, 1, Article 18 (feb 2022), 35 pages. https://doi.org/10.1145/3508038
  40. Competitive Online Optimization with Multiple Inventories: A Divide-and-Conquer Approach. Proc. ACM Meas. Anal. Comput. Syst. 6, 2, Article 36 (jun 2022), 28 pages. https://doi.org/10.1145/3530902
  41. Competitive Online Optimization under Inventory Constraints. Proc. ACM Meas. Anal. Comput. Syst. 3, 1, Article 10 (mar 2019), 28 pages. https://doi.org/10.1145/3322205.3311081
  42. Simultaneously Achieving Sublinear Regret and Constraint Violations for Online Convex Optimization with Time-Varying Constraints. Performance Evaluation 152 (2021), 102240. https://doi.org/10.1016/j.peva.2021.102240
  43. Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model. Journal of Machine Learning Research 24, 253, 1–15. http://jmlr.org/papers/v24/23-0069.html
  44. Thodoris Lykouris and Sergei Vassilvitskii. 2021. Competitive Caching with Machine Learned Advice. J. ACM 68, 4, Article 24 (July 2021), 25 pages.
  45. M. J. Neely. 2010a. Stochastic Network Optimization with Application to Communication and Queueing Systems. Morgan & Claypool.
  46. Michael J Neely. 2010b. Universal scheduling for networks with arbitrary traffic, channels, and mobility. In 49th IEEE Conference on Decision and Control (CDC). IEEE, 1822–1829.
  47. California Independent System Operator. 2023. Calfornia Renewable Datasets. https://www.caiso.com/Pages/default.aspx.
  48. Francesco Orabona. 2019. A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019).
  49. D. P. Palomar and M. Chiang. 2007. Alternative Distributed Algorithms for Network Utility Maximization: Framework and Applications. IEEE Trans. Automatic Control 52, 12 (Dec. 2007), 2254–2269.
  50. Lyapunov optimization for energy harvesting wireless sensor communications. IEEE Internet of Things Journal 5, 3 (2018), 1947–1956.
  51. Carbon-aware computing for datacenters. IEEE Transactions on Power Systems 38, 2 (2022), 1270–1280.
  52. Smoothed Online Optimization with Unreliable Predictions. Proc. ACM Meas. Anal. Comput. Syst. 7, 1, Article 12 (mar 2023), 36 pages. https://doi.org/10.1145/3579442
  53. Green ai. Commun. ACM 63, 12 (2020), 54–63.
  54. Learning for Robust Combinatorial Optimization: Algorithm and Application. In INFOCOM.
  55. The theory and practice of revenue management. Vol. 1. Springer.
  56. H. R. Varian. 1992. Microeconomic Analysis. W. W. Norton & Company.
  57. Alexander Wei and Fred Zhang. 2020. Optimal Robustness-Consistency Trade-offs for Learning-Augmented Online Algorithms. In NeurIPS.
  58. Conservative bandits. In International Conference on Machine Learning. PMLR, 1254–1262.
  59. Jianyi Yang and Shaolei Ren. 2023. Learning-Assisted Algorithm Unrolling for Online Optimization with Budget Constraints. In AAAI.
  60. Hao Yu and Michael J Neely. 2019. Learning-aided optimization for energy-harvesting devices with outdated state information. IEEE/ACM Transactions on Networking 27, 4 (2019), 1501–1514.
  61. Hao Yu and Michael J. Neely. 2020. A Low Complexity Algorithm with 𝒪⁢(T)𝒪𝑇\mathcal{O}(\sqrt{T})caligraphic_O ( square-root start_ARG italic_T end_ARG ) Regret and 𝒪⁢(1)𝒪1\mathcal{O}(1)caligraphic_O ( 1 ) Constraint Violations for Online Convex Optimization with Long Term Constraints. Journal of Machine Learning Research 21, 1 (2020), 1–24.
  62. ReLeS: A Neural Adaptive Multipath Scheduler based on Deep Reinforcement Learning. In INFOCOM.
  63. Martin Zinkevich. 2003. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (icml-03). 928–936.

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com