Projection-Free Online Convex Optimization with Time-Varying Constraints (2402.08799v1)
Abstract: We consider the setting of online convex optimization with adversarial time-varying constraints in which actions must be feasible w.r.t. a fixed constraint set, and are also required on average to approximately satisfy additional time-varying constraints. Motivated by scenarios in which the fixed feasible set (hard constraint) is difficult to project on, we consider projection-free algorithms that access this set only through a linear optimization oracle (LOO). We present an algorithm that, on a sequence of length $T$ and using overall $T$ calls to the LOO, guarantees $\tilde{O}(T{3/4})$ regret w.r.t. the losses and $O(T{7/8})$ constraints violation (ignoring all quantities except for $T$) . In particular, these bounds hold w.r.t. any interval of the sequence. We also present a more efficient algorithm that requires only first-order oracle access to the soft constraints and achieves similar bounds w.r.t. the entire sequence. We extend the latter to the setting of bandit feedback and obtain similar bounds (as a function of $T$) in expectation.
- Online convex optimization with time-varying constraints and bandit feedback. IEEE Transactions on automatic control, 64(7):2665–2680, 2018.
- A unifying framework for online optimization with long-term constraints. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 33589–33602. Curran Associates, Inc., 2022.
- Online convex optimization in the bandit setting: gradient descent without a gradient. In Proceedings of the sixteenth annual ACM-SIAM symposium on Discrete algorithms, pp. 385–394, 2005.
- Improved regret bounds for projection-free bandit convex optimization. In International Conference on Artificial Intelligence and Statistics, pp. 2196–2206. PMLR, 2020.
- New projection-free algorithms for online convex optimization with adaptive regret guarantees. In Loh, P.-L. and Raginsky, M. (eds.), Proceedings of Thirty Fifth Conference on Learning Theory, volume 178 of Proceedings of Machine Learning Research, pp. 2326–2359. PMLR, 02–05 Jul 2022.
- Projection-free online exp-concave optimization. In Neu, G. and Rosasco, L. (eds.), Proceedings of Thirty Sixth Conference on Learning Theory, volume 195 of Proceedings of Machine Learning Research, pp. 1259–1284. PMLR, 12–15 Jul 2023.
- Online convex optimization with hard constraints: Towards the best of two worlds and beyond. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 36426–36439. Curran Associates, Inc., 2022.
- Hazan, E. Introduction to online convex optimization. arXiv preprint arXiv:1909.05207, 2019.
- Faster projection-free online learning. In Conference on Learning Theory, pp. 1877–1893. PMLR, 2020.
- Efficient learning algorithms for changing environments. In Proceedings of the 26th annual international conference on machine learning, pp. 393–400, 2009.
- Projection-free online learning. In 29th International Conference on Machine Learning, ICML 2012, pp. 521–528, 2012.
- Adaptive algorithms for online convex optimization with long-term constraints. In Balcan, M. F. and Weinberger, K. Q. (eds.), Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pp. 402–411, New York, New York, USA, 20–22 Jun 2016. PMLR.
- Trading regret for efficiency: online convex optimization with long term constraints. The Journal of Machine Learning Research, 13(1):2503–2528, 2012.
- Mhammedi, Z. Efficient projection-free online convex optimization with membership oracle. arXiv preprint arXiv:2111.05818, 2021.
- Mhammedi, Z. Exploiting the curvature of feasible sets for faster projection-free online learning, 2022. URL https://arxiv.org/abs/2205.11470.
- Online convex optimization with time-varying constraints. arXiv preprint arXiv:1702.04783, 2017.
- Shalev-Shwartz, S. et al. Online learning and online convex optimization. Foundations and Trends® in Machine Learning, 4(2):107–194, 2012.
- Safety-aware algorithms for adversarial contextual bandit. In Precup, D. and Teh, Y. W. (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 3280–3288. PMLR, 06–11 Aug 2017. URL https://proceedings.mlr.press/v70/sun17a.html.
- Regret and cumulative constraint violation analysis for online convex optimization with long term constraints. In International Conference on Machine Learning, pp. 11998–12008. PMLR, 2021.
- Regret and cumulative constraint violation analysis for distributed online constrained convex optimization. IEEE Transactions on Automatic Control, 2022.
- A low complexity algorithm with O(T)𝑂𝑇O(\sqrt{T})italic_O ( square-root start_ARG italic_T end_ARG ) regret and O(1)𝑂1O(1)italic_O ( 1 ) constraint violations for online convex optimization with long term constraints. Journal of Machine Learning Research, 21(1):1–24, 2020. URL http://jmlr.org/papers/v21/16-494.html.