Online Learning and Solving Infinite Games with an ERM Oracle (2307.01689v2)
Abstract: While ERM suffices to attain near-optimal generalization error in the stochastic learning setting, this is not known to be the case in the online learning setting, where algorithms for general concept classes rely on computationally inefficient oracles such as the Standard Optimal Algorithm (SOA). In this work, we propose an algorithm for online binary classification setting that relies solely on ERM oracle calls, and show that it has finite regret in the realizable setting and sublinearly growing regret in the agnostic setting. We bound the regret in terms of the Littlestone and threshold dimensions of the underlying concept class. We obtain similar results for nonparametric games, where the ERM oracle can be interpreted as a best response oracle, finding the best response of a player to a given history of play of the other players. In this setting, we provide learning algorithms that only rely on best response oracles and converge to approximate-minimax equilibria in two-player zero-sum games and approximate coarse correlated equilibria in multi-player general-sum games, as long as the game has a bounded fat-threshold dimension. Our algorithms apply to both binary-valued and real-valued games and can be viewed as providing justification for the wide use of double oracle and multiple oracle algorithms in the practice of solving large games.
- Private pac learning implies finite littlestone dimension. In Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, pages 852–860, 2019.
- Adversarial laws of large numbers and optimal regret in online classification. In Proceedings of the 53rd annual ACM SIGACT symposium on theory of computing, pages 447–455, 2021.
- The multiplicative weights update method: a meta-algorithm and applications. Theory of computing, 8(1):121–164, 2012.
- Improved generalization bounds for adversarially robust learning. The Journal of Machine Learning Research, 23(1):7897–7927, 2022.
- The pigenhole principle and multicolor ramsey numbers. arXiv preprint arXiv:2108.08410, 2021.
- Agnostic online learning. In COLT, volume 3, page 1, 2009.
- Majorizing measures, sequential complexities, and online learning. In Conference on Learning Theory, pages 587–590. PMLR, 2021.
- Prediction, learning, and games. Cambridge University Press, Cambridge; New York, 2006.
- Settling the complexity of computing two-player nash equilibria. Journal of the ACM (JACM), 56(3):1–57, 2009.
- Fast rates for nonparametric online learning: from realizability to learning in games. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing, pages 846–859, 2022.
- The Complexity of Computing a Nash Equilibrium. SIAM Journal on Computing, 39(1):195, 2009.
- Learning and inference in the presence of corrupted inputs. In Conference on Learning Theory, pages 637–657. PMLR, 2015.
- Optimal mistake bound learning is hard. Information and Computation, 144(1):66–82, 1998.
- Developing, evaluating and scaling learning agents in multi-agent environments. AI Commun., 35(4):271–284, 2022.
- Online learning with simple predictors and a combinatorial characterization of minimax in 0/1 games. In Conference on Learning Theory, pages 2289–2314. PMLR, 2021.
- A simple adaptive procedure leading to correlated equilibrium. Econometrica, 68(5):1127–1150, 2000.
- Existence of correlated equilibria. Mathematics of Operations Research, 14(1):18–25, 1989.
- The computational power of optimization in online learning. In Proceedings of the forty-eighth annual ACM symposium on Theory of Computing, pages 128–141, 2016.
- Wilfrid Hodges et al. A shorter model theory. Cambridge university press, 1997.
- Fat shattering dimension and covering numbers. Lecture 16 notes from CMSC 35900 (Spring 2008) Learning Theory, 2008. URL https://home.ttic.edu/~tewari/lectures/lecture16.pdf.
- Nick Littlestone. Learning Quickly When Irrelevant Attributes Abound: A New Linear-threshold Algorithm. Machine Learning, 2:285–318, 1988.
- Pasin Manurangsi. Improved inapproximability of vc dimension and littlestone’s dimension via (unbalanced) biclique. arXiv preprint arXiv:2211.01443, 2022.
- Inapproximability of VC dimension and Littlestone’s dimension. In Conference on Learning Theory, pages 1432–1460. PMLR, 2017.
- Planning in the presence of cost functions controlled by an adversary. In Proceedings of the 20th International Conference on Machine Learning (ICML-03), pages 536–543, 2003.
- Computing correlated equilibria in multi-player games. Journal of the ACM (JACM), 55(3):1–29, 2008.
- Online learning: Random averages, combinatorial parameters, and learnability. Advances in Neural Information Processing Systems, 23, 2010.
- Combinatorics of random processes and sections of convex bodies. Annals of Mathematics, pages 603–648, 2006.
- Boosting: Foundations and algorithms. Kybernetes, 42(1):164–166, 2013.
- Understanding machine learning: From theory to algorithms. Cambridge university press, 2014.
- Saharon Shelah. Classification theory: and the number of non-isomorphic models. Elsevier, 1990.
- Angelos Assos (4 papers)
- Idan Attias (21 papers)
- Yuval Dagan (37 papers)
- Constantinos Daskalakis (111 papers)
- Maxwell Fishelson (10 papers)