Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Follower Agnostic Methods for Stackelberg Games (2302.01421v3)

Published 2 Feb 2023 in math.OC, cs.AI, cs.GT, and math.DS

Abstract: In this paper, we present an efficient algorithm to solve online Stackelberg games, featuring multiple followers, in a follower-agnostic manner. Unlike previous works, our approach works even when leader has no knowledge about the followers' utility functions or strategy space. Our algorithm introduces a unique gradient estimator, leveraging specially designed strategies to probe followers. In a departure from traditional assumptions of optimal play, we model followers' responses using a convergent adaptation rule, allowing for realistic and dynamic interactions. The leader constructs the gradient estimator solely based on observations of followers' actions. We provide both non-asymptotic convergence rates to stationary points of the leader's objective and demonstrate asymptotic convergence to a \emph{local Stackelberg equilibrium}. To validate the effectiveness of our algorithm, we use this algorithm to solve the problem of incentive design on a large-scale transportation network, showcasing its robustness even when the leader lacks access to followers' demand.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Inverse optimization with noisy data. Operations Research, 66(3):870–892, 2018.
  2. Jonathan F Bard. Practical bilevel optimization: algorithms and applications, volume 30. Springer Science & Business Media, 2013.
  3. Learning optimal commitment to overcome insecurity. Advances in Neural Information Processing Systems, 27, 2014.
  4. Sample-efficient learning of stackelberg equilibria in general-sum games. Advances in Neural Information Processing Systems, 34:25799–25811, 2021.
  5. Vivek S Borkar. Stochastic approximation: a dynamical systems viewpoint, volume 48. Springer, 2009.
  6. An overview of bilevel optimization. Annals of operations research, 153:235–256, 2007.
  7. On bilevel optimization without lower-level strong convexity. arXiv preprint arXiv:2301.00712, 2023.
  8. Convergence of learning dynamics in stackelberg games. arXiv preprint arXiv:1906.01217, 2019.
  9. Forward and reverse gradient-based hyperparameter optimization. In International Conference on Machine Learning, pages 1165–1173. PMLR, 2017.
  10. Bilevel programming for hyperparameter optimization and meta-learning. In International Conference on Machine Learning, pages 1568–1577. PMLR, 2018.
  11. Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007, 2004.
  12. The theory of learning in games, volume 2. MIT press, 1998.
  13. Finite-dimensional variational inequalities and complementarity problems. Springer, 2003.
  14. On differentiating parameterized argmin and argmax problems with application to bi-level optimization. arXiv preprint arXiv:1607.05447, 2016.
  15. On the iteration complexity of hypergradient computation. In International Conference on Machine Learning, pages 3748–3758. PMLR, 2020.
  16. Approximation methods for bilevel programming. arXiv preprint arXiv:1802.02246, 2018.
  17. Bilevel optimization: Convergence analysis and enhanced design. arxiv e-prints, art. arXiv preprint arXiv:2010.07962, 2020.
  18. Linear convergence of gradient and proximal-gradient methods under the polyak-łojasiewicz condition. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2016, Riva del Garda, Italy, September 19-23, 2016, Proceedings, Part I 16, pages 795–811. Springer, 2016.
  19. Learning and approximating the optimal strategy to commit to. In Algorithmic Game Theory: Second International Symposium, SAGT 2009, Paphos, Cyprus, October 18-20, 2009. Proceedings 2, pages 250–262. Springer, 2009.
  20. John M. Lee. Introduction to Smooth Manifolds. Springer Science+Business Media New York, 2013.
  21. Investigating bi-level optimization for learning and vision from a unified perspective: A survey and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):10045–10067, 2021.
  22. Inducing equilibria via incentives: Simultaneous design-and-play ensures global convergence. arXiv preprint arXiv:2110.01212, 2021.
  23. A value-function-based interior-point method for non-convex bi-level optimization. In International Conference on Machine Learning, pages 6882–6892. PMLR, 2021.
  24. Towards gradient-based bilevel optimization with non-convex followers and beyond. Advances in Neural Information Processing Systems, 34:8662–8675, 2021.
  25. A generic first-order algorithmic framework for bi-level programming beyond lower-level singleton. In International Conference on Machine Learning, pages 6305–6315. PMLR, 2020.
  26. Gradient descent only converges to minimizers. In Conference on learning theory, pages 1246–1257. PMLR, 2016.
  27. Differentiable bilevel programming for stackelberg congestion games. arXiv preprint arXiv:2209.07618, 2022.
  28. Zeroth-order methods for convex-concave min-max problems: Applications to decision-dependent risk minimization. In International Conference on Artificial Intelligence and Statistics, pages 6702–6734. PMLR, 2022.
  29. Yurii Nesterov et al. Lectures on convex optimization, volume 137. Springer, 2018.
  30. Michael Patriksson. The traffic assignment problem: models and methods. Courier Dover Publications, 2015.
  31. Fabian Pedregosa. Hyperparameter optimization with approximate gradient. In International conference on machine learning, pages 737–746. PMLR, 2016.
  32. Learning optimal strategies to commit to. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 2149–2156, 2019.
  33. Learning to play sequential games versus unknown opponents. Advances in Neural Information Processing Systems, 33:8971–8981, 2020.
  34. Truncated back-propagation for bilevel optimization. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1723–1732. PMLR, 2019.
  35. A constrained optimization approach to bilevel optimization with multiple inner minima. arXiv preprint arXiv:2203.01123, 2022.
  36. A review on bilevel optimization: From classical to evolutionary approaches and applications. IEEE Transactions on Evolutionary Computation, 22(2):276–295, 2017.
  37. James C Spall. A one-measurement form of simultaneous perturbation stochastic approximation. Automatica, 33(1):109–112, 1997.
  38. Optimization for data analysis. Cambridge University Press, 2022.
  39. Bome! bilevel optimization made easy: A simple first-order approach. arXiv preprint arXiv:2209.08709, 2022.
  40. Stackelberg-game-based modeling and optimization for supply chain design and operations: A mixed integer bilevel programming framework. Computers & Chemical Engineering, 102:81–95, 2017.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now