Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Understanding Iterative Combinatorial Auction Designs via Multi-Agent Reinforcement Learning (2402.19420v2)

Published 29 Feb 2024 in cs.GT, cs.AI, and cs.MA

Abstract: Iterative combinatorial auctions are widely used in high stakes settings such as spectrum auctions. Such auctions can be hard to analyze, making it difficult for bidders to determine how to behave and for designers to optimize auction rules to ensure desirable outcomes such as high revenue or welfare. In this paper, we investigate whether multi-agent reinforcement learning (MARL) algorithms can be used to understand iterative combinatorial auctions, given that these algorithms have recently shown empirical success in several other domains. We find that MARL can indeed benefit auction analysis, but that deploying it effectively is nontrivial. We begin by describing modelling decisions that keep the resulting game tractable without sacrificing important features such as imperfect information or asymmetry between bidders. We also discuss how to navigate pitfalls of various MARL algorithms, how to overcome challenges in verifying convergence, and how to generate and interpret multiple equilibria. We illustrate the promise of our resulting approach by using it to evaluate a specific rule change to a clock auction, finding substantially different auction outcomes due to complex changes in bidders' behavior.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Michael Abramowicz. 2020. Modeling Settlement Bargaining with Algorithmic Game Theory. Research Paper 2020-80. GWU Legal Studies.
  2. Learning to Play No-Press Diplomacy with Best Response Policy Iteration. arXiv:2006.04635 [cs.LG]
  3. Lawrence M. Ausubel. 2004. An Efficient Ascending-Bid Auction for Multiple Objects. American Economic Review 94, 5 (December 2004), 1452–1475.
  4. Lawrence M. Ausubel and Oleg Baranov. 2020. Revealed preference and activity rules in dynamic auctions. International Economic Review 61, 2 (2020), 471–502.
  5. The clock-proxy auction: A practical combinatorial auction design. In Handbook of spectrum auction design. MIT, 120–140.
  6. Human-level play in the game of Diplomacy by combining language models with strategic reasoning. Science 378 (2022), 1067 – 1074.
  7. Martino Banchio and Andrzej Skrzypacz. 2022. Artificial Intelligence and Auction Design. In Proceedings of the 23rd ACM Conference on Economics and Computation (EC ’22). Association for Computing Machinery, New York, NY, USA, 30–31. https://doi.org/10.1145/3490486.3538244
  8. The Hanabi challenge: A new frontier for AI research. Artificial Intelligence 280 (2020), 103216. https://doi.org/10.1016/j.artint.2019.103216
  9. Dimitris Bertsimas and Alex Paskov. 2022. World-class interpretable poker. Machine Learning 111, 8 (2022), 3063–3083. https://doi.org/10.1007/s10994-022-06179-8
  10. Learning equilibria in symmetric auction games using artificial neural networks. Nature Machine Intelligence 3, 8 (2021), 687–695. https://doi.org/10.1038/s42256-021-00365-4
  11. Computing Bayes–Nash Equilibrium Strategies in Auction Games via Simultaneous Online Dual Averaging. Operations Research (2023).
  12. Taming the Communication and Computation Complexity of Combinatorial Auctions: The FUEL Bid Language. Manag. Sci. 69 (2022), 2217–2238.
  13. Vitor Bosshard and Sven Seuken. 2021. The Cost of Simple Bidding in Combinatorial Auctions. In Proceedings of the 22nd ACM Conference on Economics and Computation. Association for Computing Machinery, New York, NY, USA, 157. https://doi.org/10.1145/3465456.3467609
  14. Heads-up limit hold’em poker is solved. Science 347, 6218 (2015), 145–149. https://www.science.org/doi/abs/10.1126/science.1259433
  15. Noam Brown and Tuomas Sandholm. 2018. Superhuman AI for heads-up no-limit poker: Libratus beats top professionals. Science 359, 6374 (2018), 418–424. https://www.science.org/doi/abs/10.1126/science.aao1733
  16. Noam Brown and Tuomas Sandholm. 2019a. Solving imperfect-information games via discounted regret minimization. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence (AAAI’19/IAAI’19/EAAI’19). AAAI Press, Article 225, 8 pages. https://doi.org/10.1609/aaai.v33i01.33011829
  17. Noam Brown and Tuomas Sandholm. 2019b. Superhuman AI for multiplayer poker. Science 365, 6456 (2019), 885–890. https://doi.org/10.1126/science.aay2400
  18. Greedy bidding strategies for keyword auctions. In Proceedings of the 8th ACM Conference on Electronic Commerce. 262–271.
  19. FCC. 2015. Application Procedures for Broadcast Incentive Auction Scheduled to Begin on March 29, 2016. https://www.fcc.gov/document/application-procedures-broadcast-incentive-auction. FCC Public Notice DA 15-1183 (2015).
  20. FCC. 2019. Auction 102 Clock Phase Technical Guide. https://www.fcc.gov/file/14284/download. (2019).
  21. No-Regret Learning and Mixed Nash Equilibria: They Do Not Mix. In Proceedings of the 34th International Conference on Neural Information Processing Systems (NIPS’20). Article 117, 12 pages.
  22. John C. Harsanyi and Reinhard Selten. 1988. A General Theory of Equilibrium Selection in Games. MIT Press.
  23. Strong Activity Rules for Iterative Combinatorial Auctions. Computers & Operations Research 37 (07 2010), 1271–1284.
  24. Deep Models of Interactions Across Sets. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80), Jennifer Dy and Andreas Krause (Eds.). PMLR, 1909–1918.
  25. ISED. 2012. Consultation on a Licensing Framework for Mobile Broadband Services (MBS) — 700 MHz Band. https://ised-isde.canada.ca/site/spectrum-management-telecommunications/en/spectrum-allocation/auctions/700-mhz-2014/consultation-licensing-framework-mobile-broadband-services-mbs-700-mhz-band. (2012). Accessed 2024-02-12.
  26. ISED. 2018. Technical, Policy and Licensing Framework for Spectrum in the 600 MHz Band. https://ised-isde.canada.ca/site/spectrum-management-telecommunications/sites/default/files/attachments/2022/SLPB-002-18-600MHz-decision-e.pdf. Spectrum Management and Telecommunications SLPB-002-18 (2018). Accessed 2024-02-12.
  27. ISED. 2021. Policy and Licensing Framework for Spectrum in the 3500 MHz Band. https://ised-isde.canada.ca/site/spectrum-management-telecommunications/en/spectrum-allocation/policy-and-licensing-framework-spectrum-3500-mhz-band. Spectrum Management and Telecommunications SLPB-001-20 (26 4 2021). Accessed 2024-02-12.
  28. ISED. 2022. Policy and Licensing Framework for Spectrum in the 3800 MHz Band. https://ised-isde.canada.ca/site/spectrum-management-telecommunications/en/spectrum-allocation/policy-and-licensing-framework-spectrum-3800-mhz-band. Spectrum Management and Telecommunications SPB-002-22 (06 2022). Accessed 2024-02-12.
  29. David M. Kreps and Robert Wilson. 1982. Sequential Equilibria. Econometrica 50, 4 (1982), 863–894. http://www.jstor.org/stable/1912767
  30. OpenSpiel: A Framework for Reinforcement Learning in Games. CoRR abs/1908.09453 (2019). arXiv:1908.09453 [cs.LG] http://arxiv.org/abs/1908.09453
  31. Monte Carlo sampling for regret minimization in extensive games. Advances in neural information processing systems 22 (2009).
  32. Jonathan D. Levin and Andrzej Skrzypacz. 2016. Properties of the Combinatorial Clock Auction. The American Economic Review 106 (2016), 2528–2551. https://doi.org/doi:10.1257/aer.20141212
  33. Combining Tree-Search, Generative Models, and Nash Bargaining Concepts in Game-Theoretic Reinforcement Learning. ArXiv abs/2302.00797 (2023).
  34. Paul Milgrom. 2000. Putting Auction Theory to Work: The Simultaneous Ascending Auction. Journal of Political Economy 108 (12 2000). https://doi.org/10.1086/262118
  35. Paul Milgrom. 2004. Putting Auction Theory to Work. Cambridge University Press.
  36. Hindsight and sequential rationality of correlated play. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 5584–5594.
  37. Incentive Auction Design Alternatives: A Simulation Study. Management Science. forthcoming.
  38. Bidding efficiently in Simultaneous Ascending Auctions with budget and eligibility constraints using Simultaneous Move Monte Carlo Tree Search. arXiv:2307.11428 [cs.GT]
  39. Combinatorial Auctions in Practice. SSRN Electronic Journal (2021). https://doi.org/10.2139/ssrn.3844338
  40. Mastering the game of Stratego with model-free multiagent reinforcement learning. Science 378 (2022), 990 – 996.
  41. Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI 3 (2016), 40.
  42. Computing pure Bayesian-Nash equilibria in games with finite actions and continuous types. Artificial Intelligence 195 (2013), 106–139. https://doi.org/10.1016/j.artint.2012.09.007
  43. Frank Riedel and Elmar Wolfstetter. 2006. Immediate demand reduction in simultaneous ascending-bid auctions: a uniqueness result. Economic Theory 29, 3 (2006), 721–726.
  44. Proximal Policy Optimization Algorithms. CoRR abs/1707.06347 (2017). arXiv:1707.06347 http://arxiv.org/abs/1707.06347
  45. Mastering the game of Go with deep neural networks and tree search. Nature 529 (2016), 484–489.
  46. Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. ArXiv abs/1712.01815 (2017).
  47. Policy Gradient Methods for Reinforcement Learning with Function Approximation. In Advances in Neural Information Processing Systems, S. Solla, T. Leen, and K. Müller (Eds.), Vol. 12. MIT Press.
  48. Oskari Tammelin. 2014. Solving Large Imperfect Information Games Using CFR+. arXiv:1407.5042 [cs.GT]
  49. Discovering Auctions: Contributions of Paul Milgrom and Robert Wilson. The Scandinavian Journal of Economics 123, 3 (2021), 709–750. https://doi.org/10.1111/sjoe.12441
  50. The Positronic Economist: A Computational System for Analyzing Economic Mechanisms. Proceedings of the AAAI Conference on Artificial Intelligence 31, 1 (Feb. 2017). https://doi.org/10.1609/aaai.v31i1.10592
  51. David R. M. Thompson and Kevin Leyton-Brown. 2017. Computational Analysis of Perfect-Information Position Auctions. Games and Economic Behavior 102 (2017), 583–623.
  52. Platform Behavior under Market Shocks: A Simulation Framework and Reinforcement-Learning Based Study. In Proceedings of the ACM Web Conference 2023 (Austin, TX, USA) (WWW ’23). Association for Computing Machinery, New York, NY, USA, 3592–3602. https://doi.org/10.1145/3543507.3583523
  53. Christopher J. C. H. Watkins and Peter Dayan. 1992. Q-learning. Machine Learning 8, 3 (1992), 279–292. https://doi.org/10.1007/BF00992698
  54. SATS: A Universal Spectrum Auction Test Suite. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems. São Paulo, Brazil, 51–59.
  55. Michael P. Wellman. 2006. Methods for empirical game-theoretic analysis. In AAAI, Vol. 980. 1552–1556.
  56. The AI Economist: Taxation policy design via two-level deep multiagent reinforcement learning. Science advances 8, 18 (2022), eabk2607.
  57. Regret Minimization in Games with Incomplete Information. In Advances in Neural Information Processing Systems, J. Platt, D. Koller, Y. Singer, and S. Roweis (Eds.), Vol. 20. Curran Associates, Inc.

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets