Robust and Performance Incentivizing Algorithms for Multi-Armed Bandits with Strategic Agents
Abstract: Motivated by applications such as online labor markets we consider a variant of the stochastic multi-armed bandit problem where we have a collection of arms representing strategic agents with different performance characteristics. The platform (principal) chooses an agent in each round to complete a task. Unlike the standard setting, when an arm is pulled it can modify its reward by absorbing it or improving it at the expense of a higher cost. The principle has to solve a mechanism design problem to incentivize the arms to give their best performance. However, since even with an effective mechanism agents may still deviate from rational behavior, the principal wants a robust algorithm that also gives a non-vacuous guarantee on the total accumulated rewards under non-equilibrium behavior. In this paper, we introduce a class of bandit algorithms that meet the two objectives of performance incentivization and robustness simultaneously. We do this by identifying a collection of intuitive properties that a bandit algorithm has to satisfy to achieve these objectives. Finally, we show that settings where the principal has no information about the arms' performance characteristics can be handled by combining ideas from second price auctions with our algorithms.
- Fiduciary bandits. In International Conference on Machine Learning, pages 518–527. PMLR, 2020.
- Multi-armed bandit problems with strategic arms. In A. Beygelzimer and D. Hsu, editors, Proceedings of the Thirty-Second Conference on Learning Theory, volume 99 of Proceedings of Machine Learning Research, pages 383–416. PMLR, 25–28 Jun 2019. URL https://proceedings.mlr.press/v99/braverman19b.html.
- G. Carroll. Robustness and linear contracts. American Economic Review, 105(2):536–63, 2015.
- Designing menus of contracts efficiently: The power of randomization. arXiv preprint arXiv:2202.10966, 2022.
- M. Chandrasekher. Unraveling in a repeated moral hazard model with multiple agents. Theoretical Economics, 10(1):11–49, 2015.
- Simple versus optimal contracts. In Proceedings of the 2019 ACM Conference on Economics and Computation, pages 369–387, 2019.
- The intrinsic robustness of stochastic bandits to strategic manipulation. In International Conference on Machine Learning, pages 3092–3101. PMLR, 2020.
- A. Ghosh and P. Hummel. A game-theoretic analysis of rank-order mechanisms for user-generated content. In Proceedings of the 12th ACM conference on Electronic commerce, pages 189–198, 2011.
- A. Ghosh and P. Hummel. Learning and incentives in user-generated content: Multi-armed bandits with endogenous arms. In Proceedings of the 4th conference on Innovations in Theoretical Computer Science, pages 233–246, 2013.
- A. Ghosh and P. McAfee. Incentivizing high-quality user-generated content. In Proceedings of the 20th international conference on World wide web, pages 137–146, 2011.
- B. Holmström. Moral hazard and observability. The Bell journal of economics, pages 74–91, 1979.
- Designing incentives for online question and answer forums. In Proceedings of the 10th ACM conference on Electronic commerce, pages 129–138, 2009.
- Implementing the “wisdom of the crowd”. Journal of Political Economy, 122(5):988–1012, 2014.
- Bayesian incentive-compatible bandit exploration. In Proceedings of the Sixteenth ACM Conference on Economics and Computation, pages 565–582, 2015.
- M. Mitzenmacher and E. Upfal. Probability and computing: Randomization and probabilistic techniques in algorithms and data analysis. Cambridge university press, 2017.
- W. P. Rogerson. Repeated moral hazard. Econometrica: Journal of the Econometric Society, pages 69–76, 1985.
- Multi-armed bandit algorithm against strategic replication. In International Conference on Artificial Intelligence and Statistics, pages 403–431. PMLR, 2022.
- S. Wang and L. Huang. Multi-armed bandits with compensation. Advances in Neural Information Processing Systems, 31, 2018.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.