Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Near Optimal Adversarial Attacks on Stochastic Bandits and Defenses with Smoothed Responses (2008.09312v8)

Published 21 Aug 2020 in cs.LG and stat.ML

Abstract: I study adversarial attacks against stochastic bandit algorithms. At each round, the learner chooses an arm, and a stochastic reward is generated. The adversary strategically adds corruption to the reward, and the learner is only able to observe the corrupted reward at each round. Two sets of results are presented in this paper. The first set studies the optimal attack strategies for the adversary. The adversary has a target arm he wishes to promote, and his goal is to manipulate the learner into choosing this target arm $T - o(T)$ times. I design attack strategies against UCB and Thompson Sampling that only spend $\widehat{O}(\sqrt{\log T})$ cost. Matching lower bounds are presented, and the vulnerability of UCB, Thompson sampling, and $\varepsilon$-greedy are exactly characterized. The second set studies how the learner can defend against the adversary. Inspired by literature on smoothed analysis and behavioral economics, I present two simple algorithms that achieve a competitive ratio arbitrarily close to 1.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. Handbook of mathematical functions with formulas, graphs, and mathematical tables.
  2. Taming the monster: A fast and simple algorithm for contextual bandits. In International Conference on Machine Learning, pages 1638–1646. PMLR.
  3. Analysis of thompson sampling for the multi-armed bandit problem. In Conference on learning theory, pages 39–1. JMLR Workshop and Conference Proceedings.
  4. Near-optimal regret bounds for thompson sampling. Journal of the ACM (JACM), 64(5):1–24.
  5. The multiplicative weights update method: a meta-algorithm and applications. Theory of computing, 8(1):121–164.
  6. Finite-time analysis of the multiarmed bandit problem. Machine learning, 47(2):235–256.
  7. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning, 5(1):1–122.
  8. Camerer, C. F. (2011). Behavioral game theory: Experiments in strategic interaction. Princeton university press.
  9. Simple and scalable response prediction for display advertising. ACM Transactions on Intelligent Systems and Technology (TIST), 5(4):1–34.
  10. On the pairing of the softmax activation and cross-entropy penalty functions and the derivation of the softmax activation function. In Proc. 8th Aust. Conf. on the Neural Networks, Melbourne, volume 181, page 185. Citeseer.
  11. Adversarial attacks on linear contextual bandits. Advances in Neural Information Processing Systems, 33:14362–14373.
  12. Better algorithms for stochastic bandits with adversarial corruptions. arXiv preprint arXiv:1902.08647.
  13. Smoothed analysis of online and differentially private learning. Advances in Neural Information Processing Systems, 33:9203–9215.
  14. Adversarial attacks on gaussian process bandits. In International Conference on Machine Learning, pages 8304–8329. PMLR.
  15. Nearly optimal algorithms for linear contextual bandits with adversarial corruptions. Advances in Neural Information Processing Systems, 35:34614–34625.
  16. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144.
  17. Adversarial attacks on stochastic bandits. In Advances in Neural Information Processing Systems, pages 3640–3649.
  18. A smoothed analysis of the greedy algorithm for the linear contextual bandit problem. Advances in neural information processing systems, 31.
  19. Algorithms for multi-armed bandit problems. arXiv preprint arXiv:1402.6028.
  20. Corruption-robust contextual search through density updates. In Conference on Learning Theory, pages 3504–3505. PMLR.
  21. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, pages 661–670.
  22. Data poisoning attacks on stochastic bandits. In International Conference on Machine Learning, pages 4042–4050. PMLR.
  23. Luce, R. D. (2012). Individual choice behavior: A theoretical analysis. Courier Corporation.
  24. Stochastic bandits robust to adversarial corruptions. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, pages 114–122.
  25. Data poisoning attacks in contextual bandits. In Decision and Game Theory for Security: 9th International Conference, GameSec 2018, Seattle, WA, USA, October 29–31, 2018, Proceedings 9, pages 186–204. Springer.
  26. Adversarial attacks on adversarial bandits. arXiv preprint arXiv:2301.12595.
  27. McFadden, D. L. (1976). Quantal choice analaysis: A survey. Annals of Economic and Social Measurement, Volume 5, number 4, pages 363–390.
  28. Quantal response equilibria for normal form games. Games and economic behavior, 10(1):6–38.
  29. Saving stochastic bandits from poisoning attacks via limited data verification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8054–8061.
  30. Structured linear contextual bandits: A sharp and geometric smoothed analysis. In International Conference on Machine Learning, pages 9026–9035. PMLR.
  31. Smoothed analysis of algorithms: Why the simplex algorithm usually takes polynomial time. Journal of the ACM (JACM), 51(3):385–463.
  32. A model selection approach for corruption robust reinforcement learning. In International Conference on Algorithmic Learning Theory, pages 1043–1096. PMLR.
  33. Inverse game theory for stackelberg games: the blessing of bounded rationality. Advances in Neural Information Processing Systems, 35:32186–32198.
  34. Adaptive reward-poisoning attacks against reinforcement learning. In International Conference on Machine Learning, pages 11225–11234. PMLR.
  35. Tsallis-inf: An optimal algorithm for stochastic and adversarial bandits. The Journal of Machine Learning Research, 22(1):1310–1358.
  36. Zuo, S. (2023). Corruption-robust lipschitz contextual search. arXiv preprint arXiv:2307.13903.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (1)
  1. Shiliang Zuo (9 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now