Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Federated Best Arm Identification with Heterogeneous Clients (2210.07780v3)

Published 14 Oct 2022 in cs.LG, math.ST, and stat.TH

Abstract: We study best arm identification in a federated multi-armed bandit setting with a central server and multiple clients, when each client has access to a {\em subset} of arms and each arm yields independent Gaussian observations. The goal is to identify the best arm of each client subject to an upper bound on the error probability; here, the best arm is one that has the largest {\em average} value of the means averaged across all clients having access to the arm. Our interest is in the asymptotics as the error probability vanishes. We provide an asymptotic lower bound on the growth rate of the expected stopping time of any algorithm. Furthermore, we show that for any algorithm whose upper bound on the expected stopping time matches with the lower bound up to a multiplicative constant ({\em almost-optimal} algorithm), the ratio of any two consecutive communication time instants must be {\em bounded}, a result that is of independent interest. We thereby infer that an algorithm can communicate no more sparsely than at exponential time instants in order to be almost-optimal. For the class of almost-optimal algorithms, we present the first-of-its-kind asymptotic lower bound on the expected number of {\em communication rounds} until stoppage. We propose a novel algorithm that communicates at exponential time instants, and demonstrate that it is asymptotically almost-optimal.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (29)
  1. E. Even-Dar, S. Mannor, Y. Mansour, and S. Mahadevan, “Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems.” Journal of Machine Learning Research, vol. 7, no. 6, 2006.
  2. K. Jamieson, M. Malloy, R. Nowak, and S. Bubeck, “lil’ucb: An optimal exploration algorithm for multi-armed bandits,” in Conference on Learning Theory.   PMLR, 2014, pp. 423–439.
  3. J.-Y. Audibert, S. Bubeck, and R. Munos, “Best arm identification in multi-armed bandits.” in Conference on Learning Theory, 2010, pp. 41–53.
  4. S. Bubeck, R. Munos, and G. Stoltz, “Pure exploration in finitely-armed and continuous-armed bandits,” Theoretical Computer Science, vol. 412, no. 19, pp. 1832–1852, 2011.
  5. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in 20th International Conference on Artificial Intelligence and Statistics.   PMLR, 2017, pp. 1273–1282.
  6. P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings, R. G. L. D’Oliveira, H. Eichner, S. E. Rouayheb, D. Evans, J. Gardner, Z. Garrett, A. Gascón, B. Ghazi, P. B. Gibbons, M. Gruteser, Z. Harchaoui, C. He, L. He, Z. Huo, B. Hutchinson, J. Hsu, M. Jaggi, T. Javidi, G. Joshi, M. Khodak, J. Konecný, A. Korolova, F. Koushanfar, S. Koyejo, T. Lepoint, Y. Liu, P. Mittal, M. Mohri, R. Nock, A. Özgür, R. Pagh, H. Qi, D. Ramage, R. Raskar, M. Raykova, D. Song, W. Song, S. U. Stich, Z. Sun, A. T. Suresh, F. Tramèr, P. Vepakomma, J. Wang, L. Xiong, Z. Xu, Q. Yang, F. X. Yu, H. Yu, and S. Zhao, “Advances and open problems in federated learning,” Foundations and Trends® in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021. [Online]. Available: http://dx.doi.org/10.1561/2200000083
  7. Z. Zhu, J. Zhu, J. Liu, and Y. Liu, “Federated bandit: A gossiping approach,” in Abstract Proceedings of the 2021 ACM SIGMETRICS/International Conference on Measurement and Modeling of Computer Systems, 2021, pp. 3–4.
  8. C. Shi and C. Shen, “Federated multi-armed bandits,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, 2021, pp. 9603–9611.
  9. Z. Yan, Q. Xiao, T. Chen, and A. Tajer, “Federated multi-armed bandit via uncoordinated exploration,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).   IEEE, 2022, pp. 5248–5252.
  10. J. Wang and N. B. Shah, “Your 2 is my 1, your 3 is my 9: Handling arbitrary miscalibrations in ratings,” in Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, 2019, pp. 864–872.
  11. A. Garivier and E. Kaufmann, “Optimal best arm identification with fixed confidence,” in Conference on Learning Theory,.   PMLR, 2016, pp. 998–1027.
  12. V. Moulos, “Optimal best Markovian arm identification with fixed confidence,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  13. E. Kaufmann, O. Cappé, and A. Garivier, “On the complexity of best-arm identification in multi-armed bandit models,” Journal of Machine Learning Research, vol. 17, no. 1, pp. 1–42, 2016.
  14. P. N. Karthik, K. S. Reddy, and V. Y. F. Tan, “Best arm identification in restless Markov multi-armed bandits,” arXiv preprint arXiv:2203.15236, 2022.
  15. J. Yang, Z. Zhong, and V. Y. F. Tan, “Optimal clustering with bandit feedback,” arXiv preprint arXiv:2202.04294, 2022.
  16. C. Shi, C. Shen, and J. Yang, “Federated multi-armed bandits with personalization,” in International Conference on Artificial Intelligence and Statistics.   PMLR, 2021, pp. 2917–2925.
  17. A. Dubey and A. Pentland, “Differentially-private federated linear bandits,” Advances in Neural Information Processing Systems, vol. 33, pp. 6003–6014, 2020.
  18. A. Mitra, H. Hassani, and G. Pappas, “Exploiting heterogeneity in robust federated best-arm identification,” arXiv preprint arXiv:2109.05700, 2021.
  19. K. S. Reddy, P. N. Karthik, and V. Y. F. Tan, “Almost cost-free communication in federated best arm identification,” in Proceedings of 37th AAAI Conference on Artificial Intelligence, 2023.
  20. C. Tao, Q. Zhang, and Y. Zhou, “Collaborative learning with limited interaction: Tight bounds for distributed exploration in multi-armed bandits,” in 2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS).   IEEE, 2019, pp. 126–146.
  21. N. Karpov, Q. Zhang, and Y. Zhou, “Collaborative top distribution identifications with limited interaction,” in 2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS).   IEEE, 2020, pp. 160–171.
  22. E. Hillel, Z. S. Karnin, T. Koren, R. Lempel, and O. Somekh, “Distributed exploration in multi-armed bandits,” Advances in Neural Information Processing Systems, vol. 26, 2013.
  23. N. Karpov and Q. Zhang, “Collaborative best arm identification with limited communication on non-iid data,” arXiv preprint arXiv:2207.08015, 2022.
  24. O. A. Hanna, L. Yang, and C. Fragouli, “Solving multi-arm bandit using a few bits of communication,” in International Conference on Artificial Intelligence and Statistics.   PMLR, 2022, pp. 11 215–11 236.
  25. A. Mitra, H. Hassani, and G. J. Pappas, “Linear stochastic bandits over a bit-constrained channel,” in Learning for Dynamics and Control Conference.   PMLR, 2023, pp. 1387–1399.
  26. F. Pase, D. Gündüz, and M. Zorzi, “Rate-constrained remote contextual bandits,” IEEE Journal on Selected Areas in Information Theory, 2022.
  27. Y. Wang, J. Hu, X. Chen, and L. Wang, “Distributed bandit learning: Near-optimal regret with efficient communication,” arXiv preprint arXiv:1904.06309, 2019.
  28. P. Mayekar, J. Scarlett, and V. Y. F. Tan, “Communication-constrained bandits under additive Gaussian noise,” in International Conference on Machine Learning.   PMLR, 2023, pp. 24 236–24 250.
  29. I. Cantador, P. Brusilovsky, and T. Kuflik, “Second workshop on information heterogeneity and fusion in recommender systems (hetrec2011),” in Proceedings of the Fifth ACM conference on Recommender systems, 2011, pp. 387–388.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now