Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Federated Linear Contextual Bandits with Heterogeneous Clients (2403.00116v1)

Published 29 Feb 2024 in cs.LG and cs.AI

Abstract: The demand for collaborative and private bandit learning across multiple agents is surging due to the growing quantity of data generated from distributed systems. Federated bandit learning has emerged as a promising framework for private, efficient, and decentralized online learning. However, almost all previous works rely on strong assumptions of client homogeneity, i.e., all participating clients shall share the same bandit model; otherwise, they all would suffer linear regret. This greatly restricts the application of federated bandit learning in practice. In this work, we introduce a new approach for federated bandits for heterogeneous clients, which clusters clients for collaborative bandit learning under the federated learning setting. Our proposed algorithm achieves non-trivial sub-linear regret and communication cost for all clients, subject to the communication protocol under federated learning that at anytime only one model can be shared by the server.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Improved algorithms for linear stochastic bandits. In Advances in Neural Information Processing Systems, pages 2312–2320, 2011.
  2. Gambling in a rigged casino: The adversarial multi-armed bandit problem. In Proceedings of IEEE 36th Annual Foundations of Computer Science, pages 322–331, 1995. doi: 10.1109/SFCS.1995.492488.
  3. Finite-time analysis of the multiarmed bandit problem. Machine learning, 47(2-3):235–256, 2002.
  4. What doubling tricks can and can’t do for multi-armed bandits, 2018.
  5. Towards federated learning at scale: System design. Proceedings of machine learning and systems, 1:374–388, 2019.
  6. Multi-armed bandits in the presence of side observations in social networks. In 52nd IEEE Conference on Decision and Control, pages 7309–7314. IEEE, 2013.
  7. 2nd workshop on information heterogeneity and fusion in recommender systems (hetrec 2011). In Proceedings of the 5th ACM conference on Recommender systems, RecSys 2011, New York, NY, USA, 2011. ACM.
  8. Interpretation and use of generalized chow tests. International Economic Review, pages 725–741, 1991.
  9. Leveraging side observations in stochastic bandits. CoRR, abs/1210.4839, 2012. URL http://arxiv.org/abs/1210.4839.
  10. A gang of bandits. In Advances in Neural Information Processing Systems, pages 737–745, 2013.
  11. An empirical evaluation of thompson sampling. Advances in neural information processing systems, 24, 2011.
  12. Gregory C Chow. Tests of equality between sets of coefficients in two linear regressions. Econometrica: Journal of the Econometric Society, pages 591–605, 1960.
  13. Differentially-private federated linear bandits. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 6003–6014. Curran Associates, Inc., 2020a. URL https://proceedings.neurips.cc/paper_files/paper/2020/file/4311359ed4969e8401880e3c1836fbe1-Paper.pdf.
  14. Differentially-private federated linear bandits. Advances in Neural Information Processing Systems, 33:6003–6014, 2020b.
  15. Contextual bandits for adapting treatment in a mouse model of de novo carcinogenesis. In Machine learning for healthcare conference, pages 67–82. PMLR, 2018.
  16. Openfl: the open federated learning library. Physics in Medicine & Biology, 2022. doi: 10.1088/1361-6560/ac97d9. URL http://iopscience.iop.org/article/10.1088/1361-6560/ac97d9.
  17. Online clustering of bandits. In International Conference on Machine Learning, pages 757–765, 2014.
  18. On context-dependent clustering of bandits. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1253–1262. JMLR. org, 2017.
  19. Fedml: A research library and benchmark for federated machine learning. Advances in Neural Information Processing Systems, Best Paper Award at Federate Learning Workshop, 2020.
  20. A simple and provably efficient algorithm for asynchronous federated contextual linear bandits. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=Fx7oXUVEPW.
  21. Hierarchical bayesian bandits. CoRR, abs/2111.06929, 2021. URL https://arxiv.org/abs/2111.06929.
  22. Fair algorithms for multi-agent multi-armed bandits. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021. URL https://openreview.net/forum?id=AlD5WD2ANIQ.
  23. Federated linear contextual bandits, 2021. URL https://arxiv.org/abs/2110.14177.
  24. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1–2):1–210, 2021.
  25. Distributed clustering of linear bandits in peer to peer networks. In Maria Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1301–1309, New York, New York, USA, 20–22 Jun 2016. PMLR. URL https://proceedings.mlr.press/v48/korda16.html.
  26. Asynchronous upper confidence bound algorithms for federated linear bandits. In Gustau Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera, editors, Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pages 6529–6553. PMLR, 28–30 Mar 2022. URL https://proceedings.mlr.press/v151/li22e.html.
  27. Unifying clustered and non-stationary bandits. In International Conference on Artificial Intelligence and Statistics, pages 1063–1071. PMLR, 2021.
  28. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th International Conference on World Wide Web, WWW ’10, page 661–670, New York, NY, USA, 2010a. Association for Computing Machinery. ISBN 9781605587998. doi: 10.1145/1772690.1772758. URL https://doi.org/10.1145/1772690.1772758.
  29. Collaborative filtering bandits. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pages 539–548. ACM, 2016.
  30. Improved algorithm on online clustering of bandits. arXiv preprint arXiv:1902.09162, 2019.
  31. Exploitation and exploration in a performance based contextual advertising system. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 27–36, 2010b.
  32. Federated online clustering of bandits, 2022. URL https://arxiv.org/abs/2208.14865.
  33. Fast distributed bandits for online recommendation systems. In Proceedings of the 34th ACM international conference on supercomputing, pages 1–13, 2020.
  34. From bandits to experts: On the value of side-observations. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011. URL https://proceedings.neurips.cc/paper_files/paper/2011/file/e1e32e235eee1f970470a3a6658dfdd5-Paper.pdf.
  35. Federated learning of deep networks using model averaging. CoRR, abs/1602.05629, 2016. URL http://arxiv.org/abs/1602.05629.
  36. Federated multi-armed bandits. CoRR, abs/2101.12204, 2021. URL https://arxiv.org/abs/2101.12204.
  37. Distributed bandit learning: Near-optimal regret with efficient communication. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https://openreview.net/forum?id=SJxZnR4YvB.
  38. Contextual bandits in a collaborative environment. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pages 529–538. ACM, 2016.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now