Personalized Federated Learning via Stacking (2404.10957v2)
Abstract: Traditional Federated Learning (FL) methods typically train a single global model collaboratively without exchanging raw data. In contrast, Personalized Federated Learning (PFL) techniques aim to create multiple models that are better tailored to individual clients' data. We present a novel personalization approach based on stacked generalization where clients directly send each other privacy-preserving models to be used as base models to train a meta-model on private data. Our approach is flexible, accommodating various privacy-preserving techniques and model types, and can be applied in horizontal, hybrid, and vertically partitioned federations. Additionally, it offers a natural mechanism for assessing each client's contribution to the federation. Through comprehensive evaluations across diverse simulated data heterogeneity scenarios, we showcase the effectiveness of our method.
- Communication-efficient learning of deep networks from decentralized data, 2023.
- WrapperFL: A Model Agnostic Plug-in for Industrial Federated Learning, August 2022. arXiv:2206.10407 [cs].
- A comprehensive survey of privacy-preserving federated learning: A taxonomy, review, and future directions. ACM Comput. Surv., 54(6), jul 2021.
- David H. Wolpert. Stacked generalization. Neural Networks, 5(2):241–259, 1992.
- Cynthia Dwork. Differential Privacy: A Survey of Results. In Manindra Agrawal, Dingzhu Du, Zhenhua Duan, and Angsheng Li, editors, Theory and Applications of Models of Computation, volume 4978, pages 1–19. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008.
- A Survey on Homomorphic Encryption Schemes: Theory and Implementation, October 2017. arXiv:1704.03578 [cs].
- Contribution evaluation in federated learning: Examining current approaches, 2023.
- Personalization in Federated Learning. In Heiko Ludwig and Nathalie Baracaldo, editors, Federated Learning, pages 71–98. Springer International Publishing, Cham, 2022.
- Federated Mutual Learning, September 2020. arXiv:2006.16765 [cs].
- FedMD: Heterogenous Federated Learning via Model Distillation, October 2019. arXiv:1910.03581 [cs, stat].
- Federated Meta-Learning with Fast Convergence and Efficient Communication, December 2019. arXiv:1802.07876 [cs].
- Adaptive Gradient-Based Meta-Learning Methods, December 2019. arXiv:1906.02717 [cs, stat].
- Three Approaches for Personalization with Applications to Federated Learning, July 2020. arXiv:2002.10619 [cs, stat].
- Robust Federated Learning in a Heterogeneous Environment, October 2019. arXiv:1906.06629 [cs, stat].
- An Efficient Framework for Clustered Federated Learning, June 2021. arXiv:2006.04088 [cs, stat].
- BAGAN: Data Augmentation with Balancing GAN, June 2018. arXiv:1803.09655 [cs, stat].
- Federated Learning with Non-IID Data. 2018. arXiv:1806.00582 [cs, stat].
- Privacy-Preserving Stacking with Application to Cross-organizational Diabetes Prediction. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, pages 4114–4120, Macao, China, August 2019. International Joint Conferences on Artificial Intelligence Organization.
- Differentially private empirical risk minimization. Journal of Machine Learning Research, 12(29):1069–1109, 2011.
- A Federated Learning Framework for IoT: Application to Industry 4.0. In 2022 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid), pages 565–574, Taormina, Italy, May 2022. IEEE.
- Ron Kohavi. Census Income. UCI Machine Learning Repository, 1996. DOI: https://doi.org/10.24432/C5GP7S.
- Jock Blackard. Covertype. UCI Machine Learning Repository, 1998. DOI: https://doi.org/10.24432/C50K5N.
- L&T Vehicle Loan Default Prediction. URL: https://www.kaggle.com/datasets/mamtadhaker/lt-vehicle-loan-default-prediction. Accessed 5/4/2024.
- Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
- Measuring the effects of non-identical data distribution for federated visual classification, 2019.