2000 character limit reached
A review on different techniques used to combat the non-IID and heterogeneous nature of data in FL (2401.00809v1)
Published 1 Jan 2024 in cs.LG
Abstract: Federated Learning (FL) is a machine-learning approach enabling collaborative model training across multiple decentralized edge devices that hold local data samples, all without exchanging these samples. This collaborative process occurs under the supervision of a central server orchestrating the training or via a peer-to-peer network. The significance of FL is particularly pronounced in industries such as healthcare and finance, where data privacy holds paramount importance. However, training a model under the Federated learning setting brings forth several challenges, with one of the most prominent being the heterogeneity of data distribution among the edge devices. The data is typically non-independently and non-identically distributed (non-IID), thereby presenting challenges to model convergence. This report delves into the issues arising from non-IID and heterogeneous data and explores current algorithms designed to address these challenges.
- “A Robust Aggregation Approach for Heterogeneous Federated Learning | IEEE Conference Publication | IEEE Xplore” ieeexplore.ieee.org URL: https://ieeexplore.ieee.org/abstract/document/10201227
- “Decentralized Federated Learning via Mutual Knowledge Transfer | IEEE Journals and Magazine | IEEE Xplore” ieeexplore.ieee.org URL: https://ieeexplore.ieee.org/abstract/document/9426904
- “Deep Federated Learning for Autonomous Driving | IEEE Conference Publication | IEEE Xplore” ieeexplore.ieee.org URL: https://ieeexplore.ieee.org/abstract/document/9827020
- “Distributed Dynamic Map Fusion via Federated Learning for Intelligent Networked Vehicles | IEEE Conference Publication | IEEE Xplore” ieeexplore.ieee.org URL: https://ieeexplore.ieee.org/abstract/document/9561612
- “End-to-End Federated Learning for Autonomous Driving Vehicles | IEEE Conference Publication | IEEE Xplore” ieeexplore.ieee.org URL: https://ieeexplore.ieee.org/abstract/document/9533808
- “Federated Imitation Learning: A Novel Framework for Cloud Robotic Systems With Heterogeneous Sensor Data | IEEE Journals and Magazine | IEEE Xplore” ieeexplore.ieee.org URL: https://ieeexplore.ieee.org/abstract/document/9013081
- “Federated Learning for Object Detection in Autonomous Vehicles | IEEE Conference Publication | IEEE Xplore” ieeexplore.ieee.org URL: https://ieeexplore.ieee.org/abstract/document/9564384
- “Federated Learning on Non-IID Data Silos: An Experimental Study | IEEE Conference Publication | IEEE Xplore” ieeexplore.ieee.org URL: https://ieeexplore.ieee.org/abstract/document/9835537
- “Flow-FL: Data-Driven Federated Learning for Spatio-Temporal Predictions in Multi-Robot Systems | IEEE Conference Publication | IEEE Xplore” ieeexplore.ieee.org URL: https://ieeexplore.ieee.org/abstract/document/9560791/
- Tzu-Ming Harry Hsu, Hang Qi and Matthew Brown “Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification” In arXiv:1909.06335 [cs, stat], 2019 URL: https://arxiv.org/abs/1909.06335
- “Federated Optimization: Distributed Machine Learning for On-Device Intelligence” In arXiv:1610.02527 [cs], 2016 URL: https://arxiv.org/abs/1610.02527
- “Peer-to-peer Federated Learning on Graphs” In arXiv:1901.11173 [cs, stat], 2019 URL: https://arxiv.org/abs/1901.11173
- “A review of applications in federated learning” In Computers and Industrial Engineering 149, 2020, pp. 106854 DOI: 10.1016/j.cie.2020.106854
- Boyi Liu, Lujia Wang and Ming Liu “Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems” In IEEE Robotics and Automation Letters 4, 2019, pp. 4555–4562 DOI: 10.1109/lra.2019.2931179
- Priyanka Mary Mammen “Federated Learning: Opportunities and Challenges” In arXiv:2101.05428 [cs], 2021 URL: https://arxiv.org/abs/2101.05428
- “Communication-Efficient Learning of Deep Networks from Decentralized Data” PMLR, 2017, pp. 1273–1282 proceedings.mlr.press URL: https://proceedings.mlr.press/v54/mcmahan17a?ref=https://githubhelp.com
- “Real-Time Data Processing Architecture for Multi-Robots Based on Differential Federated Learning | IEEE Conference Publication | IEEE Xplore” ieeexplore.ieee.org URL: https://ieeexplore.ieee.org/abstract/document/8560084/
- “Ensemble Distillation for Robust Model Fusion in Federated Learning” In Advances in Neural Information Processing Systems 33, 2020 URL: https://proceedings.neurips.cc/paper/2020/hash/18df51b97ccd68128e994804f3eccc87-Abstract.html
- “Federated Learning in Robotic and Autonomous Systems” In Procedia Computer Science 191, 2021, pp. 135–142 DOI: 10.1016/j.procs.2021.07.041
- “A survey on federated learning” In Knowledge-Based Systems 216, 2021, pp. 106775 DOI: 10.1016/j.knosys.2021.106775
- “Federated Learning with Non-IID Data”, 2018 arXiv.org URL: https://arxiv.org/abs/1806.00582
- Venkataraman Natarajan Iyer (1 paper)