Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Comprehensive View of Personalized Federated Learning on Heterogeneous Clinical Datasets (2309.16825v3)

Published 28 Sep 2023 in cs.LG

Abstract: Federated learning (FL) is increasingly being recognized as a key approach to overcoming the data silos that so frequently obstruct the training and deployment of machine-learning models in clinical settings. This work contributes to a growing body of FL research specifically focused on clinical applications along three important directions. First, we expand the FLamby benchmark (du Terrail et al., 2022a) to include a comprehensive evaluation of personalized FL methods and demonstrate substantive performance improvements over the original results. Next, we advocate for a comprehensive checkpointing and evaluation framework for FL to reflect practical settings and provide multiple comparison baselines. To this end, an open-source library aimed at making FL experimentation simpler and more reproducible is released. Finally, we propose an important ablation of PerFCL (Zhang et al., 2022). This ablation results in a natural extension of FENDA (Kim et al., 2016) to the FL setting. Experiments conducted on the FLamby benchmark and GEMINI datasets (Verma et al., 2017) show that the proposed approach is robust to heterogeneous clinical data and often outperforms existing global and personalized FL techniques, including PerFCL.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Siloed Federated Learning for Multi-centric Histopathology Datasets. In Shadi Albarqouni, Spyridon Bakas, Konstantinos Kamnitsas, M. Jorge Cardoso, Bennett Landman, Wenqi Li, Fausto Milletari, Nicola Rieke, Holger Roth, Daguang Xu, and Ziyue Xu, editors, Domain Adaptation and Representation Transfer, and Distributed and Collaborative Learning, pages 129–139, Cham, 2020a. Springer International Publishing. ISBN 978-3-030-60548-3.
  2. Federated survival analysis with discrete-time cox models, 2020b.
  3. Federated learning with personalization layers, 2019.
  4. Federated learning for site aware chest radiograph screening. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pages 1077–1081, 2021. 10.1109/ISBI48211.2021.9433876.
  5. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. In Sebastien Ourselin, Leo Joskowicz, Mert R. Sabuncu, Gozde Unal, and William Wells, editors, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016, pages 424–432, Cham, 2016. Springer International Publishing. ISBN 978-3-319-46723-8.
  6. Federated learning for predicting clinical outcomes in patients with COVID-19. Nature Medicine, 27(10):1735–1743, October 2021. ISSN 1546-170X. 10.1038/s41591-021-01506-3. URL https://doi.org/10.1038/s41591-021-01506-3.
  7. Adaptive personalized federated learning, 2020.
  8. L. R. Dice. Measures of the amount of ecologic association between species. Ecology, 26(3):297–302, 1945. ISSN 00129658, 19399170. URL http://www.jstor.org/stable/1932409.
  9. FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings. In NeurIPS 2022 - Thirty-sixth Conference on Neural Information Processing Systems, Proceedings of NeurIPS, New Orleans, United States, November 2022a. URL https://hal.science/hal-03900026.
  10. Collaborative federated learning behind hospitals’ firewalls for predicting histological complete response to neoadjuvant chemotherapy in triple-negative breast cancer. Journal of Clinical Oncology, 40(16_suppl):590–590, 2022b. 10.1200/JCO.2022.40.16_suppl.590. URL https://doi.org/10.1200/JCO.2022.40.16_suppl.590.
  11. Learnings from federated learning in the real world. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8767–8771, 2022. 10.1109/ICASSP43922.2022.9747113.
  12. Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS’20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
  13. FedDropoutAvg: Generalizable federated learning for histopathology image classification, 2021.
  14. Federated learning for mobile keyboard prediction, 2018.
  15. Deep learning–assisted prostate cancer detection on bi-parametric MRI: minimum training data size requirements and effect of prior knowledge. European Radiology, 32(4):2224–2234, April 2022. ISSN 1432-1084. 10.1007/s00330-021-08320-y. URL https://doi.org/10.1007/s00330-021-08320-y.
  16. A chart-based method for identification of delirium: validation compared with interviewer ratings using the confusion assessment method. J Am Geriatr Soc., 53(2):312–318, February 2005.
  17. S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 448–456, Lille, France, 07–09 Jul 2015. PMLR. URL https://proceedings.mlr.press/v37/ioffe15.html.
  18. Sparse random networks for communication-efficient federated learning. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=k1FHgri5y3-.
  19. W. Jeong and S. J. Hwang. Factorized-FL: Agnostic personalized federated learning with kernel factorization & similarity matching, 2022.
  20. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1–2):1–210, 2021. ISSN 1935-8237. 10.1561/2200000083. URL http://dx.doi.org/10.1561/2200000083.
  21. SCAFFOLD: Stochastic controlled averaging for federated learning. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 5132–5143. PMLR, 13–18 Jul 2020. URL https://proceedings.mlr.press/v119/karimireddy20a.html.
  22. Frustratingly easy neural domain adaptation. In COLING. ACL - Association for Computational Linguistics,, December 2016. URL https://www.microsoft.com/en-us/research/publication/frustratingly-easy-neural-domain-adaptation/.
  23. Model-contrastive federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021a.
  24. Federated optimization in heterogeneous networks. In I. Dhillon, D. Papailiopoulos, and V. Sze, editors, Proceedings of Machine Learning and Systems, volume 2, pages 429–450, 2020. URL https://proceedings.mlsys.org/paper/2020/file/38af86134b65d0f10fe33d30dd76442e-Paper.pdf.
  25. FedBN: Federated learning on non-IID features via local batch normalization, 2021b.
  26. Federated domain generalization: A survey, 2023.
  27. Think locally, act globally: Federated learning with local and global representations, 2020.
  28. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2):318–327, 2020. 10.1109/TPAMI.2018.2858826.
  29. I. Loshchilov and F. Hutter. Fixing weight decay regularization in ADAM, 2018.
  30. Communication-efficient learning of deep networks from decentralized data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 2017.
  31. Health insurance portability accountability act (HIPAA) regulations: effect on medical record research. Ann. Surg., 239(6):772–776, Jun 2004. 10.1097/01.sla.0000128307.98274.dc.
  32. Training mixed-domain translation models via federated learning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2576–2586, Seattle, United States, July 2022. Association for Computational Linguistics. 10.18653/v1/2022.naacl-main.186. URL https://aclanthology.org/2022.naacl-main.186.
  33. Federated adversarial domain adaptation. arXiv preprint arXiv:1911.02054, 2019.
  34. Private federated learning with domain adaptation. In 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Proceedings of NeurIPS, Vancouver, Canada, 2019.
  35. Dataset shift in machine learning. Mit Press, 2008.
  36. Adaptive federated optimization. In ICLR 2021, 2021.
  37. FedMM: A communication efficient solver for federated adversarial domain adaptation. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, AAMAS ’23, page 1808–1816, Richland, SC, 2023. International Foundation for Autonomous Agents and Multiagent Systems. ISBN 9781450394321.
  38. Privacy-preserving unsupervised domain adaptation in federated setting. IEEE Access, 8:143233–143240, 2020. 10.1109/ACCESS.2020.3014264.
  39. Revisiting unreasonable effectiveness of data in deep learning era. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 843–852, 2017. 10.1109/ICCV.2017.97.
  40. Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning Systems, pages 1–17, 2022. 10.1109/TNNLS.2022.3160699.
  41. M. Tan and Q. V. Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning (ICML), Long Beach, June 2019.
  42. Patient characteristics, resource use and outcomes associated with general internal medicine hospital care: the general medicine inpatient initiative (GEMINI) retrospective cohort study. Canadian Medical Association Open Access Journal, 5(4):E842–E849, 2017.
  43. FedMed-GAN: Federated domain translation on unsupervised cross-modality brain image synthesis. Neurocomputing, 546:126282, 2023.
  44. Federated evaluation of on-device personalization, 2019.
  45. Federated multi-target domain adaptation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1424–1433, 2022.
  46. Salvaging federated learning by local adaptation, 2020.
  47. Dive into deep learning. arXiv preprint arXiv:2106.11342, 2021.
  48. Federated domain generalization with generalization adjustment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3954–3963, 2023.
  49. Personalized federated contrastive learning. In 2022 IEEE International Conference on Big Data (Big Data), pages 4218–4225, 2022. 10.1109/BigData55660.2022.10020518.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now