Papers
Topics
Authors
Recent
Search
2000 character limit reached

MetaFed: Federated Learning among Federations with Cyclic Knowledge Distillation for Personalized Healthcare

Published 17 Jun 2022 in cs.LG and cs.CY | (2206.08516v3)

Abstract: Federated learning has attracted increasing attention to building models without accessing the raw user data, especially in healthcare. In real applications, different federations can seldom work together due to possible reasons such as data heterogeneity and distrust/inexistence of the central server. In this paper, we propose a novel framework called MetaFed to facilitate trustworthy FL between different federations. MetaFed obtains a personalized model for each federation without a central server via the proposed Cyclic Knowledge Distillation. Specifically, MetaFed treats each federation as a meta distribution and aggregates knowledge of each federation in a cyclic manner. The training is split into two parts: common knowledge accumulation and personalization. Comprehensive experiments on three benchmarks demonstrate that MetaFed without a server achieves better accuracy compared to state-of-the-art methods (e.g., 10%+ accuracy improvement compared to the baseline for PAMAP2) with fewer communication costs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. W. Lu, J. Wang, and Y. Chen, “Local and global alignments for generalizable sensor-based human activity recognition,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022.
  2. W. Lu, J. Wang, Y. Chen, S. Pan, C. Hu, and X. Qin, “Semantic-discriminative mixup for generalizable sensor-based cross-domain activity recognition,” IMWUT, 2022.
  3. M. He, J. Zhang, S. Shan, X. Liu, Z. Wu, and X. Chen, “Locality-aware channel-wise dropout for occluded face recognition,” IEEE Transactions on Image Processing, vol. 31, pp. 788–798, 2021.
  4. W. Lu, Y. Chen, J. Wang, and X. Qin, “Cross-domain activity recognition via substructural optimal transport,” Neurocomputing, vol. 454, pp. 65–75, 2021.
  5. S. Li, X. Sui, X. Luo, X. Xu, Y. Liu, and R. S. M. Goh, “Medical image segmentation using squeeze-and-expansion transformers,” in IJCAI, 2021.
  6. F. Ma, M. Ye, J. Luo, C. Xiao, and J. Sun, “Advances in mining heterogeneous healthcare data,” in KDD ’21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021, F. Zhu, B. C. Ooi, and C. Miao, Eds.   ACM, 2021, pp. 4050–4051. [Online]. Available: https://doi.org/10.1145/3447548.3470789
  7. H. Aguiar, M. Santos, P. Watkinson, and T. Zhu, “Learning of cluster-based feature importance for electronic health record time-series,” in International Conference on Machine Learning.   PMLR, 2022, pp. 161–179.
  8. Z. Liu, Y. Chen, Y. Zhao, H. Yu, Y. Liu, R. Bao, J. Jiang, Z. Nie, Q. Xu, and Q. Yang, “Contribution-aware federated learning for smart healthcare,” in Proceedings of the 34th Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-22), 2022.
  9. P. Voigt and A. Von dem Bussche, “The eu general data protection regulation (gdpr),” A Practical Guide, 1st Ed., Cham: Springer International Publishing, vol. 10, p. 3152676, 2017.
  10. Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 10, no. 2, pp. 1–19, 2019.
  11. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial Intelligence and Statistics.   PMLR, 2017, pp. 1273–1282.
  12. X. Li, M. JIANG, X. Zhang, M. Kamp, and Q. Dou, “Fedbn: Federated learning on non-iid features via local batch normalization,” in International Conference on Learning Representations, 2021.
  13. G. Hinton, O. Vinyals, J. Dean et al., “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, vol. 2, no. 7, 2015.
  14. A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio, “Fitnets: Hints for thin deep nets,” arXiv preprint arXiv:1412.6550, 2014.
  15. N. Paluru, A. Dayal, H. B. Jenssen, T. Sakinis, L. R. Cenkeramaddi, J. Prakash, and P. K. Yalavarthy, “Anam-net: Anamorphic depth embedding-based lightweight cnn for segmentation of anomalies in covid-19 chest ct images,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 3, pp. 932–946, 2021.
  16. Y. Ding, F. Tan, Z. Qin, M. Cao, K.-K. R. Choo, and Z. Qin, “Deepkeygen: a deep learning-based stream cipher generator for medical image encryption and decryption,” IEEE Transactions on Neural Networks and Learning Systems, 2021.
  17. F. Yu, L. Cui, H. Chen, Y. Cao, N. Liu, W. Huang, Y. Xu, and H. Lu, “Healthnet: A health progression network via heterogeneous medical information fusion,” IEEE Transactions on Neural Networks and Learning Systems, 2022.
  18. J. Wang, Y. Chen, S. Hao, X. Peng, and L. Hu, “Deep learning for sensor-based activity recognition: A survey,” Pattern Recognition Letters, vol. 119, pp. 3–11, 2019.
  19. K. Muhammad, S. Khan, J. D. Ser, and V. H. C. d. Albuquerque, “Deep learning for multigrade brain tumor classification in smart healthcare systems: A prospective survey,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 2, pp. 507–522, 2021.
  20. A. Z. Tan, H. Yu, L. Cui, and Q. Yang, “Towards personalized federated learning,” IEEE Transactions on Neural Networks and Learning Systems, 2022.
  21. F. R. Vogenberg, C. I. Barash, and M. Pursel, “Personalized medicine: part 1: evolution and development into theranostics,” Pharmacy and Therapeutics, vol. 35, no. 10, p. 560, 2010.
  22. T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,” Proceedings of Machine Learning and Systems, vol. 2, pp. 429–450, 2020.
  23. T. Yu, E. Bagdasaryan, and V. Shmatikov, “Salvaging federated learning by local adaptation,” arXiv preprint arXiv:2002.04758, 2020.
  24. H.-Y. Chen and W.-L. Chao, “On bridging generic and personalized federated learning for image classification,” in International Conference on Learning Representations, 2022.
  25. Y. Chen, X. Qin, J. Wang, C. Yu, and W. Gao, “Fedhealth: A federated transfer learning framework for wearable healthcare,” IEEE Intelligent Systems, vol. 35, no. 4, pp. 83–93, 2020.
  26. W. Lu, J. Wang, Y. Chen, X. Qin, R. Xu, D. Dimitriadis, and T. Qin, “Personalized federated learning with adaptive batchnorm for healthcare,” IEEE Transactions on Big Data, 2022.
  27. K. Kopparapu and E. Lin, “Fedfmc: Sequential efficient federated learning on non-iid data,” arXiv preprint arXiv:2006.10937, 2020.
  28. R. Zaccone, A. Rizzardi, D. Caldarola, M. Ciccone, and B. Caputo, “Speeding up heterogeneous federated learning with sequentially trained superclients,” arXiv, 2022.
  29. S. Zeng, Z. Li, H. Yu, Y. He, Z. Xu, D. Niyato, and H. Yu, “Heterogeneous federated learning via grouped sequential-to-parallel training,” in Database Systems for Advanced Applications - 27th International Conference, DASFAA 2022, Virtual Event, April 11-14, 2022, Proceedings, PartII, ser. Lecture Notes in Computer Science, vol. 13246.   Springer, 2022, pp. 455–471.
  30. A. G. Roy, S. Siddiqui, S. Pölsterl, N. Navab, and C. Wachinger, “Braintorrent: A peer-to-peer environment for decentralized federated learning,” arXiv, 2019.
  31. N. Rieke, J. Hancox, W. Li, F. Milletari, H. R. Roth, S. Albarqouni, S. Bakas, M. N. Galtier, B. A. Landman, K. Maier-Hein et al., “The future of digital health with federated learning,” NPJ digital medicine, vol. 3, no. 1, pp. 1–7, 2020.
  32. Y. Li, W. Zhou, H. Wang, H. Mi, and T. M. Hospedales, “Fedh2l: Federated learning with model and statistical heterogeneity,” arXiv, 2021.
  33. S. Warnat-Herresthal, H. Schultze, K. L. Shastry, S. Manamohan, S. Mukherjee, V. Garg, R. Sarveswara, K. Händler, P. Pickkers, N. A. Aziz et al., “Swarm learning for decentralized and confidential clinical machine learning,” Nature, vol. 594, no. 7862, pp. 265–270, 2021.
  34. J. Zhang, X. Yang, H. Meng, Z. Lin, Y. Xu, and L. Cui, “A survey on knowledge enhanced ehr data mining,” in 5th International Conference on Crowd Science and Engineering, 2021, pp. 124–131.
  35. H. Meng, Z. Lin, F. Yang, Y. Xu, and L. Cui, “Knowledge distillation in medical data mining: a survey,” in 5th International Conference on Crowd Science and Engineering, 2021, pp. 175–182.
  36. A. Usmanova, F. Portet, P. Lalanda, and G. Vega, “A distillation-based approach integrating continual learning and federated learning for pervasive services,” in 3rd Workshop on Continual and Multimodal Learning for Internet of Things–Co-located with IJCAI 2021, 2021.
  37. A. Afonin and S. P. Karimireddy, “Towards model agnostic federated learning using knowledge distillation,” in International Conference on Learning Representations, 2022.
  38. C. Fang, Y. Xu, and D. N. Rockmore, “Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias,” in Proceedings of the IEEE International Conference on Computer Vision, 2013, pp. 1657–1664.
  39. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in NeurIPS, vol. 25, 2012, pp. 1097–1105.
  40. D. Li, Y. Yang, Y.-Z. Song, and T. M. Hospedales, “Deeper, broader and artier domain generalization,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 5542–5550.
  41. A. Reiss and D. Stricker, “Introducing a new benchmarked dataset for activity monitoring,” in 2012 16th International Symposium on Wearable Computers.   IEEE, 2012, pp. 108–109.
  42. W. Lu, J. Wang, H. Li, Y. Chen, and X. Xie, “Domain-invariant feature exploration for domain generalization,” arXiv preprint arXiv:2207.12020, 2022.
  43. M. Yurochkin, M. Agarwal, S. Ghosh, K. Greenewald, N. Hoang, and Y. Khazaeni, “Bayesian nonparametric federated learning of neural networks,” in ICML, 2019, pp. 7252–7261.
  44. P. Bilic, P. Christ, H. B. Li, E. Vorontsov, A. Ben-Cohen, G. Kaissis, A. Szeskin, C. Jacobs, G. E. H. Mamani, G. Chartrand et al., “The liver tumor segmentation benchmark (lits),” Medical Image Analysis, p. 102680, 2022.
  45. X. Xu, F. Zhou, B. Liu, D. Fu, and X. Bai, “Efficient multiple organ localization in ct image using 3d region proposal network,” IEEE transactions on medical imaging, vol. 38, no. 8, pp. 1885–1898, 2019.
  46. J. Yang, R. Shi, and B. Ni, “Medmnist classification decathlon: A lightweight automl benchmark for medical image analysis,” in ISBI, 2021, pp. 191–195.
  47. J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni, “Medmnist v2: A large-scale lightweight benchmark for 2d and 3d biomedical image classification,” arXiv preprint arXiv:2008.#TODO, 2021.
  48. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
  49. WIKIPEDIA. COVID-19. (2021, Feburary). [Online]. Available: https://en.wikipedia.org/wiki/COVID-19#cite_ref-WSJ-20210226_7-0
  50. O. U. Press. Oxford English Dictionary. (2020, April). [Online]. Available: https://www.oed.com/view/Entry/88575495
  51. S. Salehi, A. Abedi, S. Balakrishnan, A. Gholamrezanezhad et al., “Coronavirus disease 2019 (covid-19): a systematic review of imaging findings in 919 patients,” Ajr Am J Roentgenol, vol. 215, no. 1, pp. 87–93, 2020.
  52. U. Sait, K. Lal, S. Prajapati, R. Bhaumik, T. Kumar, S. Sanjana, and K. Bhalla, “Curated dataset for covid-19 posterior-anterior chest radiography images (x-rays),” Mendeley Data, vol. 1, 2020.
  53. WIKIPEDIA. Parkinson’s disease. (2021, April). [Online]. Available: https://en.wikipedia.org/wiki/Parkinson%27s_disease
  54. D. Weintraub, C. L. Comella, and S. Horn, “Parkinson’s disease–part 1: Pathophysiology, symptoms, burden, diagnosis, and assessment,” Am J Manag Care, vol. 14, no. 2 Suppl, pp. S40–S48, 2008.
  55. W. Yang, J. L. Hamilton, C. Kopil, J. C. Beck, C. M. Tanner, R. L. Albin, E. Ray Dorsey, N. Dahodwala, I. Cintina, P. Hogan et al., “Current and projected future economic burden of parkinson’s disease in the us,” npj Parkinson’s Disease, vol. 6, no. 1, pp. 1–9, 2020.
  56. J. Jankovic, “Parkinson’s disease: clinical features and diagnosis,” Journal of neurology, neurosurgery & psychiatry, vol. 79, no. 4, pp. 368–376, 2008.
  57. Y. Chen, X. Yang, B. Chen, C. Miao, and H. Yu, “Pdassist: Objective and quantified symptom assessment of parkinson’s disease via smartphone,” in 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).   IEEE, 2017, pp. 939–945.
  58. M. D. S. T. F. on Rating Scales for Parkinson’s Disease, “The unified parkinson’s disease rating scale (updrs): status and recommendations,” Movement Disorders, vol. 18, no. 7, pp. 738–750, 2003.
Citations (40)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up