Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosis (2407.02261v2)

Published 2 Jul 2024 in cs.CV

Abstract: Medical image classification plays a crucial role in computer-aided clinical diagnosis. While deep learning techniques have significantly enhanced efficiency and reduced costs, the privacy-sensitive nature of medical imaging data complicates centralized storage and model training. Furthermore, low-resource healthcare organizations face challenges related to communication overhead and efficiency due to increasing data and model scales. This paper proposes a novel privacy-preserving medical image classification framework based on federated learning to address these issues, named FedMIC. The framework enables healthcare organizations to learn from both global and local knowledge, enhancing local representation of private data despite statistical heterogeneity. It provides customized models for organizations with diverse data distributions while minimizing communication overhead and improving efficiency without compromising performance. Our FedMIC enhances robustness and practical applicability under resource-constrained conditions. We demonstrate FedMIC's effectiveness using four public medical image datasets for classical medical image classification tasks.

PDF Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Interpretable cnn-multilevel attention transformer for rapid recognition of pneumonia from chest x-ray images. IEEE Journal of Biomedical and Health Informatics, 2023a.
  2. Recognition system for diagnosing pneumonia and bronchitis using children’s breathing sounds based on transfer learning. Intelligent Automation & Soft Computing, 37(3), 2023.
  3. Review on security of federated learning and its application in healthcare. Future Generation Computer Systems, 144:271–290, 2023.
  4. Fair federated medical image segmentation via client contribution estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16302–16311, 2023.
  5. Ensemble learning for retinal disease recognition under limited resources. Medical & Biological Engineering & Computing, pages 1–14, 2024a.
  6. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR, 2017.
  7. A comprehensive survey on federated learning techniques for healthcare informatics. Computational Intelligence and Neuroscience, 2023, 2023.
  8. Refrs: Resource-efficient federated recommender system for dynamic and diversified user preferences. ACM Transactions on Information Systems, 41(3):1–30, 2023.
  9. Comprehensive privacy analysis on federated recommender system against attribute inference attacks. IEEE Transactions on Knowledge and Data Engineering, 2023a.
  10. Lightfr: Lightweight federated recommendation with privacy-preserving matrix factorization. ACM Transactions on Information Systems, 41(4):1–28, 2023b.
  11. Prompt federated learning for weather forecasting: Toward foundation models on meteorological data. arXiv preprint arXiv:2301.09152, 2023b.
  12. Spatial-temporal prompt learning for federated weather forecasting. arXiv preprint arXiv:2305.14244, 2023c.
  13. Fedleo: An offloading-assisted decentralized federated learning framework for low earth orbit satellite networks. IEEE Transactions on Mobile Computing, 2023.
  14. Federated deep learning with prototype matching for object extraction from very-high-resolution remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 61:1–16, 2023c.
  15. Collaborative photonic crystal fiber property optimization: A new paradigm for reverse design. IEEE Photonics Technology Letters, 2023d.
  16. A distributed photonic crystal fiber reverse design framework based on multi-source knowledge fusion. Optical Fiber Technology, 84:103718, 2024.
  17. Hifuse: Hierarchical multi-scale feature fusion network for medical image classification. Biomedical Signal Processing and Control, 87:105534, 2024.
  18. Dm-cnn: Dynamic multi-scale convolutional neural network with uncertainty quantification for medical image classification. Computers in Biology and Medicine, 168:107758, 2024.
  19. Foundation models for weather and climate data understanding: A comprehensive survey. arXiv preprint arXiv:2312.03014, 2023e.
  20. Mask-cnn-transformer for real-time multi-label weather recognition. Knowledge-Based Systems, 278:110881, 2023f.
  21. Tempee: Temporal-spatial parallel transformer for radar echo extrapolation beyond auto-regression. arXiv preprint arXiv:2304.14131, 2023g.
  22. Diffusion model in semi-supervised scleral segmentation. In 2023 IEEE 6th International conference on pattern recognition and artificial intelligence (PRAI), pages 376–382. IEEE, 2023.
  23. Cmt: Interpretable model for rapid recognition pneumonia from chest x-ray images by fusing low complexity multilevel attention mechanism. arXiv preprint arXiv:2210.16584, 2022a.
  24. Dynamic multiscale fusion generative adversarial network for radar image extrapolation. IEEE Transactions on Geoscience and Remote Sensing, 60:1–11, 2022b.
  25. Less is more: Ensemble learning for retinal disease recognition under limited resources. arXiv preprint arXiv:2402.09747, 2024b.
  26. A privacy preserving system for movie recommendations using federated learning. ACM Transactions on Recommender Systems, 2023.
  27. Personalized adapter for large meteorology model on devices: Towards weather foundation models. arXiv preprint arXiv:2405.20348, 2024.
  28. Federated learning for medical applications: A taxonomy, current trends, challenges, and future research directions. IEEE Internet of Things Journal, 2023.
  29. Scoring aided federated learning on long-tailed data for wireless iomt based healthcare system. IEEE Journal of Biomedical and Health Informatics, 2023d.
  30. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429–450, 2020.
  31. Scaffold: Stochastic controlled averaging for federated learning. In International conference on machine learning, pages 5132–5143. PMLR, 2020.
  32. Overcoming forgetting in federated learning on non-iid data. arXiv preprint arXiv:1910.07796, 2019.
  33. Continual local training for better initialization of federated models. In 2020 IEEE International Conference on Image Processing (ICIP), pages 1736–1740. IEEE, 2020.
  34. Model-contrastive federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10713–10722, 2021a.
  35. Fedcorr: Multi-stage federated learning for label noise correction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10184–10193, 2022.
  36. Personalized federated learning with moreau envelopes. Advances in Neural Information Processing Systems, 33:21394–21405, 2020.
  37. Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. Advances in Neural Information Processing Systems, 33:3557–3568, 2020.
  38. Fedbn: Federated learning on non-iid features via local batch normalization. arXiv preprint arXiv:2102.07623, 2021b.
  39. Fedbe: Making bayesian model ensemble applicable to federated learning, 2021.
  40. Fedmd: Heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581, 2019.
  41. Ensemble distillation for robust model fusion in federated learning. Advances in Neural Information Processing Systems, 33:2351–2363, 2020.
  42. Communication-efficient federated distillation. arXiv preprint arXiv:2012.00632, 2020.
  43. Data-free knowledge distillation for heterogeneous federated learning. In International conference on machine learning, pages 12878–12889. PMLR, 2021.
  44. Federated dynamic sparse training: Computing less, communicating less, yet learning better. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 6080–6088, 2022.
  45. Federated learning with additional mechanisms on clients to reduce communication costs. arXiv preprint arXiv:1908.05891, 2019.
  46. Dispfl: Towards communication-efficient personalized federated learning via decentralized sparse training. arXiv preprint arXiv:2206.00187, 2022.
  47. Knowledge distillation in vision transformers: A critical review. arXiv preprint arXiv:2302.02108, 2023.
  48. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.
  49. A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the academy of marketing science, 43:115–135, 2015.
  50. Bold but cautious: Unlocking the potential of personalized federated learning through cautiously aggressive collaboration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19375–19384, 2023.
  51. Fedala: Adaptive local aggregation for personalized federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11237–11244, 2023e.
  52. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  53. Cynthia Dwork. Differential privacy. In International colloquium on automata, languages, and programming, pages 1–12. Springer, 2006.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Sufen Ren (5 papers)
  2. Yule Hu (1 paper)
  3. Shengchao Chen (12 papers)
  4. Guanjun Wang (4 papers)
Citations (1)
View on Semantic Scholar