Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

FedLPPA: Learning Personalized Prompt and Aggregation for Federated Weakly-supervised Medical Image Segmentation (2402.17502v2)

Published 27 Feb 2024 in cs.CV and eess.IV

Abstract: Federated learning (FL) effectively mitigates the data silo challenge brought about by policies and privacy concerns, implicitly harnessing more data for deep model training. However, traditional centralized FL models grapple with diverse multi-center data, especially in the face of significant data heterogeneity, notably in medical contexts. In the realm of medical image segmentation, the growing imperative to curtail annotation costs has amplified the importance of weakly-supervised techniques which utilize sparse annotations such as points, scribbles, etc. A pragmatic FL paradigm shall accommodate diverse annotation formats across different sites, which research topic remains under-investigated. In such context, we propose a novel personalized FL framework with learnable prompt and aggregation (FedLPPA) to uniformly leverage heterogeneous weak supervision for medical image segmentation. In FedLPPA, a learnable universal knowledge prompt is maintained, complemented by multiple learnable personalized data distribution prompts and prompts representing the supervision sparsity. Integrated with sample features through a dual-attention mechanism, those prompts empower each local task decoder to adeptly adjust to both the local distribution and the supervision form. Concurrently, a dual-decoder strategy, predicated on prompt similarity, is introduced for enhancing the generation of pseudo-labels in weakly-supervised learning, alleviating overfitting and noise accumulation inherent to local data, while an adaptable aggregation method is employed to customize the task decoder on a parameter-wise basis. Extensive experiments on four distinct medical image segmentation tasks involving different modalities underscore the superiority of FedLPPA, with its efficacy closely parallels that of fully supervised centralized training. Our code and data will be available.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. R. Wang et al., “Medical image segmentation using deep learning: A survey,” IET Image Processing, vol. 16, no. 5, pp. 1243–1267, 2022.
  2. S. Shit et al., “Cldice - a novel topology-preserving loss function for tubular structure segmentation,” in CVPR, 2021, pp. 16 560–16 569.
  3. L. Lin, Z. Wang et al., “Bsda-net: A boundary shape and distance aware joint learning framework for segmenting and classifying octa images,” in Medical Image Computing and Computer Assisted Intervention.   Springer, 2021, pp. 65–75.
  4. M. Goddard, “The eu general data protection regulation (gdpr): European regulation that has a global impact,” International Journal of Market Research, vol. 59, no. 6, pp. 703–705, 2017.
  5. T. Li et al., “Federated learning: Challenges, methods, and future directions,” IEEE Signal Process Mag, vol. 37, no. 3, pp. 50–60, 2020.
  6. B. McMahan et al., “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics.   PMLR, 2017, pp. 1273–1282.
  7. Q. Liu et al., “Feddg: Federated domain generalization on medical image segmentation via episodic learning in continuous frequency space,” in Proceedings of the CVPR, 2021, pp. 1013–1023.
  8. T. Li et al., “Federated optimization in heterogeneous networks,” Proceedings of Machine learning and systems, vol. 2, pp. 429–450, 2020.
  9. J. Wicaksana et al., “Fedmix: Mixed supervised federated learning for medical image segmentation,” IEEE Trans Med Imaging, 2022.
  10. A. Z. Tan et al., “Towards personalized federated learning,” IEEE Transactions on Neural Networks and Learning Systems, 2022.
  11. K. Wang et al., “Federated evaluation of on-device personalization,” arXiv preprint arXiv:1910.10252, 2019.
  12. L. Collins et al., “Exploiting shared representations for personalized federated learning,” in International conference on machine learning.   PMLR, 2021, pp. 2089–2099.
  13. X. Li et al., “Fedbn: Federated learning on non-iid features via local batch normalization,” in ICLR, 2020.
  14. W. Lu et al., “Personalized federated learning with adaptive batchnorm for healthcare,” IEEE Transactions on Big Data, 2022.
  15. Y. Chen et al., “Metafed: Federated learning among federations with cyclic knowledge distillation for personalized healthcare,” IEEE Transactions on Neural Networks and Learning Systems, 2023.
  16. L. Lin et al., “Unifying and personalizing weakly-supervised federated medical image segmentation via adaptive representation and aggregation,” in International Workshop on Machine Learning in Medical Imaging.   Springer, 2023, pp. 196–206.
  17. J. Zhang et al., “Fedala: Adaptive local aggregation for personalized federated learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 9, 2023, pp. 11 237–11 244.
  18. P. Liu et al., “Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing,” ACM Computing Surveys, vol. 55, no. 9, pp. 1–35, 2023.
  19. M. Jia et al., “Visual prompt tuning,” in European Conference on Computer Vision.   Springer, 2022, pp. 709–727.
  20. J. Liu et al., “Clip-driven universal model for organ segmentation and tumor detection,” in Proceedings of the ICCV, 2023, pp. 21 152–21 164.
  21. Z. Liang et al., “Tree energy loss: Towards sparsely annotated semantic segmentation,” in Proceedings of the CVPR, 2022, pp. 16 907–16 916.
  22. L. Lin et al., “Yolocurvseg: You only label one noisy skeleton for vessel-style curvilinear structure segmentation,” Medical Image Analysis, vol. 90, p. 102937, 2023.
  23. K. Zhang and X. Zhuang, “Cyclemix: A holistic strategy for medical image segmentation from scribble supervision,” in Proceedings of the CVPR, 2022, pp. 11 656–11 665.
  24. G. Valvano et al., “Learning to segment from scribbles using multi-scale adversarial attention gates,” IEEE Transactions on Medical Imaging, vol. 40, no. 8, pp. 1990–2001, 2021.
  25. Z. Yan et al., “Variation-aware federated learning with multi-source decentralized medical image data,” IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 7, pp. 2615–2628, 2020.
  26. C. I. Bercea et al., “Federated disentangled representation learning for unsupervised brain anomaly detection,” Nature Machine Intelligence, vol. 4, no. 8, pp. 685–695, 2022.
  27. J. Wang, Y. Jin, and L. Wang, “Personalizing federated medical image segmentation via local calibration,” in European Conference on Computer Vision.   Springer, 2022, pp. 456–472.
  28. Q. Liu et al., “Federated semi-supervised medical image classification via inter-client relation matching,” in Medical Image Computing and Computer Assisted Intervention.   Springer, 2021, pp. 325–335.
  29. H. Wu, B. Zhang, C. Chen, and J. Qin, “Federated semi-supervised medical image segmentation via prototype-based pseudo-labeling and contrastive learning,” IEEE Transactions on Medical Imaging, 2023.
  30. R. Yan et al., “Label-efficient self-supervised federated learning for tackling data heterogeneity in medical imaging,” IEEE Transactions on Medical Imaging, 2023.
  31. Y. Wu et al., “Distributed contrastive learning for medical image segmentation,” Medical Image Analysis, vol. 81, p. 102564, 2022.
  32. M. Zhu et al., “Feddm: Federated weakly supervised segmentation via annotation calibration and gradient de-conflicting,” IEEE Transactions on Medical Imaging, 2023.
  33. J. Pan, L. Lin et al., “pfedwsd: Unified weakly supervised personalized federated image segmentation via similarity-aware distillation,” Journal of Image and Graphics, 2023.
  34. H. Kervadec et al., “Bounding boxes for weakly supervised segmentation: Global constraints get close to full supervision,” in Medical imaging with deep learning.   PMLR, 2020, pp. 365–381.
  35. H. Zhang et al., “Weakly supervised segmentation with point annotations for histopathology images via contrast-based variational model,” in Proceedings of the CVPR, 2023, pp. 15 630–15 640.
  36. Z. Chen et al., “C-cam: Causal cam for weakly supervised semantic segmentation on medical image,” in Proceedings of the CVPR, 2022, pp. 11 676–11 685.
  37. J. Wang and B. Xia, “Bounding box tightness prior for weakly supervised image segmentation,” in Medical Image Computing and Computer Assisted Intervention.   Springer, 2021, pp. 526–536.
  38. X. Zhang et al., “Adaptive affinity loss and erroneous pseudo-label refinement for weakly supervised semantic segmentation,” in Proceedings of the 29th ACM MM, 2021, pp. 5463–5472.
  39. X. Luo et al., “Scribble-supervised medical image segmentation via dual-branch network and dynamically mixed pseudo labels supervision,” in Medical Image Computing and Computer Assisted Intervention.   Springer, 2022, pp. 528–538.
  40. L. Luo et al., “Oxnet: Deep omni-supervised thoracic disease detection from chest x-rays,” in Medical Image Computing and Computer Assisted Intervention.   Springer, 2021, pp. 537–548.
  41. Z. Chai et al., “Orf-net: Deep omni-supervised rib fracture detection from chest ct scans,” in Medical Image Computing and Computer Assisted Intervention.   Springer, 2022, pp. 238–248.
  42. Z. Xu et al., “Anti-interference from noisy labels: Mean-teacher-assisted confident learning for medical image segmentation,” IEEE Transactions on Medical Imaging, vol. 41, no. 11, pp. 3062–3073, 2022.
  43. H. Bahng et al., “Exploring visual prompts for adapting large-scale models,” arXiv preprint arXiv:2203.17274, 2022.
  44. J. Fu et al., “Dual attention network for scene segmentation,” in Proceedings of the CVPR, 2019, pp. 3146–3154.
  45. M. Tang et al., “Normalized cut loss for weakly-supervised cnn segmentation,” in Proceedings of the CVPR, 2018, pp. 1818–1827.
  46. J. Cheng et al., “Sam-med2d,” arXiv preprint arXiv:2308.16184, 2023.
  47. J. Sivaswamy et al., “A comprehensive retinal image dataset for the assessment of glaucoma from the optic nerve head analysis,” JSM Biomedical Imaging Data Papers, vol. 2, no. 1, p. 1004, 2015.
  48. F. Fumero et al., “Rim-one: An open retinal image database for optic nerve evaluation,” in 24th international symposium on computer-based medical systems (CBMS).   IEEE, 2011, pp. 1–6.
  49. J. I. Orlando et al., “Refuge challenge: A unified framework for evaluating automated methods for glaucoma assessment from fundus photographs,” Medical image analysis, vol. 59, p. 101570, 2020.
  50. J. Wu et al., “Gamma challenge: glaucoma grading from multi-modality images,” Medical Image Analysis, vol. 90, p. 102938, 2023.
  51. A. Agarwal et al., “The foveal avascular zone image database (fazid),” in Applications of Digital Image Processing XLIII, vol. 11510.   SPIE, 2020, pp. 507–512.
  52. M. Li et al., “Ipn-v2 and octa-500: Methodology and dataset for retinal image segmentation,” arXiv preprint arXiv:2012.07261, 2020.
  53. Y. Wang et al., “A deep learning-based quality assessment and segmentation system with a large-scale benchmark dataset for octa image,” arXiv preprint arXiv:2107.10476, 2021.
  54. Y. Ma et al., “Rose: a retinal oct-angiography vessel segmentation dataset and new model,” IEEE transactions on medical imaging, vol. 40, no. 3, pp. 928–939, 2020.
  55. J. Bernal et al., “Wm-dova maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians,” Computerized medical imaging and graphics, vol. 43, pp. 99–111, 2015.
  56. D. Vázquez et al., “A benchmark for endoluminal scene segmentation of colonoscopy images,” J Healthc Eng, vol. 2017, 2017.
  57. J. Silva et al., “Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer,” International journal of computer assisted radiology and surgery, vol. 9, pp. 283–293, 2014.
  58. D. Jha et al., “Kvasir-seg: A segmented polyp dataset,” in MultiMedia Modeling: 26th International Conference, MMM 2020.   Springer, 2020, pp. 451–462.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Li Lin (91 papers)
  2. Yixiang Liu (3 papers)
  3. Jiewei Wu (6 papers)
  4. Pujin Cheng (23 papers)
  5. Zhiyuan Cai (9 papers)
  6. Kenneth K. Y. Wong (10 papers)
  7. Xiaoying Tang (74 papers)
Citations (1)
View on Semantic Scholar