From Optimization to Generalization: Fair Federated Learning against Quality Shift via Inter-Client Sharpness Matching (2404.17805v2)
Abstract: Due to escalating privacy concerns, federated learning has been recognized as a vital approach for training deep neural networks with decentralized medical data. In practice, it is challenging to ensure consistent imaging quality across various institutions, often attributed to equipment malfunctions affecting a minority of clients. This imbalance in image quality can cause the federated model to develop an inherent bias towards higher-quality images, thus posing a severe fairness issue. In this study, we pioneer the identification and formulation of this new fairness challenge within the context of the imaging quality shift. Traditional methods for promoting fairness in federated learning predominantly focus on balancing empirical risks across diverse client distributions. This strategy primarily facilitates fair optimization across different training data distributions, yet neglects the crucial aspect of generalization. To address this, we introduce a solution termed Federated learning with Inter-client Sharpness Matching (FedISM). FedISM enhances both local training and global aggregation by incorporating sharpness-awareness, aiming to harmonize the sharpness levels across clients for fair generalization. Our empirical evaluations, conducted using the widely-used ICH and ISIC 2019 datasets, establish FedISM's superiority over current state-of-the-art federated learning methods in promoting fairness. Code is available at https://github.com/wnn2000/FFL4MIA.
- Improving generalization in federated learning by seeking flat minima. In ECCV, 2022.
- Medical federated learning with joint graph purification for noisy label learning. Medical Image Anal., 90:102976, 2023.
- Noel CF Codella et al. Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). In ISBI, pages 168–172, 2018.
- BCN20000: Dermoscopic lesions in the wild. arXiv:1908.02288, 2019.
- Federated deep learning for detecting COVID-19 lung abnormalities in CT: a privacy-preserving multinational validation study. npj Digit. Medicine, 4(1):60, 2021.
- Fairfed: Enabling group fairness in federated learning. In AAAI, 2023.
- Robust heterogeneous federated learning under data corruption. In ICCV, 2023.
- Construction of a machine learning dataset through collaboration: The RSNA 2019 brain CT hemorrhage challenge. Radiology: Artificial Intelligence, 2(3), 2020.
- Sharpness-aware minimization for efficiently improving generalization. In ICLR, 2021.
- Deep residual learning for image recognition. In CVPR, 2016.
- Benchmarking neural network robustness to common corruptions and perturbations. In ICLR, 2019.
- Learn from others and be yourself in heterogeneous federated learning. In CVPR, 2022.
- Generalizable heterogeneous federated cross-correlation and instance similarity learning. IEEE Trans. Pattern Anal. Mach. Intell., 2023.
- Rethinking federated learning with domain shift: A prototype view. In CVPR, 2023.
- Federated learning for generalization, robustness, fairness: A survey and benchmark. arXiv:2311.06750, 2023.
- Robust generalization against photon-limited corruptions via worst-case sharpness minimization. In CVPR, 2023.
- Dynamic bank learning for semi-supervised federated image diagnosis with class imbalance. In MICCAI, pages 196–206, 2022.
- Fair federated medical image segmentation via client contribution estimation. In CVPR, pages 16302–16311, 2023.
- IOP-FL: Inside-outside personalization for federated medical image segmentation. IEEE Trans. Medical Imaging, 2023.
- Fair resource allocation in federated learning. In ICLR, 2020.
- FedBN: Federated learning on non-iid features via local batch normalization. In ICLR, 2021.
- FedDG: Federated domain generalization on medical image segmentation via episodic learning in continuous frequency space. In CVPR, pages 1013–1023, 2021.
- Collaborative fairness in federated learning. Federated Learning: Privacy and Incentive, pages 189–204, 2020.
- Communication-efficient learning of deep networks from decentralized data. In AISTATS, pages 1273–1282, 2017.
- Long-tail learning via logit adjustment. In ICLR, 2021.
- Agnostic federated learning. In ICML, pages 4615–4625, 2019.
- Minimax demographic group fairness in federated learning. In ACM FAccT, pages 142–159, 2022.
- Generalized federated learning via sharpness aware minimization. In ICML, 2022.
- Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. In ICLR, 2020.
- Dynamic regularized sharpness aware minimization in federated learning: Approaching global consistency and smooth landscape. In ICML, 2023.
- The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data, 5(1):1–9, 2018.
- Fednoro: Towards noise-robust federated learning by addressing class imbalance and label noise heterogeneity. In IJCAI, 2023.
- Fediic: Towards robust federated learning for class-imbalanced medical image classification. In MICCAI, 2023.
- Feda3i: Annotation quality-aware aggregation for federated medical image segmentation against heterogeneous annotation noise. In AAAI, 2024.
- Gradient driven rewards to guarantee fairness in collaborative machine learning. 2021.
- Heterogeneous federated learning: State-of-the-art and research challenges. ACM Computing Surveys, 56(3):1–44, 2023.
- Federated learning with label distribution skew via logits calibration. In ICML, 2022.
- Federated domain generalization with generalization adjustment. In CVPR, 2023.
- Imbsam: A closer look at sharpness-aware minimization in class-imbalanced recognition. In ICCV, pages 11345–11355, 2023.
- Surrogate gap minimization improves sharpness-aware training. In ICLR, 2022.