FedMix: Approximation of Mixup under Mean Augmented Federated Learning (2107.00233v1)

Abstract: Federated learning (FL) allows edge devices to collectively learn a model without directly sharing data within each device, thus preserving privacy and eliminating the need to store data globally. While there are promising results under the assumption of independent and identically distributed (iid) local data, current state-of-the-art algorithms suffer from performance degradation as the heterogeneity of local data across clients increases. To resolve this issue, we propose a simple framework, Mean Augmented Federated Learning (MAFL), where clients send and receive averaged local data, subject to the privacy requirements of target applications. Under our framework, we propose a new augmentation algorithm, named FedMix, which is inspired by a phenomenal yet simple data augmentation method, Mixup, but does not require local raw data to be directly shared among devices. Our method shows greatly improved performance in the standard benchmark datasets of FL, under highly non-iid federated settings, compared to conventional algorithms.

• The paper introduces FedMix, a novel algorithm approximating Mixup under MAFL to tackle non-iid data in federated learning.
• It leverages averaged local data sharing to preserve privacy while approximating effective data augmentation.
• Empirical evaluations on benchmark datasets show that FedMix significantly outperforms traditional federated learning methods.

"FedMix: Approximation of Mixup under Mean Augmented Federated Learning" addresses the challenges posed by non-iid (non-independent and identically distributed) data in federated learning (FL). Federated learning enables multiple edge devices to collaboratively train machine learning models without sharing their local data, preserving data privacy and eliminating the need for centralized data storage. However, one of the critical issues in FL is that performance can significantly degrade when local data across clients is heterogeneous.

The authors propose a framework called Mean Augmented Federated Learning (MAFL) to mitigate this issue. The key idea behind MAFL is to allow clients to share and receive averaged local data while still adhering to privacy constraints. This method posits that by approximating the data distribution across different clients, it is possible to reduce the negative impact of data heterogeneity.

Building upon the MAFL framework, the paper introduces FedMix, an innovative data augmentation algorithm. FedMix draws inspiration from Mixup, a well-known and effective data augmentation technique that traditionally requires mixing raw data samples from different sources. However, to ensure that raw data is not directly shared among clients, FedMix approximates Mixup by using the averaged data exchanged under the MAFL framework. This approach retains the augmentation benefits of Mixup without compromising privacy.

The empirical evaluations demonstrate that FedMix significantly outperforms existing federated learning algorithms, particularly in settings where data across clients is highly non-iid. The standard benchmark datasets used in the paper show that FedMix achieves better model performance, making it a promising solution for practical applications of federated learning where data heterogeneity is a prevalent issue.

In summary, through MAFL and the FedMix algorithm, this paper provides an effective approach to improving federated learning under non-iid conditions, maintaining data privacy, and enhancing overall model performance.