FedEM: Disambiguating Federated Learning Methods

• FedEM is an acronym denoting two distinct federated learning methods: a federated EM approach with compressed sufficient statistics and a privacy‐preserving error minimization method.
• The federated EM variant employs heterogeneity mitigation via memory terms and unbiased quantization to handle latent-variable models with finite-time convergence guarantees.
• The error minimization variant defends against gradient leakage by injecting bounded, adaptive perturbations into client inputs, balancing privacy with utility.

Searching arXiv for the primary FedEM papers to ground the article and confirm metadata. Searching for "Federated Expectation Maximization with heterogeneity mitigation and variance reduction" and "FedEM". {"query":"Federated Expectation Maximization with heterogeneity mitigation and variance reduction FedEM", "max_results": 5} FedEM is an acronym with two distinct meanings in federated learning. In one usage, FedEM denotes “Federated Expectation Maximization with heterogeneity mitigation and variance reduction,” introduced as a federated extension of the classical EM algorithm for latent-variable models, with compressed communication of complete-data sufficient statistics, support for partial participation, and finite-time guarantees for smooth non-convex problems (Dieuleveut et al., 2021). In another usage, FedEM denotes “Federated Error Minimization,” a privacy-preserving framework that defends against gradient leakage attacks by injecting bounded adaptive perturbations into client-side data before gradient computation (Xu et al., 8 Mar 2025). The term therefore requires contextual disambiguation: in unsupervised federated inference it refers to federated EM, whereas in privacy-defense work it refers to an input-perturbation mechanism for standard FL.

1. Terminological scope and disambiguation

The literature surrounding “FedEM” is unusually sensitive to acronym collision. The following names occur in closely related federated-learning contexts but denote materially different methods.

Term	Expansion	Characterization
FedEM (Dieuleveut et al., 2021)	Federated Expectation Maximization	Federated EM with compressed sufficient statistics, heterogeneity mitigation, and variance reduction
FedEM (Xu et al., 8 Mar 2025)	Federated Error Minimization	Privacy-preserving FL via bounded adaptive perturbations against gradient leakage
FedEPM (Zhou et al., 2022)	Federated Exact Penalty Method	Exact-penalty federated optimization with periodic communication and Laplace-noised uploads
FedMon (Zehra et al., 11 Oct 2025)	Federated eBPF Monitoring	Federated eBPF-based anomaly detection for multi-cluster Kubernetes monitoring

A further neighboring line of work studies federated empirical risk minimization via a distributed interior-point method with Hessian sketching, but that work is framed as federated ERM rather than as FedEM (Bian et al., 2023). This suggests that citation by arXiv identifier is particularly important when the acronym appears without expansion.

2. Federated Expectation Maximization as a latent-variable federated method

In the 2021 formulation, FedEM extends EM to the federated setting by exploiting the structural fact that EM operates on complete-data sufficient statistics rather than on raw gradients. The paper considers a non-convex finite-sum objective

$$F(\theta)=\frac{1}{n}\sum_{i=1}^n f_i(\theta)+R(\theta), \qquad \Theta\subseteq \mathbb{R}^d,$$

with local losses of the latent-variable form

$$f_i(\theta)=-\frac{1}{m}\sum_{j=1}^m \log p(y_{ij};\theta), \qquad p(y;\theta)=\int_Z p(y,z;\theta)\,\mu(dz).$$

The complete-data model is assumed to belong to a curved exponential family,

$$p(y,z;\theta)=\rho(y,z)\exp\left\{ \langle s(y,z),\phi(\theta)\rangle - \psi(\theta)\right\},$$

where $s(y,z)\in\mathbb{R}^q$ is the complete-data sufficient statistic. For worker $i$, the E-step computes

$$\bar s_i(\theta)=\frac{1}{m}\sum_{j=1}^m \bar s_{ij}(\theta), \qquad \bar s_{ij}(\theta)=\int_Z s(y_{ij},z)\,p(z\mid y_{ij};\theta)\,\mu(dz),$$

and the global statistic is

$$\bar s(\theta)=\frac{1}{n}\sum_{i=1}^n \bar s_i(\theta).$$

The M-step is represented by

$$T(s)\in \arg\min_{\theta\in\Theta}\left\{\psi(\theta)+R(\theta)-\langle s,\phi(\theta)\rangle\right\}.$$

A central conceptual move in the paper is the expectation-space view of EM. Instead of writing only the parameter iterate $\theta_{k+1}=T(\bar s(\theta_k))$, the method tracks an expectation-space state $S_k$ through

$$f_i(\theta)=-\frac{1}{m}\sum_{j=1}^m \log p(y_{ij};\theta), \qquad p(y;\theta)=\int_Z p(y,z;\theta)\,\mu(dz).$$0

Defining

$$f_i(\theta)=-\frac{1}{m}\sum_{j=1}^m \log p(y_{ij};\theta), \qquad p(y;\theta)=\int_Z p(y,z;\theta)\,\mu(dz).$$1

the EM fixed-point condition becomes $$f_i(\theta)=-\frac{1}{m}\sum_{j=1}^m \log p(y_{ij};\theta), \qquad p(y;\theta)=\int_Z p(y,z;\theta)\,\mu(dz).$$2, and stationarity is measured via $$f_i(\theta)=-\frac{1}{m}\sum_{j=1}^m \log p(y_{ij};\theta), \qquad p(y;\theta)=\int_Z p(y,z;\theta)\,\mu(dz).$$3. This formulation is important because it makes clear why federated EM differs from federated gradient methods: the transmitted object is a sufficient-statistic surrogate tied to the E-step/M-step decomposition rather than a parameter gradient (Dieuleveut et al., 2021).

3. Core FedEM mechanism: compressed sufficient statistics and heterogeneity mitigation

FedEM is a federated stochastic-approximation variant of EM in which active workers communicate compressed differences of complete-data sufficient statistics and maintain local memory variables to mitigate heterogeneity. At round $$f_i(\theta)=-\frac{1}{m}\sum_{j=1}^m \log p(y_{ij};\theta), \qquad p(y;\theta)=\int_Z p(y,z;\theta)\,\mu(dz).$$4, each active worker $$f_i(\theta)=-\frac{1}{m}\sum_{j=1}^m \log p(y_{ij};\theta), \qquad p(y;\theta)=\int_Z p(y,z;\theta)\,\mu(dz).$$5 builds an approximation $$f_i(\theta)=-\frac{1}{m}\sum_{j=1}^m \log p(y_{ij};\theta), \qquad p(y;\theta)=\int_Z p(y,z;\theta)\,\mu(dz).$$6 of its local EM statistic $$f_i(\theta)=-\frac{1}{m}\sum_{j=1}^m \log p(y_{ij};\theta), \qquad p(y;\theta)=\int_Z p(y,z;\theta)\,\mu(dz).$$7, forms the residual

$$f_i(\theta)=-\frac{1}{m}\sum_{j=1}^m \log p(y_{ij};\theta), \qquad p(y;\theta)=\int_Z p(y,z;\theta)\,\mu(dz).$$8

compresses $$f_i(\theta)=-\frac{1}{m}\sum_{j=1}^m \log p(y_{ij};\theta), \qquad p(y;\theta)=\int_Z p(y,z;\theta)\,\mu(dz).$$9 with an unbiased quantizer $$p(y,z;\theta)=\rho(y,z)\exp\left\{ \langle s(y,z),\phi(\theta)\rangle - \psi(\theta)\right\},$$0, sends the compressed message to the server, and updates its memory through

$$p(y,z;\theta)=\rho(y,z)\exp\left\{ \langle s(y,z),\phi(\theta)\rangle - \psi(\theta)\right\},$$1

The server-side updates are

$$p(y,z;\theta)=\rho(y,z)\exp\left\{ \langle s(y,z),\phi(\theta)\rangle - \psi(\theta)\right\},$$2

$$p(y,z;\theta)=\rho(y,z)\exp\left\{ \langle s(y,z),\phi(\theta)\rangle - \psi(\theta)\right\},$$3

$$p(y,z;\theta)=\rho(y,z)\exp\left\{ \langle s(y,z),\phi(\theta)\rangle - \psi(\theta)\right\},$$4

followed by the M-step $$p(y,z;\theta)=\rho(y,z)\exp\left\{ \langle s(y,z),\phi(\theta)\rangle - \psi(\theta)\right\},$$5.

The heterogeneity-mitigation device is the memory term $$p(y,z;\theta)=\rho(y,z)\exp\left\{ \langle s(y,z),\phi(\theta)\rangle - \psi(\theta)\right\},$$6. A naive design would compress $$p(y,z;\theta)=\rho(y,z)\exp\left\{ \langle s(y,z),\phi(\theta)\rangle - \psi(\theta)\right\},$$7, but the paper argues that under heterogeneous client distributions those local differences need not vanish at the optimum, so compression noise can remain large. FedEM instead compresses the residual after subtracting the memory estimate $$p(y,z;\theta)=\rho(y,z)\exp\left\{ \langle s(y,z),\phi(\theta)\rangle - \psi(\theta)\right\},$$8, with the intended effect that $$p(y,z;\theta)=\rho(y,z)\exp\left\{ \langle s(y,z),\phi(\theta)\rangle - \psi(\theta)\right\},$$9 and $s(y,z)\in\mathbb{R}^q$0 becomes a smaller correction term. In that sense, FedEM is not only a communication-compression method; it is a compression method explicitly adapted to heterogeneity in latent-variable sufficient statistics (Dieuleveut et al., 2021).

4. Compression model, partial participation, and finite-time guarantees

FedEM assumes an unbiased quantizer with bounded second moment,

$s(y,z)\in\mathbb{R}^q$1

where $s(y,z)\in\mathbb{R}^q$2 measures compression noise and $s(y,z)\in\mathbb{R}^q$3 corresponds to no compression. The paper discusses examples such as dithering and block quantization. Compression reduces communication per round, while the unbiasedness condition preserves the correctness of the server-side estimate in expectation.

Partial participation is modeled by Bernoulli activity variables $s(y,z)\in\mathbb{R}^q$4, which modify the memory and aggregation steps to

$s(y,z)\in\mathbb{R}^q$5

$s(y,z)\in\mathbb{R}^q$6

The effective noise level becomes

$s(y,z)\in\mathbb{R}^q$7

so reduced participation worsens the effective noise and shrinks the largest stable step size. This formalizes the intuitive point that communication sparsity and participation sparsity are not interchangeable: both save bandwidth, but the latter directly perturbs the stochastic field seen by the server.

The theory is stated for smooth non-convex problems under assumptions including that $s(y,z)\in\mathbb{R}^q$8 is $s(y,z)\in\mathbb{R}^q$9 with globally Lipschitz gradient, that $i$0 with $i$1 positive definite and eigenvalues in $i$2, that each $i$3 is globally Lipschitz, and that local stochastic estimators are unbiased with bounded variance. For full participation, the paper derives a bound of the form

$i$4

and summarizes the resulting optimization complexity as

$i$5

The paper’s stated interpretation is that the algorithm behaves like standard stochastic non-convex optimization in the high-noise regime, but can recover an $i$6-type dependence in low-noise or finite-sum settings (Dieuleveut et al., 2021).

5. Variance reduction, VR-FedEM, and federated missing-data imputation

The same paper introduces VR-FedEM, a variance-reduced extension inspired by SPIDER-style control variates. In the finite-sum setting, each worker periodically computes a full local statistic at the start of an outer loop and then updates it during inner iterations with a minibatch-difference estimator,

$i$7

The same compressed-memory mechanism is then applied through

$i$8

When $i$9 and $$\bar s_i(\theta)=\frac{1}{m}\sum_{j=1}^m \bar s_{ij}(\theta), \qquad \bar s_{ij}(\theta)=\int_Z s(y_{ij},z)\,p(z\mid y_{ij};\theta)\,\mu(dz),$$0, VR-FedEM reduces to SPIDER-EM. The paper gives an optimization complexity

$$\bar s_i(\theta)=\frac{1}{m}\sum_{j=1}^m \bar s_{ij}(\theta), \qquad \bar s_{ij}(\theta)=\int_Z s(y_{ij},z)\,p(z\mid y_{ij};\theta)\,\mu(dz),$$1

with $$\bar s_i(\theta)=\frac{1}{m}\sum_{j=1}^m \bar s_{ij}(\theta), \qquad \bar s_{ij}(\theta)=\int_Z s(y_{ij},z)\,p(z\mid y_{ij};\theta)\,\mu(dz),$$2 chosen constant and independent of the number of local samples $$\bar s_i(\theta)=\frac{1}{m}\sum_{j=1}^m \bar s_{ij}(\theta), \qquad \bar s_{ij}(\theta)=\int_Z s(y_{ij},z)\,p(z\mid y_{ij};\theta)\,\mu(dz),$$3. The stated benefit is lower variance and better finite-time behavior while retaining communication savings (Dieuleveut et al., 2021).

Empirically, the paper evaluates FedEM and VR-FedEM on synthetic Gaussian mixture models, MNIST-based GMM inference, and a biodiversity missing-values imputation task. On synthetic GMMs, both methods converge and VR-FedEM exhibits lower variance and smoother trajectories. On the MNIST-based GMM task, the algorithms estimate mixture weights and reduce the mean-field norm as expected. The application case, termed FedMissEM, specializes the framework to federated missing-data imputation. For observer $$\bar s_i(\theta)=\frac{1}{m}\sum_{j=1}^m \bar s_{ij}(\theta), \qquad \bar s_{ij}(\theta)=\int_Z s(y_{ij},z)\,p(z\mid y_{ij};\theta)\,\mu(dz),$$4, the data are a partially observed matrix $$\bar s_i(\theta)=\frac{1}{m}\sum_{j=1}^m \bar s_{ij}(\theta), \qquad \bar s_{ij}(\theta)=\int_Z s(y_{ij},z)\,p(z\mid y_{ij};\theta)\,\mu(dz),$$5 with observed indices $$\bar s_i(\theta)=\frac{1}{m}\sum_{j=1}^m \bar s_{ij}(\theta), \qquad \bar s_{ij}(\theta)=\int_Z s(y_{ij},z)\,p(z\mid y_{ij};\theta)\,\mu(dz),$$6, and the low-rank parameterization is

$$\bar s_i(\theta)=\frac{1}{m}\sum_{j=1}^m \bar s_{ij}(\theta), \qquad \bar s_{ij}(\theta)=\int_Z s(y_{ij},z)\,p(z\mid y_{ij};\theta)\,\mu(dz),$$7

with regularized objective

$$\bar s_i(\theta)=\frac{1}{m}\sum_{j=1}^m \bar s_{ij}(\theta), \qquad \bar s_{ij}(\theta)=\int_Z s(y_{ij},z)\,p(z\mid y_{ij};\theta)\,\mu(dz),$$8

The eBird case study uses French eBird data comprising 2,465 observers, 9,721 sites, and 525 months for Common Buzzard and Mallard, and the recovered national-level temporal trends are reported to align well with LOESS-smoothed empirical trends (Dieuleveut et al., 2021).

6. Later theory: FedGrEM and the analysis of federated EM algorithms

Subsequent work generalized the theory of federated EM-style procedures. “Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithms” introduces FedGrEM, a federated gradient EM algorithm for mixture models that explicitly incorporates task heterogeneity and adversarial contamination (Tian et al., 2023). In that formulation, good tasks $$\bar s_i(\theta)=\frac{1}{m}\sum_{j=1}^m \bar s_{ij}(\theta), \qquad \bar s_{ij}(\theta)=\int_Z s(y_{ij},z)\,p(z\mid y_{ij};\theta)\,\mu(dz),$$9 satisfy

$$\bar s(\theta)=\frac{1}{n}\sum_{i=1}^n \bar s_i(\theta).$$0

with component parameters close to task-central representatives in the sense

$$\bar s(\theta)=\frac{1}{n}\sum_{i=1}^n \bar s_i(\theta).$$1

while a fraction $$\bar s(\theta)=\frac{1}{n}\sum_{i=1}^n \bar s_i(\theta).$$2 of tasks may be arbitrarily corrupted.

FedGrEM performs a local E-step, a local gradient M-step,

$$\bar s(\theta)=\frac{1}{n}\sum_{i=1}^n \bar s_i(\theta).$$3

and a central aggregation step with a quadratic penalty toward a common center. The paper explicitly relates this construction to FedEM and FedGMM, noting that if the penalty parameter $$\bar s(\theta)=\frac{1}{n}\sum_{i=1}^n \bar s_i(\theta).$$4, FedGrEM reduces to a simplified gradient analogue of FedEM/FedGMM. Its main theorem yields an error decomposition

$$\bar s(\theta)=\frac{1}{n}\sum_{i=1}^n \bar s_i(\theta).$$5

The interpretation given in the paper is that federated EM improves over local single-task learning when shared component parameters are sufficiently similar across tasks and the outlier fraction is small.

This later analysis also makes explicit a practical complication that is implicit in many mixture-model implementations: label permutation. Because federated aggregation requires a common component indexing across tasks, the paper studies alignment procedures and proves recovery of a common permutation across clean tasks under sufficient separation conditions. In this sense, FedGrEM functions both as a new algorithm and as a theoretical lens through which earlier FedEM-style methods can be understood (Tian et al., 2023).

7. Alternative acronym usage: Federated Error Minimization

A separate 2025 paper uses FedEM to mean “Federated Error Minimization,” a privacy-preserving FL framework aimed at gradient leakage attacks such as Deep Leakage from Gradients under a semi-honest server model (Xu et al., 8 Mar 2025). Its core idea is to move perturbation from gradient space into data space. Standard federated learning is written as

$$\bar s(\theta)=\frac{1}{n}\sum_{i=1}^n \bar s_i(\theta).$$6

while FedEM optimizes over both the model and client perturbations,

$$\bar s(\theta)=\frac{1}{n}\sum_{i=1}^n \bar s_i(\theta).$$7

Each client iteratively updates a bounded perturbation vector $$\bar s(\theta)=\frac{1}{n}\sum_{i=1}^n \bar s_i(\theta).$$8, applies the perturbed input $$\bar s(\theta)=\frac{1}{n}\sum_{i=1}^n \bar s_i(\theta).$$9, computes local gradients on perturbed data, and uploads those gradients. The perturbation update is a projected sign-gradient step followed by projection onto the annulus $$T(s)\in \arg\min_{\theta\in\Theta}\left\{\psi(\theta)+R(\theta)-\langle s,\phi(\theta)\rangle\right\}.$$0. The stated purpose is not to sabotage learning, but to make gradients harder to invert while preserving downstream utility.

The experimental setup uses MNIST, FashionMNIST, and CIFAR-10; 4 clients; 50 global rounds on MNIST and FMNIST; 100 global rounds on CIFAR-10; learning rate 0.01; SGD; perturbation radius initialized to 8; and early stopping patience of 30 epochs. The baselines are FedSGD without privacy protection, Gaussian DP, Laplace DP, and DP-Clip. Utility is evaluated with test accuracy and validation accuracy, while privacy is evaluated with MSE, SSIM, and PSNR computed between DLG reconstructions and original images, with higher MSE and lower SSIM/PSNR interpreted as stronger privacy. On MNIST, the paper reports the FedSGD-to-FedEM comparison as test accuracy $$T(s)\in \arg\min_{\theta\in\Theta}\left\{\psi(\theta)+R(\theta)-\langle s,\phi(\theta)\rangle\right\}.$$1, validation accuracy $$T(s)\in \arg\min_{\theta\in\Theta}\left\{\psi(\theta)+R(\theta)-\langle s,\phi(\theta)\rangle\right\}.$$2, test MSE $$T(s)\in \arg\min_{\theta\in\Theta}\left\{\psi(\theta)+R(\theta)-\langle s,\phi(\theta)\rangle\right\}.$$3, feature/“FEA” MSE $$T(s)\in \arg\min_{\theta\in\Theta}\left\{\psi(\theta)+R(\theta)-\langle s,\phi(\theta)\rangle\right\}.$$4, SSIM $$T(s)\in \arg\min_{\theta\in\Theta}\left\{\psi(\theta)+R(\theta)-\langle s,\phi(\theta)\rangle\right\}.$$5, and PSNR $$T(s)\in \arg\min_{\theta\in\Theta}\left\{\psi(\theta)+R(\theta)-\langle s,\phi(\theta)\rangle\right\}.$$6. On FMNIST, test accuracy changes from $$T(s)\in \arg\min_{\theta\in\Theta}\left\{\psi(\theta)+R(\theta)-\langle s,\phi(\theta)\rangle\right\}.$$7 to $$T(s)\in \arg\min_{\theta\in\Theta}\left\{\psi(\theta)+R(\theta)-\langle s,\phi(\theta)\rangle\right\}.$$8, and on CIFAR-10 from $$T(s)\in \arg\min_{\theta\in\Theta}\left\{\psi(\theta)+R(\theta)-\langle s,\phi(\theta)\rangle\right\}.$$9 to $\theta_{k+1}=T(\bar s(\theta_k))$0, with privacy gains reported as weaker on CIFAR-10 than on MNIST and FMNIST.

This 2025 FedEM is conceptually unrelated to federated expectation-maximization. It addresses privacy leakage in gradient-sharing FL rather than latent-variable inference, and it operates through bounded adaptive perturbations on local inputs rather than through compressed sufficient statistics. The same distinction separates both meanings of FedEM from adjacent methods such as FedEPM’s exact-penalty formulation (Zhou et al., 2022), the second-order federated ERM interior-point method (Bian et al., 2023), and FedMon’s federated eBPF anomaly-detection pipeline (Zehra et al., 11 Oct 2025).