---
title: 'FedEve: Drift-Compensated Federated Learning'
url: https://www.emergentmind.com/topics/fedeve
type: topic
---

# FedEve: Drift-Compensated Federated Learning

FedEve is a federated learning method for cross-device settings that addresses two distinct sources of optimization bias under data heterogeneity: **client drift**, induced by multiple local updates on non-iid data, and **period drift**, induced by partial client participation when the sampled clients at a round do not reflect the full population distribution. The method is introduced as a predict-observe framework in which server-side optimization provides a prediction, client-side local training provides an observation, and the two are fused through a Bayesian/Kalman filter so that the two forms of drift can compensate each other and reduce update variance [2508.14539].

## 1. Problem formulation in cross-device federated learning

FedEve is formulated in the standard federated optimization setting with $N$ clients collaboratively minimizing a weighted global objective
$$
F(w) \triangleq \sum_{i=1}^N p_i F_i(w) = \mathbb{E}_{i\sim p} [F_i(w)],
$$
where $p_i \ge 0$, $\sum_{i=1}^N p_i = 1$, and
$$
F_i(w) = \mathbb{E}_{x\sim \mathcal{D}_i} \, f_i(w;x), \quad \nabla F_i(w) = \mathbb{E}_{x\sim \mathcal{D}_i} \, \nabla f_i(w;x).
$$
In cross-device FL, only a small subset of clients participates at each communication round,
$$
S_t \subset \{1,\dots,N\}, \quad |S_t| \ll N,
$$
which induces a sampled objective
$$
F_{S_t}(w) \triangleq \sum_{i\in S_t} \tilde p_i F_i(w), \quad \tilde p_i = \frac{p_i}{\sum_{j\in S_t} p_j}.
$$
Under non-iid data, $F_{S_t}(w)$ can deviate from $F(w)$ because the participating set may not reflect the population distribution [2508.14539].

The method is positioned against the standard FedAvg update pattern. Clients receive $w_t$, perform $K$ local steps with step-size $\eta$,
$$
w_{t,0}^{(i)} = w_t, \quad w_{t,k+1}^{(i)} = w_{t,k}^{(i)} - \eta \,\nabla F_i\!\left(w_{t,k}^{(i)}\right), \quad k=0,\ldots,K-1,
$$
and the server aggregates
$$
w_{t+1} = w_t + \sum_{i\in S_t} \alpha_i \big(w_{t,K}^{(i)} - w_t\big), \quad \alpha_i = \frac{p_i}{\sum_{j\in S_t} p_j}.
$$
Using $\Delta w_t^{(i)} \triangleq w_t - w_{t,K}^{(i)}$ and $\Delta w_t \triangleq \sum_{i\in S_t} \alpha_i \Delta w_t^{(i)}$, the update becomes
$$
w_{t+1} = w_t - \Delta w_t.
$$
FedEve departs from this baseline by explicitly modeling the discrepancy between the sampled objective and the global objective as well as the discrepancy introduced by local training.

## 2. Client drift, period drift, and their interaction

FedEve distinguishes two forms of drift. **Client drift** arises from multiple local steps on heterogeneous local objectives. One convenient measure is
$$
D_{\text{client}(t)} \triangleq \left\| \sum_{i\in S_t} \alpha_i \big(w_{t,K}^{(i)} - w_t\big) + \eta K \,\nabla F(w_t) \right\|.
$$
The paper also formalizes it through local-optima mismatch:
$$
\text{Client Drift} := \mathbb{E}_{i \in S_t}\!\left[ \left\|\nabla F_i(w_i^*) - \nabla F_i(w_{S_t}^*)\right\|^2 \right],
$$
where $w_i^*$ is the local minimizer of $F_i$ and $w_{S_t}^*$ is the minimizer of $F_{S_t}$ [2508.14539].

**Period drift** is specific to cross-device FL with partial participation. It measures the shift in objective or gradient across rounds because different subsets of clients are sampled:
$$
D_{\text{period}(t)} \triangleq \left\| \nabla F_{S_t}(w_t) - \nabla F(w_t) \right\|,
$$
or, in expected squared norm form,
$$
\text{Period Drift} := \mathbb{E}_{S_t}\!\left[ \left\| \frac{1}{|S_t|}\sum_{i\in S_t} \nabla F_i(w_t) - \nabla F(w_t) \right\|^2 \right].
$$
The distinction is causal as well as analytic: client drift is tied to $K>1$ local steps on non-iid data, whereas period drift is tied to biased client sampling under non-iid data.

The reported empirical finding is that, as heterogeneity increases—for example when Dirichlet $\alpha$ decreases—period drift becomes particularly detrimental in cross-device FL. The stated intuition is that client drift scales with local steps $K$ and heterogeneity $H$ as $\propto K\cdot H$, whereas period drift scales as $\propto H/|S_t|$. In cross-device regimes where $|S_t|$ is small, the objective itself shifts across rounds, so period drift can have a larger impact on convergence and accuracy than client drift [2508.14539]. This suggests that methods designed only to stabilize local training can remain inadequate when the dominant disturbance is the round-to-round movement of the sampled objective.

## 3. Predict-observe fusion and the FedEve algorithm

FedEve treats server-side optimization as a **prediction** of the next update step and client-side local training as an **observation** of that step. The framework assumes Gaussian-like independent noises:
$$
\hat{w}_{t+1} = g(w_t) + Q_t,\quad Q_t\sim \mathcal{N}(0,\sigma_{Q_t}^2),
$$
$$
\tilde{w}_{t+1} = h(\hat{w}_{t+1}) + R_t,\quad R_t\sim \mathcal{N}(0,\sigma_{R_t}^2),
$$
where $Q_t$ corresponds to period drift and $R_t$ corresponds to client drift. The independence assumption is made with respect to $w_t$ and between the two noise sources [2508.14539].

The instantiated method specializes this framework as a Kalman filter with Nesterov momentum. The server maintains a momentum variable $M_t$ and predicts
$$
\hat{w}_{t+1} = w_t - \eta_g M_t.
$$
Clients receive $\hat{w}_{t+1}$ rather than $w_t$, initialize from it, run local training, and return local models. The server forms the observed average update at the predicted state,
$$
\Delta \tilde{w}_t = \sum_{i\in S_t} \alpha_i\big(\hat{w}_{t+1} - w_{t,K}^{(i)}\big).
$$
It then updates the predictive variance and scalar Kalman gain,
$$
\hat{\sigma}_{t+1}^2 = \sigma_t^2 + \sigma_{Q_t}^2,\quad G_{\text{kal}} = \frac{\hat{\sigma}_{t+1}^2}{\hat{\sigma}_{t+1}^2 + \sigma_{R_t}^2},
$$
and fuses prediction with observation through
$$
M_{t+1} = M_t + G_{\text{kal}}\big(\Delta \tilde{w}_t - M_t\big),
$$
followed by
$$
w_{t+1} = w_t - \eta_g M_{t+1},\quad \sigma_{t+1}^2 = (1 - G_{\text{kal}})\hat{\sigma}_{t+1}^2.
$$

The paper further defines server-side estimators for the two variance components:
$$
\sigma_{Q_t}^2 := \frac{\sum_{i=1}^d \big(M_t^{(i)} - \Delta \tilde{w}_t^{(i)}\big)^2}{|S_t|\,d},
$$
$$
\sigma_{R_t}^2 := \frac{\sum_{k\in S_t}\sum_{i=1}^d \big(\Delta \tilde{w}_t^{(k,i)} - \Delta \tilde{w}_t^{(i)}\big)^2}{|S_t|^2\,d}.
$$
Here $\sigma_{Q_t}^2$ measures disagreement between the server prediction and the observed average update, while $\sigma_{R_t}^2$ measures the heterogeneity of client updates around that average.

Operationally, the server initializes $w_0$, $M_0=0$, and $\sigma_0^2=0$, chooses $\eta_g$—the paper gives $\eta_g=1$ as an example—and repeats prediction, client sampling, broadcast, collection of local models, variance estimation, Kalman fusion, and global update. On the client side, each client receives $\hat{w}_{t+1}$, sets $w_{t,0}^{(i)}\leftarrow \hat{w}_{t+1}$, performs local SGD for $K$ steps, and returns $w_{t,K}^{(i)}$. The principal hyperparameters are the global learning rate $\eta_g$, local learning rate $\eta$, number of local epochs $E$, local steps per epoch, client sampling size $|S_t|$, and optionally the server momentum type. The reported experiments use $\eta_g=1$ and, by default in cross-device settings, $E=1$ [2508.14539].

## 4. Theoretical properties

The theoretical analysis assumes smoothness and Lipschitz continuity of the per-example loss, bounded variance of client gradients, and Gaussian-like independent drift noises. Specifically, for any data $z$, $l(w;z)$ is $L$-smooth and $L$-Lipschitz,
$$
\|\nabla l(w;z) - \nabla l(w';z)\| \le L\|w - w'\|,\quad |l(w;z)-l(w';z)|\le L\|w-w'\|,
$$
and for each client $i$,
$$
\mathbb{E}\big\|\nabla f_i(w) - \nabla f(w)\big\| \le \sigma.
$$
The drift variables satisfy $Q_t \sim \mathcal{N}(0,\sigma_{Q_t}^2)$ and $R_t\sim \mathcal{N}(0,\sigma_{R_t}^2)$, independent of $w_t$ and of each other [2508.14539].

A central analytical statement is the closed-form fusion rule. If the prediction has mean $\mu_1$ and variance $\hat{\sigma}_{t+1}^2$, and the observation has mean $\mu_2$ and variance $\sigma_{R_t}^2$, then the fused estimator has
$$
\mu_{\text{fused}} = \frac{\mu_1\,\sigma_{R_t}^2 + \mu_2\,\hat{\sigma}_{t+1}^2}{\hat{\sigma}_{t+1}^2 + \sigma_{R_t}^2},\quad
\sigma_{\text{fused}}^2 = \frac{\hat{\sigma}_{t+1}^2\,\sigma_{R_t}^2}{\hat{\sigma}_{t+1}^2 + \sigma_{R_t}^2}.
$$
The fused variance is therefore strictly smaller than either component variance, or equivalently
$$
\sigma_{\text{fused}}^2 \le \min\{\hat{\sigma}_{t+1}^2,\sigma_{R_t}^2\}.
$$

Under the same assumptions and $\eta_g \le \frac{1}{L}$, the method satisfies the non-convex convergence bound
$$
\frac{1}{T}\sum_{t=0}^{T-1}\mathbb{E}\big[\|\nabla F(w_t)\|^2\big] \le \mathcal{O}\!\left(\frac{L\big(F(w_0)-F^*\big)}{T} + \frac{G_{\text{kal}}^2 \sigma^2}{S}\left(1-\frac{S}{N}\right)\right),
$$
where $S=|S_t|$. The first term is the standard $\mathcal{O}(1/T)$ optimization term, while the second is an irreducible variance term that vanishes when $S=N$.

Two lemmas isolate the role of partial participation and Kalman fusion. For random client subsets without replacement,
$$
\mathbb{E}\big[\|\nabla F_{S_t}(w_t) - \nabla F(w_t)\|^2\big] \le \frac{\sigma^2}{S}\left(1 - \frac{S}{N}\right),
$$
which quantifies the sampling variance associated with period drift. The momentum update obeys
$$
\mathbb{E}\big[\|M_{t+1}-M_t\|^2\big] \le G_{\text{kal}}^2\!\left(\frac{\sigma^2}{S}\Big(1-\frac{S}{N}\Big) + \mathbb{E}\big[\|\nabla F(w_t) - M_t\|^2\big]\right),
$$
showing that the Kalman gain controls variance arising from both sampling noise and estimation error. The proof sketch reported in the paper uses a descent lemma from smoothness, the unbiasedness of the fused estimator, and telescoping over $T$ rounds [2508.14539].

## 5. Empirical behavior in cross-device settings

The reported evaluation spans computer vision and recommendation tasks. On **FEMNIST (LEAF)**, the setup uses 3400 users, 62 classes, and a LeNet-5 model, with both the natural partition by user and Dirichlet label-skew partitions with $\alpha\in\{1,0.1,0.01\}$. On **CIFAR-100**, the model is ResNet-18 with GroupNorm and the same Dirichlet choices. On **MovieLens-1M (CTR)**, the setup uses 6,040 users, a DIN model, leave-one-out evaluation, and AUC/HR/NDCG metrics. The cross-device configuration uses $T=1500$ rounds and $|S_t|=10$ clients per round for the computer-vision tasks, and $T=1000$ rounds with $|S_t|=20$ for the recommendation task. Local training uses $E=1$ epoch, local learning rate $\eta_l=0.01$, and server global learning rate $\eta_g=1$ for FedEve [2508.14539].

The baseline set comprises FedAvg, FedAvgM, FedProx, SCAFFOLD, and FedOpt. Across natural and Dirichlet partitions on FEMNIST and CIFAR-100, FedEve is reported as consistently best or tied for best, with improvements more pronounced at higher heterogeneity, especially at $\alpha=0.01$. On MovieLens-1M, the method achieves the best AUC, HR@10, and NDCG@10 among the compared approaches.

| Benchmark setting | Reported alternatives | FedEve |
|---|---|---|
| FEMNIST, $\alpha=0.01$ | FedAvg 73.23±1.36; FedAvgM 74.96±2.34; FedProx 74.16±1.19; SCAFFOLD 5.13±0.00; FedOpt 5.13±0.00 | 75.99±1.61 |
| CIFAR-100, $\alpha=0.01$ | FedAvg 30.11±0.53; FedAvgM 31.59±0.98; FedProx 30.10±0.55; FedOpt 32.17±1.38 | 32.68±0.62 |
| MovieLens-1M | Outperforming FedOpt and FedProx | AUC 0.7967±0.0016; HR@10 0.4460±0.0088; NDCG@10 0.2407±0.0037 |

The ablation results vary local epochs over $E\in\{1,3,5\}$ and heterogeneity over different $\alpha$ values. FedEve remains stable and top-performing, while methods that address only client drift can fail under strong period drift; the reported examples are SCAFFOLD and FedOpt failures in FEMNIST natural and $\alpha=0.01$ conditions. Visualizations are said to demonstrate the variability of period drift across rounds and its dominant effect when heterogeneity is high and participation is low. The Kalman gain analysis shows that $G_{\text{kal}}$ increases as heterogeneity increases, thereby shifting weight toward observation when prediction-observation mismatch grows, and that $G_{\text{kal}}$ is relatively insensitive to $E$ under cross-device constraints. The method’s overhead is described as negligible relative to FedAvg and FedAvgM because it requires only simple variance estimates and a scalar Kalman gain on the server, with no extra client storage or communication [2508.14539].

## 6. Relation to prior work, practical guidance, and ambiguity of nomenclature

FedEve is situated among methods that each address different parts of the heterogeneity problem. FedAvg is the baseline averaging method and, in the paper’s characterization, suffers from both client drift and period drift without explicit correction. FedAvgM and FedOpt improve stability through server-side momentum or adaptive optimization, but do not explicitly model the distinction between prediction and observation, and FedOpt’s adaptivity can be misled when the optimization objective shifts across rounds. FedProx adds a proximal term to reduce deviation from $w_t$ in local objectives and thus targets client drift, but it does not address period drift and requires tuning a proximal coefficient. SCAFFOLD uses control variates to correct client drift, yet in cross-device FL those control variates can become stale because participation is infrequent; the paper states that this can undermine effectiveness and potentially cause instability when per-client data sizes differ substantially. MIME, MOON, FedNova, and related methods are described as targeting client-side heterogeneity, personalization, normalization, or representation alignment, whereas FedEve’s stated novelty is the explicit modeling and compensation of both period drift and client drift through predict-observe fusion with principled variance weighting and no extra client state or communication [2508.14539].

The practical guidance reported for deployment is correspondingly specific. Server-side momentum, described as Nesterov, is used to form $M_t$, and a typical momentum $\beta$ around $0.9$ is said to work well in practice. The paper recommends a mild global learning rate such as $\eta_g=1.0$, local learning rate $\eta_l=0.01$, and $E=1$ in cross-device settings. Larger $|S_t|$ reduces period drift, and monitoring $\sigma_{Q_t}^2$ and $\sigma_{R_t}^2$ provides a diagnostic of whether period drift or client drift is dominant. When $\sigma_{Q_t}^2$ is large, period drift dominates and FedEve correspondingly increases $G_{\text{kal}}$ to trust observations more.

The stated limitations follow directly from the assumptions used in the theory and method design. FedEve assumes random client sampling and Gaussian-like independent drift noises; severe violations, such as highly biased or correlated sampling over time, may reduce effectiveness. In extreme non-iid regimes with extremely small $|S_t|$, both forms of drift may remain large, so convergence may still be slow even if variance is reduced. The theoretical analysis does not cover adversarial or privacy-constrained noise shaping. The future directions listed in the paper include non-Gaussian noise models, time-correlated drifts, adaptive client sampling, personalized variants, and integration with secure aggregation and privacy accounting [2508.14539].

A separate source of potential confusion is terminological rather than algorithmic. The details accompanying the holistic FL evaluation framework **FedEval** describe “FedEve/FedEval” as a standardized evaluation solution spanning privacy, robustness, effectiveness, and efficiency [2011.09655]. By contrast, the 2025 cross-device FL paper uses **FedEve** specifically for a predict-observe optimization method that bridges client drift and period drift [2508.14539]. This suggests a nomenclatural ambiguity across sources: in the cross-device optimization literature, FedEve denotes a Kalman-filter-based training algorithm, whereas in the evaluation-framework context the name appears in alignment with FedEval.

Source: https://www.emergentmind.com/topics/fedeve