FedProx Algorithm Overview

• FedProx is a federated optimization algorithm that adds a proximal term to each client’s local objective to mitigate client drift under heterogeneity.
• It modifies FedAvg by constraining client updates without altering server aggregation, which enhances stability in non-IID and partially participating scenarios.
• Variants like FedExProx and FedProx+ adapt the proximal coefficient to optimize convergence and performance across a range of applications.

FedProx is a federated optimization algorithm that generalizes and re-parameterizes Federated Averaging (FedAvg) by replacing each client’s unconstrained local objective with a proximal-regularized subproblem centered at the current global model. Its defining purpose is to stabilize federated training under the two heterogeneities emphasized in federated learning: statistical heterogeneity, in which client data are non-IID, and systems heterogeneity, in which clients perform variable amounts of local work or exhibit straggling behavior. In the canonical formulation, these modifications are intentionally minimal at the server side—aggregation remains averaging—while materially changing the client-side optimization geometry (Li et al., 2018).

1. Formal definition and relation to FedAvg

Let $K$ denote the number of clients, $F_k(w)$ the local objective on client $k$, and $p_k \ge 0$ aggregation weights with $\sum_k p_k = 1$. The standard federated objective is

$F(w) = \sum_{k=1}^{K} p_k F_k(w).$

FedProx modifies the local problem solved at round $t$ by client $k$ to

$$\min_{w}\; h_k(w; w^t) = F_k(w) + \frac{\mu}{2}\,\|w - w^{t}\|^2,$$

where $w^t$ is the global model broadcast by the server and $F_k(w)$0 is the proximal coefficient. A common first-order instantiation is

$F_k(w)$1

followed by server aggregation

$F_k(w)$2

with $F_k(w)$3 the participating clients in round $F_k(w)$4 (Kuili et al., 25 Jan 2025).

The critical algorithmic distinction from FedAvg is therefore local, not global. FedAvg and FedProx share the same aggregation rule; they differ in the client-side objective and update. FedAvg is recovered exactly when $F_k(w)$5 (Amgain et al., 2024). This equivalence is important both conceptually and practically: FedProx is not a different federation protocol so much as a proximal perturbation of standard local training.

In implementations, the proximal term is added to whatever local task loss is already used. The surveyed applications instantiate this with cross-entropy, binary cross-entropy, Dice-loss-based objectives, or logistic loss, and optimize locally with SGD or Adam, but the canonical abstraction is independent of the specific solver (Kuili et al., 25 Jan 2025).

2. Heterogeneity, client drift, and the original convergence framework

FedProx was introduced to address a failure mode of local training under heterogeneity: client drift. When clients minimize only $F_k(w)$6, their local iterates can move toward client-specific optima that are poorly aligned with the global objective, especially when local datasets are non-IID or when clients complete different amounts of work before synchronization. The proximal term constrains this movement by tethering local optimization to the current global iterate (Li et al., 2018).

The original framework explicitly models both statistical and systems heterogeneity. Statistical heterogeneity is captured through non-identical local objectives and a bounded dissimilarity condition. Systems heterogeneity is handled by allowing variable inexactness in local solves. In the original analysis, client $F_k(w)$7 returns a $F_k(w)$8-inexact solution $F_k(w)$9 satisfying

$k$0

A smaller $k$1 corresponds to more accurate local optimization and, operationally, to more local work. This formulation makes partial work a first-class object rather than an implementation nuisance (Li et al., 2018).

The same work formalizes heterogeneity through $k$2-local dissimilarity, with

$k$3

outside a small-gradient region. Under $k$4-smoothness, a lower Hessian bound $k$5, and $k$6, the analysis derives an expected decrease bound for the global objective. Sufficient—though not necessary—conditions include $k$7 and $k$8, making explicit the interaction among local inexactness, heterogeneity, and participation level (Li et al., 2018).

This construction yields a particular interpretation of FedProx: it is neither merely a regularizer nor merely a robustness heuristic. It is a proximal-point style federated method whose local subproblem is designed to remain well behaved even when local computation budgets differ and local objectives are misaligned.

3. Later theory: beyond bounded dissimilarity and beyond stationarity

Subsequent theory substantially broadened the analytical picture. One line of work shows that favorable FedProx complexity bounds do not fundamentally require the local gradient dissimilarity assumptions used in early analyses. Through an algorithmic-stability argument, later results establish convergence guarantees for smooth and non-smooth federated problems without assuming local dissimilarity conditions, and derive linear speedup in the smooth minibatch setting with respect to minibatch size and the number of sampled devices (Yuan et al., 2022).

In the smooth case, the minibatch stochastic proximal extension studied there attains rates of the form

$k$9

where $p_k \ge 0$0 is the number of rounds, $p_k \ge 0$1 the local minibatch size, and $p_k \ge 0$2 the number of sampled devices. In the non-smooth weakly convex setting, the analysis proceeds through Moreau-envelope stationarity rather than ordinary gradient stationarity (Yuan et al., 2022). A central consequence is that local dissimilarity and smoothness are not treated as indispensable prerequisites for meaningful FedProx guarantees.

A different theoretical development, cast in RKHS non-parametric regression, reaches an apparently paradoxical conclusion: FedAvg and FedProx may fail to reach stationary points of the global empirical objective even for homogeneous linear regression, yet still achieve strong statistical rates. In that analysis, the empirical and RKHS estimation errors decay as $p_k \ge 0$3 in general and exponentially for finite-rank kernels, and the authors introduce “federation gain” as the reduction in estimation error a client obtains by participating in federated learning rather than training locally (Su et al., 2021). This sharply separates optimization stationarity from statistical efficiency.

Taken together, these later analyses reposition FedProx from a method justified only by bounded-dissimilarity non-IID theory to one understood through multiple lenses: proximal-point stability, weakly convex analysis, statistical efficiency, and client-level benefit under heterogeneity.

4. Variants, extrapolation, and adaptive proximal weighting

FedProx has also served as the base algorithm for several extensions. The most mathematically explicit is server-side extrapolation. FedExProx updates

$p_k \ge 0$4

which is equivalent to gradient descent on a Moreau-envelope surrogate with stepsize $p_k \ge 0$5. In the exact-prox, smooth convex interpolation regime, the optimal extrapolation parameter is reported as $p_k \ge 0$6, and the theory yields improved constants over vanilla FedProx (Li et al., 2024).

A later analysis studies FedExProx with inexact proximal solutions. Under an absolute inexactness model, the method converges linearly only to a neighborhood of the optimum. Under a relative inexactness model, it retains exact linear convergence provided the inexactness is controlled; in the sharper biased-compression analysis, this requires $p_k \ge 0$7 while preserving the optimal extrapolation choice $p_k \ge 0$8 (Li et al., 2024). This gives a precise statement of how imperfect local prox computation degrades, but need not destroy, the benefits of extrapolation.

Task-specific variants modify the proximal coefficient itself. In wireless signal enhancement and recognition, FedProx+ replaces the global $p_k \ge 0$9 with client-specific $\sum_k p_k = 1$0 values and updates them through an unspecified AdaptiveMu rule driven by local and global performance metrics; the local objective becomes

$\sum_k p_k = 1$1

The exact analytic form of AdaptiveMu is not given, but the method is described as increasing or decreasing $\sum_k p_k = 1$2 according to the ratio of local to global performance (Zhang et al., 19 Sep 2025).

Another extension, G-FedProx, augments FedProx with Gaussian normalization of client updates. It filters or downweights updates outside the interval $\sum_k p_k = 1$3, where $\sum_k p_k = 1$4 and $\sum_k p_k = 1$5 are server-side empirical statistics, and reports “approximately 90% better convergence” in proxy experiments on regression statistics rather than end-to-end federated neural training (Sarkar et al., 2024).

These variants are best understood as descendants of the canonical FedProx idea rather than redefinitions of it. What remains invariant is the proximal anchoring of local updates to a server-provided reference model.

5. Empirical record across domains

Across the surveyed literature, FedProx is most consistently advantageous in non-IID or partially participating settings; under IID partitions, its effect is often small. The following studies illustrate the range of reported behavior.

Domain	Setup	Reported FedProx-related outcome
Foundational heterogeneous benchmarks (Li et al., 2018)	Synthetic, MNIST, FEMNIST, Shakespeare, Sent140; high heterogeneity and stragglers	FedProx improved absolute test accuracy by 22% on average in highly heterogeneous settings
5G jamming detection (Kuili et al., 25 Jan 2025)	Two-stage framework; supervised stage uses FedProx with $\sum_k p_k = 1$6	With 6 clients: precision 0.94, recall 0.90, F1-score 0.92, accuracy 0.92; robust convergence within 30 rounds
OCT image classification (Amgain et al., 2024)	10 clients; label-skew via 2SPC and 3SPC	2SPC: 90.57 ± 3.09 vs 87.22 ± 5.97 for FedAvg; 3SPC: 91.01 ± 2.51 vs 88.32 ± 2.30
Heart disease prediction (Asad et al., 23 Jan 2026)	4 heterogeneous hospital clients; logistic regression	$\sum_k p_k = 1$7: 85.00% accuracy vs 84.58% for FedAvg and 83.33% centralized; 18 vs 22 rounds to 95% of final accuracy
Mortality prediction on MIMIC-IV (Tertulino, 3 Sep 2025)	5 care-unit clients; SMOTE-Tomek preprocessing	FedProx achieved F1-score 0.8831, accuracy 0.9062, precision 0.8857, recall 0.8812, with stable convergence
Ultrasound breast cancer segmentation (Gad et al., 21 Oct 2025)	3 non-IID clients; Attention U-Net with FedProx	Round 6: accuracy 0.9607, IoU 0.5494, F1-score 0.7076, specificity 0.9919

The medical and wireless studies exhibit two recurrent patterns. First, under stronger heterogeneity, FedProx frequently improves robustness rather than merely the final mean metric. In the OCT study, this is reflected not only in higher mean accuracy under 2SPC and 3SPC label skew, but also in reduced variance in the more heterogeneous setting (Amgain et al., 2024). Second, the strongest results often come from composite pipelines in which FedProx is one stabilizing component among others, such as a frozen pretrained encoder in a two-stage federated classifier or class-balancing via SMOTE-Tomek before federated optimization (Kuili et al., 25 Jan 2025).

This suggests that FedProx is often most effective when used to control optimization instability created by heterogeneity, rather than as a stand-alone remedy for every pathology in federated learning.

6. Hyperparameters, misconceptions, and limitations

The proximal coefficient $\sum_k p_k = 1$8 is the central FedProx hyperparameter, and the surveyed studies show that its useful range is task dependent. Reported values include $\sum_k p_k = 1$9 in 5G jamming detection, $F(w) = \sum_{k=1}^{K} p_k F_k(w).$0 in OCT classification, and $F(w) = \sum_{k=1}^{K} p_k F_k(w).$1 as the best value in a heart-disease simulation after a grid search over $F(w) = \sum_{k=1}^{K} p_k F_k(w).$2 (Kuili et al., 25 Jan 2025, Amgain et al., 2024, Asad et al., 23 Jan 2026). In the large-scale Flower-based IoT study, $F(w) = \sum_{k=1}^{K} p_k F_k(w).$3 and $F(w) = \sum_{k=1}^{K} p_k F_k(w).$4 were explored under extreme straggler rates, with $F(w) = \sum_{k=1}^{K} p_k F_k(w).$5 reported to exhibit more stable convergence in those simulations (Zahri et al., 2023).

A consistent finding is that $F(w) = \sum_{k=1}^{K} p_k F_k(w).$6 mediates a stability–progress trade-off. Too small a value yields behavior close to FedAvg; too large a value over-constrains local updates. The heart-disease study makes this explicit: $F(w) = \sum_{k=1}^{K} p_k F_k(w).$7 achieved 85.00% accuracy, whereas $F(w) = \sum_{k=1}^{K} p_k F_k(w).$8 fell to 81.25% despite reaching 95% of final accuracy in fewer rounds (Asad et al., 23 Jan 2026). Several application papers also note that they do not provide a full $F(w) = \sum_{k=1}^{K} p_k F_k(w).$9-sensitivity sweep, so reported values should be treated as tuned task-specific operating points rather than defaults (Amgain et al., 2024).

Several misconceptions recur in discussions of FedProx. One is that it changes server aggregation; in fact, canonical FedProx keeps the FedAvg-style averaging step and alters only the local client objective (Amgain et al., 2024). Another is that FedProx should always outperform FedAvg. The OCT study reports that under IID partitions both methods are “on par” with centralized training and that the proximal term has negligible effect in that regime (Amgain et al., 2024). A third is that FedProx itself provides formal privacy guarantees. It preserves data locality, but differential privacy is a separate mechanism. When DP is added, accuracy can decline materially: one IoT study reports FedProx accuracy up to 0.9146 at round 20 without DP and 0.8353 with DP_FedProx, while a clinical DP pipeline identifies a privacy–utility frontier and shows FedProx outperforming FedAvg at comparable privacy levels rather than eliminating the privacy–utility trade-off (Zahri et al., 2023, Tertulino, 6 Aug 2025).

Finally, FedProx is not a universal cure for federation pathologies. It does not by itself solve extreme label absence, class imbalance, poisoned updates, or the absence of formal secure aggregation. What it does provide is a precise proximal control mechanism for local drift. In the literature surveyed here, that mechanism is most valuable when heterogeneity is genuine, participation is partial or uneven, and local optimization would otherwise diverge, oscillate, or overfit to silo-specific structure.