Asynchronous Federated Learning (AFL)

• Asynchronous Federated Learning (AFL) is a distributed training paradigm that updates global models immediately without waiting for slow client responses, reducing delays.
• It employs methodologies like staleness-aware weighting, buffered aggregation, and adaptive communication strategies to manage heterogeneity and participation imbalances.
• Empirical studies show AFL can reduce communication overhead by over 50% and cut training times by up to 55% while maintaining high model accuracy.

Asynchronous Federated Learning (AFL) is a federated optimization paradigm in which the global model is updated without a round-level synchronization barrier. Instead of waiting for all selected clients to finish local computation, the server incorporates each arriving client update immediately, or after a small buffer of arrivals fills, thereby mitigating the straggler problem and improving wall-clock efficiency on heterogeneous edge systems. The resulting design space is substantially broader than “FedAvg without waiting”: AFL must explicitly manage stale updates, client scheduling, participation imbalance, communication overhead, privacy, security, and, increasingly, decentralized or hierarchical execution models (Xu et al., 2021).

1. Core formulation and asynchronous update semantics

At the objective level, AFL inherits the standard federated optimization target

$$\min_{w\in\mathbb R^d} F(w)=\sum_{k=1}^K p_k F_k(w), \qquad p_k=\frac{n_k}{\sum_j n_j},$$

with client-local objectives $$F_k(w)=\mathbb E_{\xi\sim\mathcal D_k}[\ell(w;\xi)]$$ or equivalent finite-sum forms. What changes is the control flow. In synchronous FL, the server broadcasts $w_t$, waits for a cohort of local updates, and then aggregates. In AFL, each client may pull a stale model $w_{t-\tau_k}$, compute locally, and return an update that is consumed without waiting for slower peers (Zhou et al., 2021, Ma et al., 2023).

A common unified server-side expression is

$$w_{t+1} = w_t + \eta_t \sum_{k=1}^K \alpha_{k,t}\,\Delta w_{k,t-\tau_{k,t}},$$

where $\tau_{k,t}\ge 0$ denotes staleness and $\alpha_{k,t}$ is often a staleness-aware weight (Xu et al., 2021). In event-driven formulations, a single arrival from client $m$ can trigger

$w^{j+1}=\beta_j\,w^j+(1-\beta_j)\,w_i^m,$

with $j-i$ the delay of the received update (Ma et al., 2023). The raw local quantity may be represented either as a model $$F_k(w)=\mathbb E_{\xi\sim\mathcal D_k}[\ell(w;\xi)]$$0, a model difference $$F_k(w)=\mathbb E_{\xi\sim\mathcal D_k}[\ell(w;\xi)]$$1, or a stochastic gradient.

AFL is not limited to purely immediate aggregation. Buffered variants maintain a server-side buffer of the last $$F_k(w)=\mathbb E_{\xi\sim\mathcal D_k}[\ell(w;\xi)]$$2 or $$F_k(w)=\mathbb E_{\xi\sim\mathcal D_k}[\ell(w;\xi)]$$3 arrivals and aggregate only when the buffer is full, which trades off throughput against staleness. FedStaleWeight uses a buffered asynchronous setting with a server buffer $$F_k(w)=\mathbb E_{\xi\sim\mathcal D_k}[\ell(w;\xi)]$$4 and normalized per-update weights, while BASA uses a circular buffer of size $$F_k(w)=\mathbb E_{\xi\sim\mathcal D_k}[\ell(w;\xi)]$$5 for secure aggregation in cross-device AFL (Ma et al., 2024, Wang et al., 2024). DeFedAvg introduces client-side receive and send buffers so that clients continuously train on the latest available model in their receive buffer and overwrite their send buffer with the most recent local difference $$F_k(w)=\mathbb E_{\xi\sim\mathcal D_k}[\ell(w;\xi)]$$6 (Wang et al., 2024).

2. Staleness, heterogeneity, and participation imbalance

The defining difficulty of AFL is not merely delay, but the interaction between delay and client heterogeneity. In non-IID settings, a fast client contributes more often, so its local objective can become over-represented in the global trajectory. ACE formalizes this by decomposing the update error into sampling noise, participation bias, and delay error, with the $$F_k(w)=\mathbb E_{\xi\sim\mathcal D_k}[\ell(w;\xi)]$$7 interaction identified as the mathematical signature of “heterogeneity amplification” (Chang et al., 24 Nov 2025). In that analysis, the participation-bias term vanishes only when the server always aggregates the most recent cached gradient from all clients.

This has made fairness a first-class topic in AFL. FedStaleWeight treats buffered asynchronous aggregation as a mechanism-design problem: naively upweighting slower clients can create incentive issues because truly fast clients may falsely report slower speeds to increase their contribution to model training. Its weighting rule depends only on observed average staleness, with unnormalized weight

$$F_k(w)=\mathbb E_{\xi\sim\mathcal D_k}[\ell(w;\xi)]$$8

followed by normalization within the buffer (Ma et al., 2024). The central claim is not that slow clients should dominate, but that aggregation should be more equitable on a per-agent basis while remaining strategy-proof.

Several papers also argue that the standard “round-gap only” view of staleness is too coarse. FedPSA replaces pure version-gap weighting with a parameter-sensitivity-based view of model obsolescence. Each client computes a sensitivity vector from a calibration batch, compresses it by random projection, and the server weights buffered updates by a softmax over cosine similarities in the sketch space, with a dynamic momentum queue adjusting tolerance for stale information across training phases (Lu, 17 Feb 2026). This directly targets a common oversimplification in AFL: two updates with the same age need not be equally informative.

Communication overhead is a parallel source of instability. VAFL identifies communication as the main bottleneck in AFL and frames the central optimization problem as balancing model performance against communication cost. Its premise is that many asynchronous updates are low-value, so selective communication can be preferable to unconditional aggregation (Zhou et al., 2021).

3. Aggregation, scheduling, buffering, and compression strategies

One major line of work modifies aggregation weights to account for staleness, model quality, or client conditions. CSMAAFL proposes a time- and staleness-adaptive aggregation coefficient,

$$F_k(w)=\mathbb E_{\xi\sim\mathcal D_k}[\ell(w;\xi)]$$9

which increases the weight of fresher updates and down-weights highly stale ones. It couples this with client scheduling that prioritizes contenders whose last upload was longer ago and adjusts local epochs according to compute capacity so that slow clients do not starve (Ma et al., 2023). In vehicular AFL, a related design multiplies each local model by upload-delay and training-delay factors before exponential-moving-average aggregation at the RSU, explicitly tying aggregation to channel state, mobility, data size, and compute frequency (Wang et al., 2022).

A second line of work estimates the informational value of each update. VAFL defines a communication-value score

$w_t$0

and the server aggregates only those clients whose $w_t$1 is at least the average $w_t$2. This produces a “value-aware” asynchronous FedAvg that filters low-value communications rather than compressing tensors via quantization or sparsification (Zhou et al., 2021).

Buffered AFL has inspired more elaborate server-side selection. AFBS clusters clients by encrypted label-distribution sketches using random projection, partitions buffered gradients by cluster, scores each gradient by

$w_t$3

and discards low-value gradients before aggregation. The aim is to avoid the failure mode in semi-asynchronous methods where a full buffer contains many stale gradients and blindly averaging them harms training (Lu et al., 15 Jun 2025). FedStaleWeight, by contrast, keeps the full buffer but reweights updates toward clients with larger observed average staleness (Ma et al., 2024).

A further systems-oriented direction jointly optimizes local training frequency and communication compression. FedLuck models each device by local-compute time $w_t$4, communication time $w_t$5, local-update count $w_t$6, and compression rate $w_t$7, then minimizes the dominant convergence factor

$w_t$8

under box constraints on $w_t$9 and $w_{t-\tau_k}$0. The resulting policy adaptively chooses both local update frequency and top-$w_{t-\tau_k}$1 compression rate per device (Song et al., 2024).

4. Convergence theory and rate regimes

Theoretical analyses of AFL now cover convex, strongly convex, and smooth non-convex objectives, although under different assumptions. The survey literature reports that, under standard $w_{t-\tau_k}$2-smoothness, bounded variance, and bounded staleness assumptions, non-convex AFL admits

$w_{t-\tau_k}$3

while strongly convex objectives achieve $w_{t-\tau_k}$4 with diminishing step sizes (Xu et al., 2021).

CSMAAFL shows that if aggregation coefficients are chosen so that, over each cycle, the effective weights match the synchronous coefficients, then asynchronous training can recover the $w_{t-\tau_k}$5 convergence rate of FedAvg in the smooth convex setting. Its combined scheduling and adaptive-weight variant retains

$w_{t-\tau_k}$6

under smoothness, bounded variance, and infinite-client-visit assumptions (Ma et al., 2023).

FedStaleWeight provides an ergodic non-convex guarantee for buffered AFL under unbiased stochastic gradients, bounded local and global variance, bounded gradient norms, Lipschitz smoothness, and bounded staleness. The bound is expressed on

$w_{t-\tau_k}$7

and includes explicit terms for buffer size, worst-case staleness, and deviation of normalized staleness weights from uniform aggregation (Ma et al., 2024). A separate non-convex analysis with heterogeneous datasets studies both staleness-aware aggregation $w_{t-\tau_k}$8 and delay-aware local step sizes $w_{t-\tau_k}$9, obtaining $$w_{t+1} = w_t + \eta_t \sum_{k=1}^K \alpha_{k,t}\,\Delta w_{k,t-\tau_{k,t}},$$0 convergence of the expected gradient norm under appropriate scaling (Forootani et al., 3 Aug 2025).

DeFedAvg is notable for proving linear speedup. Under bounded delay, unbiased stochastic gradients, bounded variance, and, in the non-IID case, bounded heterogeneity and bounded client gradient norms, DeFedAvg-nIID and DeFedAvg-IID achieve

$$w_{t+1} = w_t + \eta_t \sum_{k=1}^K \alpha_{k,t}\,\Delta w_{k,t-\tau_{k,t}},$$1

and the authors characterize it as the first AFL algorithm with provable linear speedup (Wang et al., 2024).

In the strongly convex regime, a later martingale-based analysis assumes strongly convex local objective functions, bounded stochastic-gradient variance, bounded heterogeneity, and bounded staleness, and derives exponential decay of a weighted-average iterate up to asymptotic variance terms determined by $$w_{t+1} = w_t + \eta_t \sum_{k=1}^K \alpha_{k,t}\,\Delta w_{k,t-\tau_{k,t}},$$2, $$w_{t+1} = w_t + \eta_t \sum_{k=1}^K \alpha_{k,t}\,\Delta w_{k,t-\tau_{k,t}},$$3, and delay-induced drift (Forootani et al., 2024). Across these results, the common pattern is that AFL retains classical stochastic-optimization rates only when delay, heterogeneity, and local-drift terms are explicitly controlled.

5. Privacy, robustness, and architectural extensions

AFL’s asynchronous control flow complicates privacy accounting and secure aggregation. The differential-privacy line begins with the first convergence analysis of AFL under DP, where clipped local gradients are perturbed by Laplace or Gaussian noise and consumed asynchronously by a cloud server. MAPA, the multi-stage adjustable private algorithm, dynamically reduces the clipping threshold and adjusts the learning rate stage-wise so that the induced noise variance remains commensurate with stochastic variance, thereby improving the privacy-utility trade-off relative to single-stage private AFL (Li et al., 2019).

Secure aggregation in AFL requires breaking with round-synchronous cryptographic protocols. BASA is presented as the first secure aggregation protocol fully compatible with AFL in cross-device settings, with each user requiring one round of communication with the server and no synchronous interaction among users. It uses CP-ABE, pairwise PRG masks, and a server buffer, and preserves the same buffered AFL update rule as prior asynchronous methods while protecting individual updates from an honest-but-curious server (Wang et al., 2024).

Robustness against poisoning is a separate challenge because stale benign updates and malicious updates can both appear noisy. AFLGuard addresses Byzantine-robust asynchronous FL by filtering each received client gradient against a server-side gradient computed on a small trusted dataset. An update is accepted only if its deviation from the trusted gradient is within a tolerance proportional to the trusted gradient norm. Under smoothness, strong convexity, and trusted-data assumptions, AFLGuard proves convergence to a small neighborhood and empirically resists untargeted, targeted, and adaptive attacks (Fang et al., 2022).

Architecturally, AFL has expanded beyond a single server. DBAFL replaces a dedicated central server with a rotating committee leader among RSUs and uses a permissioned Hyperledger Fabric blockchain that stores only model hashes. Its dynamic scaling factor

$$w_{t+1} = w_t + \eta_t \sum_{k=1}^K \alpha_{k,t}\,\Delta w_{k,t-\tau_{k,t}},$$4

weights a stale local model by its measured accuracy relative to the current global model, while committee-based consensus is designed to improve reliability at low latency (Xu et al., 2022). DAG-AFL removes conventional blockchain bottlenecks by organizing updates as DAG tips and selecting parents using temporal freshness, reachability, and feature-signature similarity, with hash-linked trusted verification (Zhang et al., 28 Jul 2025). FedAH introduces asynchronous hierarchical FL, where cluster-level aggregators asynchronously absorb client updates and periodically forward summarized updates to the server, reducing server traffic and keeping client-server hops short (Wang et al., 2022).

6. Empirical behavior, application domains, and open research problems

Empirically, AFL methods show their largest gains under heterogeneous compute, heterogeneous communication, and non-IID data. VAFL reports a 51.02% reduction in number of communications and a 48.23% average communication-compression rate across four experimental settings, while maintaining or improving convergence speed (Zhou et al., 2021). CSMAAFL attains the same final test accuracy as synchronous FedAvg and converges $$w_{t+1} = w_t + \eta_t \sum_{k=1}^K \alpha_{k,t}\,\Delta w_{k,t-\tau_{k,t}},$$5–$$w_{t+1} = w_t + \eta_t \sum_{k=1}^K \alpha_{k,t}\,\Delta w_{k,t-\tau_{k,t}},$$6 faster in wall-clock time, with MNIST non-IID reaching $$w_{t+1} = w_t + \eta_t \sum_{k=1}^K \alpha_{k,t}\,\Delta w_{k,t-\tau_{k,t}},$$7 accuracy in roughly 20 time slots versus 40 for FedAvg (Ma et al., 2023). FedLuck reports, over three tasks, a 55% reduction in training time and a 56% reduction in total uplink data by jointly optimizing local update frequency and gradient compression (Song et al., 2024).

Several recent methods specifically target the stale-update accuracy gap. FedRevive combines parameter-space aggregation with server-side data-free knowledge distillation and reports up to 32.1% faster training and up to 21.5% higher final accuracy compared to asynchronous baselines across vision and text benchmarks (Askin et al., 1 Nov 2025). AFBS improves accuracy by up to 4.8% over the previous best algorithm on CIFAR-100 and reduces the time to reach target accuracy by 75% through privacy-preserving cluster-aware gradient selection inside the buffer (Lu et al., 15 Jun 2025). FedPSA reports up to 6.37% improvement over baseline methods and 1.93% over the current state-of-the-art method by replacing round-gap staleness with sensitivity-based behavioral similarity and a dynamic momentum queue (Lu, 17 Feb 2026).

AFL is already embedded in domain-specific systems. In smart public transportation, DBAFL targets buses and RSUs with asynchronous aggregation, committee-based blockchain consensus, and dynamic stale-model weighting (Xu et al., 2022). In edge-assisted vehicular networks, AFL is motivated by the possibility that vehicles may leave RSU coverage before synchronous training completes, and the proposed mobility-aware scheme jointly considers data size, compute capability, and vehicle mobility (Wang et al., 2022). A later case study applies AFL to climate-aware wind power forecasting over Germany using CMIP6 data and a 3-layer LSTM, reporting stable convergence and shortened round times under heterogeneous delays (Forootani et al., 2024).

The open problems remain substantial. The survey literature highlights personalized AFL, vertical AFL, fine-grained differential-privacy accounting under irregular updates, efficient secure consensus for edge environments, and standardized heterogeneous testbeds as major gaps (Xu et al., 2021). Robustness work still leaves non-convex Byzantine theory largely open (Fang et al., 2022). Secure-aggregation work calls for malicious-client and malicious-server extensions beyond the honest-but-curious model (Wang et al., 2024). More recent staleness research suggests that the field is moving from coarse round-based delay penalties toward behavior-aware, fairness-aware, and information-aware aggregation, but a unified theory covering non-IID data, extreme asynchrony, buffering, selection, compression, and security remains incomplete.