DySTop: Asynchronous Decentralized Federated Learning

• DySTop is an asynchronous decentralized federated learning mechanism that jointly optimizes worker activation and neighbor selection to control staleness and communication overhead.
• It leverages a Lyapunov drift-plus-penalty framework to balance training speed and model accuracy by tuning staleness bounds and activation frequency in heterogeneous edge environments.
• Experimental evaluations show DySTop reduces completion time by up to 81% and communication overhead by 57% compared to state-of-the-art methods while maintaining model accuracy.

Searching arXiv for the DySTop paper and closely related decentralized federated learning work. DySTop is an asynchronous decentralized federated learning mechanism for heterogeneous edge environments that jointly optimizes dynamic staleness control and topology construction. It is designed for settings in which devices are slow, links are unstable, and local datasets are non-IID, so that naïve asynchronous communication yields stale aggregation, frequent model transmission, and degraded training behavior. In DySTop, each round activates multiple workers, selects a subset of their neighbors for model transmission and aggregation, and then performs local training; the method’s central claim is that worker activation and neighbor selection are coupled control variables rather than separable design choices (Shi et al., 4 Aug 2025).

1. Problem formulation and design motivation

DySTop is situated in the transition from centralized federated learning to decentralized federated learning. In centralized FL, all workers send updates to a parameter server, which creates a scalability bottleneck and a single point of failure. Synchronous DFL removes the server but forces every worker to wait for all neighbors before aggregation, which creates severe straggler effects in heterogeneous edge networks and is brittle under dynamic connectivity. Asynchronous DFL avoids global synchronization, but prior mechanisms still suffer from staleness accumulation, excessive communication, and weak handling of non-IID data (Shi et al., 4 Aug 2025).

The mechanism therefore targets two coupled failure modes. The first is stale model aggregation: workers may incorporate outdated models from slow neighbors, and the resulting staleness harms convergence. The second is communication inefficiency: exchanging models with all neighbors is costly and often unnecessary. DySTop treats both as a single optimization problem. At each round it chooses which workers should be activated to control staleness and which neighbor links should be used so that communication remains sparse while data heterogeneity is mitigated.

The model used by the paper associates worker $v_i$ with a local model $\mathbf{w}_t^i$ and a staleness variable $\tau_t^i$, where the model available at round $t$ is

$\mathbf{w}_t^i = \mathbf{w}_{t-\tau_t^i}^i.$

This definition is central: DySTop is not merely asynchronous message passing, but explicit round-by-round control over the age of exchanged models.

2. Round-by-round system operation

DySTop operates in rounds under a coordinator that gathers worker status, determines the active set $\mathcal{A}_t$, constructs a round-specific topology $\mathcal{G}_t$, and sends EXECUTE messages to workers in $\mathcal{A}_t$ (Shi et al., 4 Aug 2025). The worker status used at the coordinator includes bandwidth, current staleness, data distribution, and training times.

The staleness state evolves according to

$\tau_{t+1}^i = (\tau_t^i + 1)(1-a_t^i),$

where $a_t^i = 1$ if worker $\mathbf{w}_t^i$0 is activated and $\mathbf{w}_t^i$1 otherwise. Hence active workers reset staleness to $\mathbf{w}_t^i$2, whereas inactive workers accumulate one unit of staleness.

When an active worker $\mathbf{w}_t^i$3 receives EXECUTE and has completed local training, it sends PULL messages to its in-neighbors $\mathbf{w}_t^i$4, receives their models, aggregates them with data-size weights, and then performs local SGD. The aggregation rule is

$\mathbf{w}_t^i$5

with

$\mathbf{w}_t^i$6

The local training step is

$\mathbf{w}_t^i$7

Workers also maintain a pushing thread: whenever a worker receives a PULL message from one of its out-neighbors, it pushes its current model. This pull-based asynchronous behavior avoids unnecessary broadcast and makes topology selection operationally meaningful rather than merely descriptive.

3. Objective function and convergence analysis

DySTop defines the global weighted model as

$\mathbf{w}_t^i$8

Its optimization problem $\mathbf{w}_t^i$9 is to minimize total completion time across rounds while satisfying accuracy, staleness, bandwidth, and binary decision constraints:

$\tau_t^i$0

subject to

$\tau_t^i$1

The round duration is determined by the slowest active worker:

$\tau_t^i$2

with

$\tau_t^i$3

The convergence analysis assumes that each local loss $\tau_t^i$4 is $\tau_t^i$5-smooth and $\tau_t^i$6-strongly convex. It characterizes data heterogeneity using gradient divergence,

$\tau_t^i$7

and local gradient noise at the local optimum,

$\tau_t^i$8

With learning rate $\tau_t^i$9, the one-round local update satisfies

$t$0

where

$t$1

The global convergence theorem then gives

$t$2

making explicit the dependence on maximum staleness $t$3, activating frequency $t$4, and data heterogeneity through $t$5 and $t$6 (Shi et al., 4 Aug 2025). The paper derives three direct consequences: decreasing $t$7 improves the convergence bound, increasing $t$8 improves the bound, and more non-IID data worsens it.

4. Lyapunov reformulation and the two control algorithms

To operationalize $t$9, DySTop introduces a staleness queue

$\mathbf{w}_t^i = \mathbf{w}_{t-\tau_t^i}^i.$0

which converts the staleness constraint into a queue-stability problem. Using Lyapunov drift-plus-penalty, the per-round subproblem $\mathbf{w}_t^i = \mathbf{w}_{t-\tau_t^i}^i.$1 becomes

$\mathbf{w}_t^i = \mathbf{w}_{t-\tau_t^i}^i.$2

The first term penalizes staleness violations, the second penalizes round duration, and $\mathbf{w}_t^i = \mathbf{w}_{t-\tau_t^i}^i.$3 governs the trade-off. The corresponding optimality-gap statement is

$\mathbf{w}_t^i = \mathbf{w}_{t-\tau_t^i}^i.$4

From this reformulation, DySTop derives two algorithms.

The Worker Activation Algorithm (WAA) solves the activation component of $\mathbf{w}_t^i = \mathbf{w}_{t-\tau_t^i}^i.$5. It takes as input the current staleness values $\mathbf{w}_t^i = \mathbf{w}_{t-\tau_t^i}^i.$6, queue backlogs $\mathbf{w}_t^i = \mathbf{w}_{t-\tau_t^i}^i.$7, and per-worker round cost $\mathbf{w}_t^i = \mathbf{w}_{t-\tau_t^i}^i.$8. It begins with an empty active set, sorts workers by ascending $\mathbf{w}_t^i = \mathbf{w}_{t-\tau_t^i}^i.$9, adds workers one by one, computes the $\mathcal{A}_t$0 objective for each partial active set, and retains the set with the smallest objective value. This procedure is intended to balance staleness control against short rounds: activating a worker resets its staleness, but activating too many slow workers lengthens the round.

The Phase-Aware Topology Construction Algorithm (PTCA) solves the topology component of $\mathcal{A}_t$1. It uses the active set $\mathcal{A}_t$2, communication-range candidate sets $\mathcal{A}_t$3, bandwidth budgets $\mathcal{A}_t$4, and the current training phase. In the early phase, it emphasizes non-IID mitigation by preferring neighbors with very different data distributions, measured by Earth Mover’s Distance:

$\mathcal{A}_t$5

The early-phase priority is

$\mathcal{A}_t$6

In the later phase, PTCA shifts focus toward freshness and diversity of exchange. Its priority becomes

$\mathcal{A}_t$7

For each active worker, PTCA sorts candidate neighbors by the phase-specific priority, adds neighbors greedily while bandwidth permits, updates in-neighbor and out-neighbor sets, and stops when bandwidth consumption stops changing. The resulting topology is sparse, budget-aware, and phase-adaptive (Shi et al., 4 Aug 2025).

5. Experimental evaluation

The paper evaluates DySTop in both simulation and a real testbed (Shi et al., 4 Aug 2025). The simulation uses 100 heterogeneous workers in a $\mathcal{A}_t$8 area on a workstation with an Intel Xeon CPU and 4 RTX 3090Ti GPUs. The datasets are FMNIST and CIFAR-10; the models are CNN and ResNet-18; non-IID partitions are generated by a Dirichlet distribution with $\mathcal{A}_t$9. The baselines are MATCHA, AsyDFL, and SA-ADFL, and the reported metrics are test accuracy, training loss, communication overhead, and completion time.

The real testbed consists of 1 coordinator and 15 workers using Jetson Nano, Jetson Orin Nano, Jetson Orin NX, Jetson Orin, and Jetson Xavier AGX devices, with a Dell T640 server as coordinator. The datasets and models are SVHN with SqueezeNet and CIFAR-100 with MobileNet-V2, under non-IID levels $\mathcal{G}_t$0 and $\mathcal{G}_t$1.

The headline result is that DySTop reduces completion time by 51.8% and communication resource consumption by 57.1% compared to state-of-the-art solutions while maintaining the same model accuracy. In the FMNIST simulation with CNN under high non-IID ($\mathcal{G}_t$2), DySTop achieves a completion time of 80.16 s, compared with 349.27 s for AsyDFL, 166.35 s for SA-ADFL, and 422.76 s for MATCHA. This corresponds to a 77.04% reduction versus AsyDFL, a 51.81% reduction versus SA-ADFL, and an 81.03% reduction versus MATCHA. To reach 80% accuracy on CNN/FMNIST in IID settings, DySTop uses 58.17% less communication than AsyDFL and 38.94% less communication than SA-ADFL.

The paper also studies sensitivity to the staleness bound and to the Lyapunov parameter $\mathcal{G}_t$3. A too-small $\mathcal{G}_t$4 makes the system too synchronous and harms efficiency; a too-large $\mathcal{G}_t$5 allows excessive staleness and harms accuracy. A moderate value such as $\mathcal{G}_t$6 gives the best trade-off. For $\mathcal{G}_t$7, small values strengthen staleness control but may slow progress, whereas large values favor faster rounds but weaken control and may worsen accuracy; the reported comparisons use $\mathcal{G}_t$8.

On the real testbed, for SqueezeNet on SVHN in the IID setting, the time to reach 80% accuracy is 721 s for DySTop, compared with 1079 s for AsyDFL, 1516 s for SA-ADFL, and 1865 s for MATCHA. These correspond to reductions of 33.17%, 52.44%, and 61.34%, respectively. The paper further notes that MATCHA performs worse on the testbed than in simulation because device heterogeneity is stronger there, making synchronous waiting particularly harmful.

6. Interpretation, significance, and limitations

DySTop’s conceptual contribution is to reframe asynchronous decentralized federated learning as a joint control problem rather than a pure communication problem. The mechanism does not assume that asynchrony alone resolves heterogeneity; instead, it argues that convergence quality is jointly shaped by maximum staleness, worker activation frequency, and data heterogeneity, and that these factors must be reflected in both activation and topology decisions (Shi et al., 4 Aug 2025). A plausible implication is that DySTop is best understood as a scheduling-and-communication policy layered on top of decentralized optimization, rather than as a new local training rule.

The paper also corrects a common simplification in asynchronous DFL: sparse communication is not automatically beneficial unless the chosen links are aligned with the training phase. PTCA therefore uses statistically diverse and physically close neighbors early in training to mitigate non-IID bias, then shifts toward less-pulled neighbors with similar staleness later in training to preserve freshness and stable progress. This suggests a phase-dependent view of topology as an optimization variable rather than a fixed network property.

Several limitations are explicit in the formulation and experiments. DySTop depends on a coordinator that gathers worker status and constructs the round-specific active set and topology. Its convergence theory is derived under $\mathcal{G}_t$9-smoothness and $\mathcal{A}_t$0-strong convexity assumptions. Its performance depends on tuning $\mathcal{A}_t$1, $\mathcal{A}_t$2, and the number of selected neighbors. The experiments also show diminishing returns from increasing the neighbor count: accuracy improves, but communication overhead rises, and the gains flatten after a moderate number of neighbors.

Within the asynchronous DFL literature, DySTop’s distinctiveness lies in combining theoretical dependence on staleness and heterogeneity with two concrete algorithms, WAA and PTCA, that translate those dependencies into per-round decisions. In the formulation presented by the paper, the essential claim is that selecting the right workers to activate and the right neighbors to communicate with yields substantial reductions in completion time and communication overhead without sacrificing model accuracy (Shi et al., 4 Aug 2025).