Asynchronous Message Updates

• Asynchronous message updates are mechanisms that allow independent progress by processing delayed, partial, or reordered information without a global barrier.
• They incorporate adaptive scheduling and corrective weighting to ensure convergence despite challenges like staleness and variable participation.
• Applications span belief propagation, federated learning, decentralized SGD, distributed filesystems, and graph neural networks, leading to efficiency improvements.

Asynchronous message updates denote a broad class of mechanisms in which message production, propagation, and incorporation proceed without a single global lockstep, so that senders, workers, or nodes can advance at different rates and recipients integrate delayed, partial, or reordered information later. In the literature, this notion spans asynchronous coordinate updates in belief propagation, partial-neighbor averaging in decentralized SGD, stale client updates in federated learning, pairwise gossip dissemination in consensus, deferred metadata materialization in distributed filesystems, runtime send/receive splitting in choreographies, and selective node updates in graph neural networks (Elidan et al., 2012, Xiong et al., 2023, Cruz-Filipe et al., 2017, Xu et al., 2024, Bose et al., 8 Sep 2025).

1. Multiple meanings of asynchrony

The term is not uniform across domains. In fixed-point inference, asynchronous message updates mean that only one message coordinate is updated at a time, replacing the global synchronous operator

$$f^s(v_1,\dots,v_{|M|}) = \bigl(f_1(v),\dots,f_{|M|}(v)\bigr)$$

with coordinate-wise updates

$$f_m^a(v_1,\dots,v_{|M|}) = (v_1,\dots,f_m(v),\dots,v_{|M|}),$$

so later updates can immediately use freshly updated messages rather than stale previous-round values (Elidan et al., 2012).

In choreography semantics, asynchrony is defined by two properties: messages can be sent without the intended receiver being ready, and all sent messages are eventually received. This is realized by splitting a communication such as $p.e \to q$ into runtime send and receive pieces, so that sender-side progress can be reordered ahead of receiver-side consumption (Cruz-Filipe et al., 2017).

In wireless random access, by contrast, asynchrony refers to the signal model rather than the execution schedule of the inference algorithm. Users are synchronous at symbol level but asynchronous at frame level, with an unknown integer delay

$t_n \in \{0,1,\dots,D\},$

and inference must jointly recover activity, delay, and channel state (Zhu et al., 2021).

Collaborative perception uses yet another notion: multi-source fusion is asynchronous when the latest updates from different sources may have different generation times, but fusion is admissible only if the generation-time gap remains within a threshold $T_n<d_n$. The age process resets only when this temporal alignment condition is met (Wang et al., 2023).

This diversity matters because a common misconception is that “asynchronous” always means wait-free computation. In the cited literature it may instead mean delayed visibility, frame-level timing mismatch, non-blocking sends, or selective participation under a logical rather than wall-clock iteration counter.

2. Scheduling, delay control, and partial participation

In asynchronous belief propagation, the central theoretical question is whether scheduling changes convergence. Under a max-norm contraction assumption and a minimal fairness condition stating that every message is updated infinitely often, any asynchronous propagation schedule converges to the unique fixed point. The same analysis shows that round-robin asynchronous BP has a worst-case contraction factor $\rho_A$ satisfying

$\rho_A \le \rho_S,$

so its guaranteed convergence rate is never worse than that of synchronous propagation. On top of this, residual belief propagation prioritizes the message with largest residual

$$r_m(v)=\|f_m(v)-v_m\|_m,\qquad m^*=\arg\max_m r_m(v^{(t)}),$$

using a priority queue to focus updates where local inconsistency is greatest (Elidan et al., 2012).

In decentralized stochastic optimization, the main tension is between synchronization barriers and stale parameters. DSGD-AAU replaces both full-neighbor synchronous averaging and fully wait-free asynchronous mixing by adaptive partial averaging. At iteration $k$, only a subset of workers $\mathcal N(k)\subset \mathcal N$ is active, and each active worker averages over a subset $$f_m^a(v_1,\dots,v_{|M|}) = (v_1,\dots,f_m(v),\dots,v_{|M|}),$$0. The worker update is

$$f_m^a(v_1,\dots,v_{|M|}) = (v_1,\dots,f_m(v),\dots,v_{|M|}),$$1

with a time-varying doubly stochastic Metropolis matrix. The adaptive rule is connectivity-driven rather than threshold-based: the Pathsearch procedure keeps waiting only until a new useful communication path is established, and the proof upper bounds staleness periods in DSGD-AAU by $$f_m^a(v_1,\dots,v_{|M|}) = (v_1,\dots,f_m(v),\dots,v_{|M|}),$$2. Under bounded connectivity time, the method attains the canonical decentralized rate

$$f_m^a(v_1,\dots,v_{|M|}) = (v_1,\dots,f_m(v),\dots,v_{|M|}),$$3

which the paper interprets as linear speedup in $$f_m^a(v_1,\dots,v_{|M|}) = (v_1,\dots,f_m(v),\dots,v_{|M|}),$$4 despite asynchronous partial participation (Xiong et al., 2023).

Federated optimization makes the same issue explicit at the aggregation rule. The generalized asynchronous update is

$$f_m^a(v_1,\dots,v_{|M|}) = (v_1,\dots,f_m(v),\dots,v_{|M|}),$$5

with stochastic aggregation weights

$$f_m^a(v_1,\dots,v_{|M|}) = (v_1,\dots,f_m(v),\dots,v_{|M|}),$$6

Here $$f_m^a(v_1,\dots,v_{|M|}) = (v_1,\dots,f_m(v),\dots,v_{|M|}),$$7 records the recency of the global model seen by client $$f_m^a(v_1,\dots,v_{|M|}) = (v_1,\dots,f_m(v),\dots,v_{|M|}),$$8, while $$f_m^a(v_1,\dots,v_{|M|}) = (v_1,\dots,f_m(v),\dots,v_{|M|}),$$9 can correct for unequal arrival frequencies. The framework shows that asynchronous arrivals induce a time-varying effective objective

$p.e \to q$0

so naive asynchrony can optimize the wrong mixture of client objectives. FedFix addresses this by aggregating all messages received in a fixed interval $p.e \to q$1, with weights

$p.e \to q$2

thereby interpolating between synchronous FedAvg and one-message-at-a-time asynchronous aggregation (Fraboni et al., 2022).

3. Learning systems built around asynchronous updates

AsyncMesh extends asynchrony across both data and pipeline parallelism. In pipeline parallelism, a stage update is based on delayed parameters,

$p.e \to q$3

and staleness is mitigated with weight look-ahead. In data parallelism, workers exchange only a random sparse subset of parameters and apply delayed sparse averages,

$p.e \to q$4

then correct the stale average with an EMA-based estimate of drift,

$p.e \to q$5

The paper proves consensus in expectation for sparse averaging and delayed averaging with EMA, and reports experiments on LLMs up to $p.e \to q$6B parameters that match the fully synchronous baseline while significantly reducing communication overhead (Ajanthan et al., 30 Jan 2026).

PD-ASGD moves the same discussion inside backpropagation. A forward thread computes a loss on the latest visible parameters $p.e \to q$7, sends that loss to a backward thread, and then continues immediately. Each backward thread iterates from layer $p.e \to q$8 to $p.e \to q$9, computes $t_n \in \{0,1,\dots,D\},$0, applies

$t_n \in \{0,1,\dots,D\},$1

and writes the updated layer back to shared memory immediately. This creates a pipeline of loss messages and layer-wise parameter writes rather than one fully synchronized forward–backward unit. The paper models staleness as additive perturbation $t_n \in \{0,1,\dots,D\},$2, derives the stale-gradient expansion

$t_n \in \{0,1,\dots,D\},$3

and proves convergence to a stationary distribution of the induced SDE. Empirically, layer-wise updates achieve higher accuracy than block updates on CIFAR-10 and CIFAR-100, while asynchronous variants yield substantial time-to-accuracy gains over LAPSGD and LPPSGD (Fokam et al., 2024).

FedRevive addresses stale asynchronous updates in federated learning by treating them as knowledge sources rather than merely as noisy parameter deltas. If a client trained on $t_n \in \{0,1,\dots,D\},$4 returns $t_n \in \{0,1,\dots,D\},$5, the server forms the raw update

$t_n \in \{0,1,\dots,D\},$6

and a data-free distillation update

$t_n \in \{0,1,\dots,D\},$7

then combines them through a staleness-dependent weight $t_n \in \{0,1,\dots,D\},$8: $t_n \in \{0,1,\dots,D\},$9 The server maintains a rolling KD buffer of recent client models and uses a meta-learned generator to synthesize pseudo-samples for multi-teacher distillation. Reported gains are up to $T_n<d_n$0 faster training and up to $T_n<d_n$1 higher final accuracy than asynchronous baselines (Askin et al., 1 Nov 2025).

4. State dissemination, consensus, and storage

Asynchronous message updates are equally central in state-replication systems. The ACA protocol adopts a hybrid pull/push gossip mechanism in which each node periodically sends a Synchronisation Request containing its Gossip List and Lamport time, and the contacted peer replies with its Gossip List, Lamport time, and a bundle of events believed unknown to the requester. Event insertion is incremental and local: frame numbers and flag tables are recomputed rather than transmitted. This yields repeated bilateral gossip synchronization without global rounds, but the paper explicitly states that the current version does not cover node failure or significantly delayed response (Zakharov, 2020).

Atomic snapshot in asynchronous message-passing systems uses a different structure: long-lived generalized lattice agreement. A call $T_n<d_n$2 first disseminates $T_n<d_n$3, buffered values are gathered in $T_n<d_n$4, then candidate proposals are spread through $T_n<d_n$5, validated after receiving support $T_n<d_n$6, and learned values are accelerated by $T_n<d_n$7. The paper’s main latency results are: at most $T_n<d_n$8 rounds in the good case without contention, at most $T_n<d_n$9 rounds when no correct node receives a message from a faulty one during the operation, worst-case $\rho_A$0 rounds where $\rho_A$1 is the number of active faulty nodes, and constant amortized latency. It also introduces IRA, a round-based metric for long-lived asynchronous executions with holes, because conventional latency notions become inadequate in that setting (Bezerra et al., 2024).

AsyncFS applies the same principle to filesystem metadata. Operations such as create, delete, and mkdir can return early after the target inode has been durably logged and updated locally and the parent-directory effect has only been appended to a change-log. The parent directory is then marked scattered in an in-network stale set stored on a programmable switch. Before statdir or readdir returns, if the directory is scattered, the owner aggregates change-logs from all servers, applies them, and only then answers the read. The switch tracks 49-bit directory fingerprints, supports linearizable insert, query, and remove, and enables batching plus change-log recast before reads. On skewed workloads, the reported improvements are up to $\rho_A$2 and $\rho_A$3 higher throughput, and $\rho_A$4 and $\rho_A$5 lower latency than InfiniFS and CFS-KV, respectively (Xu et al., 2024).

Other storage-oriented treatments show the same pattern of deferred visibility plus later repair. In Byzantine asynchronous read/write memory, updates are disseminated by reliable broadcast, accepted by quorums of size $\rho_A$6, and later reads execute a catch_up phase to prevent read inversion; in the enterprise MSMQ design, successfully executed SQL statements are inserted into a special table and shipped as transactional MSMQ messages to the remote site, where they are stored and then replayed against the destination database (Mosteafoui et al., 2015, 0912.2134).

5. Program semantics, runtime systems, and graph-structured updates

At the language-semantics level, asynchronous message updates can be represented explicitly rather than hidden in queues. In asynchronous choreographies, a source communication $\rho_A$7 is unfolded into a sender-local runtime action and a pending receive term, so that send-side progress can be reordered by structural precongruence. The semantics is asynchronous precisely when a send can occur before the receiver is ready and a pending receive can later consume the message, and the paper proves asynchronous equivalence to the original synchronous semantics as well as preservation of deadlock-freedom (Cruz-Filipe et al., 2017).

Rumpsteak extends this idea to practical Rust async/await code through multiparty session types. Its distinctive feature is support for arbitrary local reordering of sends and receives, verified by asynchronous multiparty subtyping and, when needed, $\rho_A$8-multiparty compatibility. The framework allows optimizations such as double buffering, streaming unrolling, and send-before-receive patterns in a ring, while preserving deadlock-freedom by construction. The new subtyping algorithm is sound and decidable, and the evaluation reports runtime improvements of about $\rho_A$9–$\rho_A \le \rho_S,$0 over previous Rust session-type implementations, with much better verification scalability than global $\rho_A \le \rho_S,$1-MC on the benchmarks considered (Cutner et al., 2021).

In decentralized dynamic graph processing, the same abstraction appears as an active-message execution model. On AM-CCA, an action is sent to the memory locality that owns the target vertex, can mutate local state, and can spawn more actions. Edge streaming uses insert-edge-action; if local adjacency storage is full, a remote allocate action is issued, the current computation is suspended via a continuation, and a future stores the eventual ghost-vertex pointer. Once insertion succeeds, a user-level bfs-action can be propagated immediately so that previous results are updated without recomputing from scratch (Chandio et al., 2024).

Graph neural networks adapt asynchrony at the level of node updates rather than threads or servers. CAMP orders nodes by centrality and partitions them into layer-specific batches $\rho_A \le \rho_S,$2. At layer $\rho_A \le \rho_S,$3, only nodes in the current batch update: $\rho_A \le \rho_S,$4 This means different nodes carry representations with different last-updated layers, so information moves sequentially through bottlenecks instead of being compressed everywhere at once. The paper proves a higher feature-sensitivity bound than the standard synchronous bound and reports about $\rho_A \le \rho_S,$5 improvement on REDDIT-BINARY and $\rho_A \le \rho_S,$6 improvement on Peptides-struct (Bose et al., 8 Sep 2025).

6. Recurrent conditions, metrics, and limitations

A recurrent theme is that asynchrony alone is not a guarantee of efficiency or correctness. Fixed-point inference needs contraction plus fairness (Elidan et al., 2012). Decentralized optimization needs bounded connectivity time, and its “fully asynchronous” alternative can suffer from arbitrarily stale buffered information unless constrained by adaptive partial waiting (Xiong et al., 2023). Federated learning shows that asynchronous arrivals can bias the optimized objective unless aggregation weights are corrected so that expected participation matches intended client importance (Fraboni et al., 2022).

Another recurring point is that long-lived asynchronous systems need explicit visibility control. AsyncFS does not preserve synchronous semantics by materializing all effects immediately; it preserves them because every directory read first queries stale state and, if necessary, reconciles delayed logs before returning (Xu et al., 2024). Atomic snapshot does not measure latency by naive wall-clock-style rounds; it introduces IRA because holes and silent periods make older metrics inadequate (Bezerra et al., 2024). This suggests that asynchronous message updates are often inseparable from a companion notion of pending-state metadata.

The literature also warns against collapsing distinct phenomena under one label. In massive connectivity, “asynchronous” means user transmissions arrive with unknown delay bins rather than that AMP itself is asynchronously scheduled (Zhu et al., 2021). In collaborative perception, asynchronous status updates are acceptable only when the generation-time spread stays within $\rho_A \le \rho_S,$7, so the scheduling problem is not merely rate feasibility but also offset and gcd compatibility under periodic schedules (Wang et al., 2023). In ACA, the communication pattern is asynchronous and gossip-based, but the protocol explicitly does not cover node failure or significantly delayed response (Zakharov, 2020).

Taken together, these results show that asynchronous message updates are best viewed not as a single algorithmic trick but as a design space. The common structure is the removal of a global barrier; the hard part is always what replaces it: residual-based scheduling, bounded-connectivity windows, stochastic aggregation weights, stale-set tracking, runtime send/receive splitting, local futures and continuations, or delayed-update correction in function space. The most successful systems are those that make this replacement explicit and analyzable rather than treating asynchrony as an implementation detail.