Federated Linear Stochastic Approximation

• Federated LSA is a distributed stochastic approximation method that solves global linear fixed-point problems through local updates and periodic averaging.
• It mitigates client heterogeneity by balancing local training drift with server-side control variate corrections to align with a global solution.
• It addresses trade-offs in communication, computation, and variance reduction, incorporating techniques like Markovian sampling and bias correction.

Searching arXiv for recent and foundational papers on federated linear stochastic approximation. Federated Linear Stochastic Approximation (LSA) is a class of distributed stochastic recursive methods for solving a global linear fixed-point or linear root-finding problem when data are split across multiple agents and only intermittent communication is permitted. In its most direct formulation, each client $c$ has access to stochastic linear observations whose expectations define a local system $\bar A^{[c]}\theta=\bar b^{[c]}$, while the federated objective is the global linear system

$$\mathbf A \theta_\star = \bar b, \qquad \mathbf A = \frac1N \sum_{c=1}^N \mathbf A^{[c]}, \qquad \bar b = \frac1N \sum_{c=1}^N \bar b^{[c]}.$$

A federated LSA algorithm alternates between local stochastic approximation steps and server-side aggregation, typically through periodic averaging or control-variate correction. The topic lies at the intersection of stochastic approximation, federated learning, distributed linear algebra, and reinforcement learning, especially temporal-difference (TD) learning with linear function approximation. Recent work has clarified three central issues: the effect of local training under client heterogeneity, the role of Markovian sampling, and the communication–computation trade-off induced by multiple local steps (Mangold et al., 2024, Zhu et al., 15 Apr 2025, Khodadadian et al., 2022).

1. Conceptual and mathematical framework

Federated LSA is most naturally posed as a global linear SA problem over $N$ clients. In the abstract matrix form emphasized by recent work, each client $c$ has local mean quantities $(\mathbf A^{[c]},\bar b^{[c]})$, local stochastic estimators $\mathbf A^c(Z)$ and $\mathbf b^c(Z)$, and a local root

$$\theta_\star^{[c]} \in \mathbb R^d, \qquad \mathbf A^{[c]} \theta_\star^{[c]} = \bar b^{[c]}.$$

The federated target is instead the global solution

$\theta_\star = \mathbf A^{-1}\bar b,$

when $\bar A^{[c]}\theta=\bar b^{[c]}$0 is invertible (Levin et al., 19 May 2026). This distinction between local roots and the root of the averaged system is the defining structural issue of heterogeneous federated LSA (Mangold et al., 2024, Zhu et al., 15 Apr 2025).

A standard single-stream LSA recursion takes the form

$\bar A^{[c]}\theta=\bar b^{[c]}$1

or, equivalently,

$\bar A^{[c]}\theta=\bar b^{[c]}$2

with target determined by the mean equation $\bar A^{[c]}\theta=\bar b^{[c]}$3 or $\bar A^{[c]}\theta=\bar b^{[c]}$4 (Huo et al., 2022, Lakshminarayanan et al., 2017, Huo et al., 2023). In the federated setting, local copies evolve separately and are periodically synchronized. One canonical client/server recursion is

$\bar A^{[c]}\theta=\bar b^{[c]}$5

which is the baseline FedLSA scheme analyzed in detail in "SCAFFLSA: Taming Heterogeneity in Federated Linear Stochastic Approximation and TD Learning" (Mangold et al., 2024). In a broader stochastic-approximation notation, federated methods can also be written as local recursions for client-specific operators $\bar A^{[c]}\theta=\bar b^{[c]}$6 or $\bar A^{[c]}\theta=\bar b^{[c]}$7, followed by averaging at a centralized coordinator (Doan, 2020, Khodadadian et al., 2022).

This abstract viewpoint covers several problem classes. In optimization language, the mean field may be linear with $\bar A^{[c]}\theta=\bar b^{[c]}$8, so the limiting ODE is $\bar A^{[c]}\theta=\bar b^{[c]}$9 (V et al., 2024). In reinforcement learning, linear TD updates generate precisely this structure after suitable centering (Khodadadian et al., 2022, Doan, 2020). In row-action linear system solvers, randomized Kaczmarz becomes a stochastic projection method whose mean operator is linear in the classical single-row case and state-dependent in federated variants (Jeong et al., 14 May 2025). This suggests that federated LSA is less a single algorithm than a family of federated root-finding procedures with a shared linear mean-field structure.

2. Baseline local-update methods and their drift pathology

The simplest federated LSA protocol is a local-update method analogous to FedAvg or local SGD: broadcast a global model, run $$\mathbf A \theta_\star = \bar b, \qquad \mathbf A = \frac1N \sum_{c=1}^N \mathbf A^{[c]}, \qquad \bar b = \frac1N \sum_{c=1}^N \bar b^{[c]}.$$0 local stochastic approximation steps at each client, and average the local endpoints. This template appears in several forms. "Local Stochastic Approximation: A Unified View of Federated Learning and Distributed Multi-Task Reinforcement Learning Algorithms" (Doan, 2020) studies the recursion

$$\mathbf A \theta_\star = \bar b, \qquad \mathbf A = \frac1N \sum_{c=1}^N \mathbf A^{[c]}, \qquad \bar b = \frac1N \sum_{c=1}^N \bar b^{[c]}.$$1

under Markovian local data. "Federated Stochastic Approximation under Markov Noise and Heterogeneity" introduces FedSAM, a periodic averaging algorithm

$$\mathbf A \theta_\star = \bar b, \qquad \mathbf A = \frac1N \sum_{c=1}^N \mathbf A^{[c]}, \qquad \bar b = \frac1N \sum_{c=1}^N \bar b^{[c]}.$$2

with synchronization every $$\mathbf A \theta_\star = \bar b, \qquad \mathbf A = \frac1N \sum_{c=1}^N \mathbf A^{[c]}, \qquad \bar b = \frac1N \sum_{c=1}^N \bar b^{[c]}.$$3 steps (Khodadadian et al., 2022). In the explicit federated linear SA treatment of (Mangold et al., 2024), the same structure is analyzed directly in matrix form.

The central difficulty is that local training is unbiased only in the single-local-step regime. In FedLSA, the global iterate admits the exact decomposition

$$\mathbf A \theta_\star = \bar b, \qquad \mathbf A = \frac1N \sum_{c=1}^N \mathbf A^{[c]}, \qquad \bar b = \frac1N \sum_{c=1}^N \bar b^{[c]}.$$4

where $$\mathbf A \theta_\star = \bar b, \qquad \mathbf A = \frac1N \sum_{c=1}^N \mathbf A^{[c]}, \qquad \bar b = \frac1N \sum_{c=1}^N \bar b^{[c]}.$$5 is a deterministic heterogeneity bias, $$\mathbf A \theta_\star = \bar b, \qquad \mathbf A = \frac1N \sum_{c=1}^N \mathbf A^{[c]}, \qquad \bar b = \frac1N \sum_{c=1}^N \bar b^{[c]}.$$6 is a random fluctuation of that bias, and $$\mathbf A \theta_\star = \bar b, \qquad \mathbf A = \frac1N \sum_{c=1}^N \mathbf A^{[c]}, \qquad \bar b = \frac1N \sum_{c=1}^N \bar b^{[c]}.$$7 is the usual stochastic noise term (Mangold et al., 2024). The deterministic bias satisfies

$$\mathbf A \theta_\star = \bar b, \qquad \mathbf A = \frac1N \sum_{c=1}^N \mathbf A^{[c]}, \qquad \bar b = \frac1N \sum_{c=1}^N \bar b^{[c]}.$$8

and vanishes when $$\mathbf A \theta_\star = \bar b, \qquad \mathbf A = \frac1N \sum_{c=1}^N \mathbf A^{[c]}, \qquad \bar b = \frac1N \sum_{c=1}^N \bar b^{[c]}.$$9: $N$0 (Mangold et al., 2024). For $N$1, however, local training drives each client toward $N$2 rather than $N$3, producing drift that cannot be removed by plain averaging.

A sharper negative result appears in "Achieving Tighter Finite-Time Rates for Heterogeneous Federated Stochastic Approximation under Markovian Sampling" (Zhu et al., 15 Apr 2025). In the noiseless linear case with $N$4, vanilla local SA satisfies

$N$5

where

$N$6

This establishes that under heterogeneity, plain local training can converge to the wrong point even without stochastic noise (Zhu et al., 15 Apr 2025).

A plausible implication is that federated LSA should not be understood as a direct linear generalization of centralized SA with parallel variance reduction. The dominant obstacle is not merely noise aggregation but the incompatibility between local equilibria and the global target.

3. Heterogeneity, control variates, and bias-corrected algorithms

The first dedicated matrix-level response to client drift in federated LSA is SCAFFLSA (Mangold et al., 2024). Its starting point is the ideal control variate

$N$7

which would exactly recenter client $N$8's local dynamics around the global solution rather than the local one. Since $N$9 is unknown, SCAFFLSA learns client-specific controls $c$0 and performs corrected local steps

$c$1

(Mangold et al., 2024). In the deterministic block-communication version, clients run $c$2 corrected local steps, average endpoints, and update the controls by

$c$3

This produces a Lyapunov recursion in

$c$4

leading to logarithmic communication complexity in $c$5 (Mangold et al., 2024).

FedHSA generalizes the same principle beyond linear SA and makes the linear case explicit (Zhu et al., 15 Apr 2025). The key local update is

$c$6

followed by

$c$7

In linear SA notation $c$8, this is a heterogeneity-corrected affine recursion whose fixed point is the root of the averaged operator, not the local roots (Zhu et al., 15 Apr 2025). The correction term

$c$9

plays the same conceptual role as the client-specific control variate in SCAFFLSA: it removes first-order drift toward $(\mathbf A^{[c]},\bar b^{[c]})$0.

The contrast between FedLSA and SCAFFLSA in (Mangold et al., 2024), and between vanilla local SA and FedHSA in (Zhu et al., 15 Apr 2025), resolves a common misconception. The difficulty with local training under heterogeneity is not merely slower convergence; it is convergence to the wrong limit unless local drift is explicitly neutralized.

4. Markovian sampling and reinforcement-learning structure

A distinctive feature of federated SA, compared with classical federated optimization, is that local sampling is often Markovian rather than i.i.d. This is fundamental in reinforcement learning, streaming data, and on-device sequential logs. "Federated Stochastic Approximation under Markov Noise and Heterogeneity" (Khodadadian et al., 2022) and "Local Stochastic Approximation" (Doan, 2020) both formulate federated or local SA with agent-specific ergodic Markov chains. FedSAM assumes each client $(\mathbf A^{[c]},\bar b^{[c]})$1 has a Markov process $(\mathbf A^{[c]},\bar b^{[c]})$2, local operator $(\mathbf A^{[c]},\bar b^{[c]})$3, additive term $(\mathbf A^{[c]},\bar b^{[c]})$4, and expected operator $(\mathbf A^{[c]},\bar b^{[c]})$5, with geometric mixing: $(\mathbf A^{[c]},\bar b^{[c]})$6 (Khodadadian et al., 2022).

The linear TD specialization is especially important because it is a bona fide federated LSA instance. In the on-policy case with linear function approximation,

$(\mathbf A^{[c]},\bar b^{[c]})$7

so after centering around $(\mathbf A^{[c]},\bar b^{[c]})$8, the expected operator is linear and the fixed point is $(\mathbf A^{[c]},\bar b^{[c]})$9 (Khodadadian et al., 2022). In "Local Stochastic Approximation" (Doan, 2020), distributed TD(0) is written explicitly in local-SA form with

$\mathbf A^c(Z)$0

and root equation

$\mathbf A^c(Z)$1

(Doan, 2020).

Recent federated RL analyses sharpen this connection. (Zhu et al., 15 Apr 2025) maps federated TD with linear function approximation into a general heterogeneous SA framework through

$\mathbf A^c(Z)$2

which is linear in $\mathbf A^c(Z)$3. (Mangold et al., 19 Dec 2025) shows that federated SARSA with linear function approximation can be viewed as federated linear SA with a parameter-dependent mean field. Conditional on the frozen policy during a communication block, the local SARSA update is

$\mathbf A^c(Z)$4

with blockwise mean operator $\mathbf A^c(Z)$5 (Mangold et al., 19 Dec 2025). This suggests that federated LSA is not confined to static linear systems; it also serves as the local linear backbone of policy-evaluation and control algorithms with periodically frozen policies.

The contrast with centralized Markovian LSA is also instructive. (Huo et al., 2022) proves that constant-stepsize LSA with Markovian data converges not to $\mathbf A^c(Z)$6 but to a stationary law whose mean has bias

$\mathbf A^c(Z)$7

with $\mathbf A^c(Z)$8 proportional to Markov mixing in the reversible setting (Huo et al., 2022). Averaging reduces variance but not this bias. This suggests that in federated settings with Markovian local sampling, there are two distinct bias mechanisms: client-drift bias from heterogeneity and stationary Markovian bias from constant local steps. A plausible implication is that control variates solve only the first of these unless stepsizes or extrapolation schemes also address the second.

5. Finite-time rates, sample complexity, and communication complexity

The modern federated LSA literature is organized around a three-way complexity trade-off between the number of clients, the number of local steps, and the desired accuracy.

In FedLSA, the key finite-time quantities are the local-noise scale

$\mathbf A^c(Z)$9

and the heterogeneity fluctuation term

$\mathbf b^c(Z)$0

The MSE bound shows that variance-like terms improve with $\mathbf b^c(Z)$1, but the deterministic heterogeneity bias $\mathbf b^c(Z)$2 must itself be kept below the target accuracy (Mangold et al., 2024). This yields the characteristic FedLSA result:

• Sample/oracle complexity: with $\mathbf b^c(Z)$3 chosen carefully, total local updates satisfy

$\mathbf b^c(Z)$4

which exhibits linear speed-up in $\mathbf b^c(Z)$5 (Mangold et al., 2024).

• Communication complexity: the number of communication rounds contains a polynomial $\mathbf b^c(Z)$6 term because $\mathbf b^c(Z)$7 must scale like $\mathbf b^c(Z)$8 to keep $\mathbf b^c(Z)$9 small (Mangold et al., 2024).

SCAFFLSA changes this picture. In the randomized-communication setting, the Lyapunov recursion

$$\theta_\star^{[c]} \in \mathbb R^d, \qquad \mathbf A^{[c]} \theta_\star^{[c]} = \bar b^{[c]}.$$0

shows that heterogeneity no longer appears as a persistent bias floor (Mangold et al., 2024). In the deterministic block version,

$$\theta_\star^{[c]} \in \mathbb R^d, \qquad \mathbf A^{[c]} \theta_\star^{[c]} = \bar b^{[c]}.$$1

from which the number of communication rounds becomes

$$\theta_\star^{[c]} \in \mathbb R^d, \qquad \mathbf A^{[c]} \theta_\star^{[c]} = \bar b^{[c]}.$$2

(Mangold et al., 2024). The logarithmic dependence on $$\theta_\star^{[c]} \in \mathbb R^d, \qquad \mathbf A^{[c]} \theta_\star^{[c]} = \bar b^{[c]}.$$3 is the principal communication improvement over FedLSA.

FedHSA obtains an analogous no-bias finite-time result in a broader SA setting with Markovian sampling. Under strong monotonicity of the average operator and geometric mixing, Theorem 2 of (Zhu et al., 15 Apr 2025) yields

$$\theta_\star^{[c]} \in \mathbb R^d, \qquad \mathbf A^{[c]} \theta_\star^{[c]} = \bar b^{[c]}.$$4

where $$\theta_\star^{[c]} \in \mathbb R^d, \qquad \mathbf A^{[c]} \theta_\star^{[c]} = \bar b^{[c]}.$$5, $$\theta_\star^{[c]} \in \mathbb R^d, \qquad \mathbf A^{[c]} \theta_\star^{[c]} = \bar b^{[c]}.$$6, and $$\theta_\star^{[c]} \in \mathbb R^d, \qquad \mathbf A^{[c]} \theta_\star^{[c]} = \bar b^{[c]}.$$7 (Zhu et al., 15 Apr 2025). The dominant variance term scales as $$\theta_\star^{[c]} \in \mathbb R^d, \qquad \mathbf A^{[c]} \theta_\star^{[c]} = \bar b^{[c]}.$$8, and the linear-speedup corollary gives

$$\theta_\star^{[c]} \in \mathbb R^d, \qquad \mathbf A^{[c]} \theta_\star^{[c]} = \bar b^{[c]}.$$9

This is one of the strongest currently available finite-time statements for heterogeneous federated SA with Markovian sampling (Zhu et al., 15 Apr 2025).

A recurring point of comparison is FedSAM (Khodadadian et al., 2022). Its finite-time error bound

$\theta_\star = \mathbf A^{-1}\bar b,$0

exhibits the same decomposition into transient decay, $\theta_\star = \mathbf A^{-1}\bar b,$1 variance reduction, and local-update penalty (Khodadadian et al., 2022). Although not formulated in matrix LSA notation, it provides a closely related communication–computation trade-off for contractive Markovian SA.

6. Statistical fluctuations, Gaussian approximation, and inference

Recent work extends federated LSA theory beyond convergence rates to distributional approximation and statistical inference. "Gaussian Approximation and Multiplier Bootstrap for Federated Linear Stochastic Approximation" (Levin et al., 19 May 2026) studies the last iterate in a federated local-update recursion

$\theta_\star = \mathbf A^{-1}\bar b,$2

under i.i.d. local samples and full participation. The analysis decomposes the last-iterate error into transient, deterministic heterogeneity bias, heterogeneity-induced fluctuations, and stochastic fluctuation: $\theta_\star = \mathbf A^{-1}\bar b,$3 (Levin et al., 19 May 2026). Heterogeneity is quantified by

$\theta_\star = \mathbf A^{-1}\bar b,$4

The main Gaussian approximation theorem states that for polynomial stepsize and local-step schedules,

$\theta_\star = \mathbf A^{-1}\bar b,$5

where $\theta_\star = \mathbf A^{-1}\bar b,$6 and $\theta_\star = \mathbf A^{-1}\bar b,$7 (Levin et al., 19 May 2026). This explicitly shows how increasing local steps worsens distributional accuracy through the heterogeneity-dependent remainder terms.

The same paper develops an online multiplier bootstrap. Bootstrap local updates take the form

$\theta_\star = \mathbf A^{-1}\bar b,$8

with i.i.d. multipliers $\theta_\star = \mathbf A^{-1}\bar b,$9, and conditional validity is proved for the law of $\bar A^{[c]}\theta=\bar b^{[c]}$00 as an approximation to that of $\bar A^{[c]}\theta=\bar b^{[c]}$01 (Levin et al., 19 May 2026). This is the first fully federated inference result for LSA in the available literature.

These developments connect back to centralized LSA inference. (Samsonov et al., 2024) gives Berry–Esseen and bootstrap results for Polyak–Ruppert averaged centralized LSA with decreasing step sizes, and (Mou et al., 2020) derives a fixed-stepsize CLT whose covariance is the sum of the classical PR covariance and a stepsize correction term. A plausible implication is that the federated Gaussian approximation in (Levin et al., 19 May 2026) plays the same role for local-update schemes that (Mou et al., 2020) plays for centralized fixed-step SA: it exposes the exact statistical cost of persistent local dispersion.

7. Related projection-based and ODE-based perspectives

Not all federated linear SA methods are expressed as affine recursions. "A federated Kaczmarz algorithm" (Jeong et al., 14 May 2025) studies FedRK, a projection-based method for distributed linear systems $\bar A^{[c]}\theta=\bar b^{[c]}$02 with rows partitioned across clients. At round $\bar A^{[c]}\theta=\bar b^{[c]}$03, selected clients run $\bar A^{[c]}\theta=\bar b^{[c]}$04 local randomized Kaczmarz steps, produce displacements $\bar A^{[c]}\theta=\bar b^{[c]}$05, and the server constructs a surrogate system $\bar A^{[c]}\theta=\bar b^{[c]}$06 and applies $\bar A^{[c]}\theta=\bar b^{[c]}$07 server-side Kaczmarz steps. In the idealized $\bar A^{[c]}\theta=\bar b^{[c]}$08, $\bar A^{[c]}\theta=\bar b^{[c]}$09 regime, the conditional mean update is

$\bar A^{[c]}\theta=\bar b^{[c]}$10

This is state-dependent and therefore outside canonical linear SA, but classical randomized Kaczmarz appears as a special case where the mean operator is linear (Jeong et al., 14 May 2025). This suggests an alternative geometric route to federated linear system solving in which communication exchanges projected constraints rather than model averages.

At the opposite end of the abstraction spectrum, "Stochastic Approximation Approach to Federated Machine Learning" (V et al., 2024) formulates periodic aggregation in a general SA form and shows that the synchronized averages track the ODE

$\bar A^{[c]}\theta=\bar b^{[c]}$11

Specializing $\bar A^{[c]}\theta=\bar b^{[c]}$12 yields the linear ODE $\bar A^{[c]}\theta=\bar b^{[c]}$13 (V et al., 2024). The paper is not about LSA specifically, but it provides an ODE-based conceptual bridge from local SA recursions to federated mean-field dynamics.

These two perspectives clarify a methodological split in the literature. Some work treats federated linear SA as a matrix perturbation problem with exact finite-time recursions (Mangold et al., 2024, Levin et al., 19 May 2026). Other work embeds it into broader operator-SA or ODE-tracking frameworks (Khodadadian et al., 2022, V et al., 2024). The two views are complementary rather than competing.

8. Scope, limitations, and open directions

Current federated LSA theory remains precise but narrow in several respects. Most matrix-level analyses assume i.i.d. local samples across time (Mangold et al., 2024, Levin et al., 19 May 2026), while Markovian analyses often work at the level of general operators or RL specializations rather than canonical $\bar A^{[c]}\theta=\bar b^{[c]}$14 matrix theory (Khodadadian et al., 2022, Zhu et al., 15 Apr 2025). Full participation is the dominant setting; partial participation, asynchronous communication, and compression are generally absent (Mangold et al., 2024, Levin et al., 19 May 2026). Last-iterate distribution theory exists for the homogeneous i.i.d. setting with heterogeneity-aware remainders (Levin et al., 19 May 2026), but an equally sharp treatment for Markovian federated LSA remains open.

Several technical gaps are explicit in the literature. (Mangold et al., 2024) shows that FedLSA preserves linear speed-up in sample complexity while SCAFFLSA restores communication efficiency, but no current analysis achieves both simultaneously. (Zhu et al., 15 Apr 2025) removes heterogeneity bias and proves linear speed-up in a broader SA framework, but its theorems are not specialized to the matrix-structured constants of pure LSA. (Levin et al., 19 May 2026) develops bootstrap inference under i.i.d. local data, but not under Markovian sampling or client subsampling. Centralized Markovian LSA work shows that constant stepsizes induce an $\bar A^{[c]}\theta=\bar b^{[c]}$15 stationary bias (Huo et al., 2022, Huo et al., 2023, Levin et al., 7 Aug 2025), yet a full federated synthesis of that bias with heterogeneity-induced local drift has not been established.

A common misconception is that federated LSA is now a solved analogue of local SGD for linear systems. Existing results instead suggest a more delicate picture. Heterogeneity changes the target of naive local training; Markovian sampling creates a distinct stationary bias under constant local steps; and communication reduction is statistically costly unless local drift is corrected. The main unresolved problem is to combine four properties in a single theorem: heterogeneity robustness, Markovian sampling, linear speed-up in the number of clients, and logarithmic communication dependence on $\bar A^{[c]}\theta=\bar b^{[c]}$16.

A plausible implication is that future progress will require hybrid techniques: matrix-level exact expansions as in (Mangold et al., 2024), operator-level drift correction as in (Zhu et al., 15 Apr 2025), and bias-cancellation mechanisms inspired by Markovian constant-step extrapolation (Huo et al., 2022, Levin et al., 7 Aug 2025). In reinforcement learning, the exact multi-step expansions recently developed for federated SARSA (Mangold et al., 19 Dec 2025) suggest that such synthesis may be possible for broader classes of projected and policy-dependent linear SA recursions.