FedSL1ESN Under Heterogeneous Client Distributions

Determine whether the reported accuracy and sparsity behavior of FedSL1ESN transfers from uniform IID partitions to non-IID client distributions, including the effects of client drift and minority-client-specific readout directions.

Background

The federated experiments use uniformly partitioned IID data and therefore do not establish how FedSL1ESN behaves when clients have heterogeneous data distributions. Under client drift, local Gauss-Newton Hessians may describe curvature around different local optima, potentially biasing their sum and the resulting global Newton direction.

A further concern is that a single global shrinkage threshold could remove readout directions that are important only to minority clients. The paper identifies Newton-type federated optimization methods for heterogeneous data as possible aggregation replacements, but leaves the interaction between such heterogeneity, Smooth-L1 continuation, and shared-threshold sparsification unresolved.

References

We do not claim that the reported behavior transfers to the non-IID case. Two risks are specific to our scheme: under client drift~\citep{shi2022optimization} the local Gauss-Newton Hessians describe curvature around different local optima, so their plain sum can bias the global Newton direction, and a single global shrinkage threshold could prune directions that only minority clients rely on.

— Sparse Federated Echo State Networks for Time-Series Classification via Smooth-L1 Regularization  (2610.21742 - Lyu et al., 15 Oct 2026) in Section 6, subsection “Limitations,” item (2), Non-IID data

how the $\alpha$-continuation and the shared threshold interact with client-heterogeneous sparsity patterns is the question specific to FedSL1ESN.

— Sparse Federated Echo State Networks for Time-Series Classification via Smooth-L1 Regularization  (2610.21742 - Lyu et al., 15 Oct 2026) in Section 6, subsection “Limitations,” item (2), Non-IID data