Synchronization and consistency in federated learning

Develop robust, efficient synchronization and communication protocols that maintain consistent model updates and convergence across heterogeneous, intermittently connected devices in federated learning.

Background

Federated learning distributes model training across many devices without centralizing raw data. The paper explains that communication disruptions, device heterogeneity, and statistical diversity pose unique systems challenges.

Among these, ensuring that distributed devices remain in sync during training is highlighted as a core unresolved issue affecting training robustness and convergence.

References

Getting all the federated devices to talk to each other and stay in sync is still an open issue.

— Modern Computing: Vision and Challenges  (2401.02469 - Gill et al., 2024) in Section 4.4 (Decentralized Computing → Technologies/Impact Areas) — Federated Learning

Partial participation, client sampling and straggler effects are not evaluated here and are left to future work

— FedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantization  (2610.01537 - Zou et al., 1 Oct 2026) in Section VII, subsection “Scalability across client counts”

These methods are designed for step-level gradient synchronization in homogeneous clusters rather than round-level $g_{i,t}$ synchronization in heterogeneous GDT; adapting these methods to the GDT setting and a direct comparison remain future work, which we discuss in Appendix \ref{app:related_work}.

— GeoMesh: Workload-Balanced and Sign-Compressed Geo-Distributed LLM Training  (2609.18388 - Shin et al., 16 Sep 2026) in Section 6, Limitations, subsection “CSS design space”

Analyzing non-independent availability with uncertain probabilistic trajectories in~\prettyref{ass: prob lower bound} is in general challenging. For example, the involved entanglement of stochastic gradient and availability statistics fundamentally complicates the theoretical analysis. We conjecture that independent participation may be only for the technical convenience of our analysis.

— Resilience Beyond Stationary Client Unavailability: Unlocking Efficient and Unbiased Federated Learning  (2609.04763 - Xiang et al., 4 Sep 2026) in Remark 2.1, Section 2 (Problem Formulation)

Full asynchronous evaluation with real-world network conditions remains future work.

— RegionFed: Federated Learning for Personalized Query Understanding in Heterogeneous Retail Environments  (2609.05403 - Nguyen et al., 4 Sep 2026) in Appendix, Section 'Supplementary Algorithms and Extensions', subsection 'Asynchronous Federated Operations'

Asynchronous protocols that preserve convergence and privacy accounting while tolerating stragglers and site failures are, in our view, the most consequential open systems problem here.

— From Pilots to Production: Lessons in Cross-Institutional Federated Training and Artificial Intelligence for Science  (2609.39803 - Kotevska et al., 30 Sep 2026) in Section 4, “What Remains Unsolved, and a Shared Agenda”

For larger networks, a natural extension is to partition nodes into clusters and circulate one model per cluster with periodic inter-cluster synchronization, trading a part of the communication savings for parallelism; we leave this hierarchical extension to future work.

— Tram-FL: Reducing Communication and Computation Costs through Sequential Model Circulation in Decentralized Federated Learning  (2610.07859 - Maejima et al., 6 Oct 2026) in Section 3.1, System Model

Moreover, extension of BLEND to fully-decentralized/serverless FedVLMs remains open.

— BLEND: Balancing Personalization vs. Generalization in Federated Vision-Language Models  (2610.12907 - Abdisarabshali et al., 9 Oct 2026) in Section Limitations