Formal Convergence Analysis of GeoMesh

Prove formal convergence guarantees for GeoMesh, including its adaptive workload balancing and compressed sign synchronization, rather than relying only on empirical evidence and a reconstruction-error bound.

Background

The paper provides empirical evidence that GeoMesh achieves convergence behavior comparable to synchronous baselines and gives a structural reconstruction-error bound for compressed sign synchronization when Lion is used as the inner optimizer. However, that bound concerns reconstruction fidelity rather than optimization convergence. A formal convergence proof for GeoMesh remains unresolved.

References

GeoMesh preserves the training dynamics of the baseline synchronous GDT (\S\ref{sec:awb}) and achieves DiLoCo-level zero-shot accuracy (\S\ref{sec:zeroshot}), suggesting that its convergence behavior is consistent with that of previous synchronous GDT methods, but a formal convergence proof remains future work.

— GeoMesh: Workload-Balanced and Sign-Compressed Geo-Distributed LLM Training  (2609.18388 - Shin et al., 16 Sep 2026) in Section 6, Limitations, subsection “Theoretical guarantees”