Reduce the stochastic and asynchrony-induced error terms

Reduce both the non-vanishing stochastic error term arising from Byzantine-robust aggregation and the asynchrony-induced error term in the convergence bound for Throttle, including the \(c\delta\sigma^2\) and \(G^2\|\mathcal{B}\|^2/(q-1)^2\) contributions.

Background

The convergence bound for Throttle contains a stochastic error term associated with Byzantine-robust aggregation and an additional error term caused by asynchronous updates and finite throttling parameter $q$. The paper notes that momentum or variance reduction can remove the aggregation-related term in synchronous settings, but that these techniques alone do not remove the principal asynchronous error; combining momentum with stale gradients is also described as nontrivial. The authors explicitly leave the joint reduction of both terms unresolved.