Gradient tracking with clipping without bounded heterogeneity

Establish whether baseline gradient tracking with gradient clipping can simultaneously eliminate the bounded heterogeneity condition and retain order-optimal convergence rates in decentralized optimization under heavy-tailed noise.

Background

The paper’s clipped decentralized SGD guarantees require a bounded heterogeneity assumption. The authors note that gradient tracking can remove the heterogeneity bound in related normalized methods, but their clipped baseline method does not do so. This raises the unresolved question of whether combining gradient tracking with clipping can remove that condition without sacrificing the order-optimal rates established in the paper.

The problem is specifically concerned with retaining the paper’s optimal convergence behavior while relaxing a structural assumption on the local objective gradients.

References

Another important direction is whether baseline GT with clipping can simultaneously eliminate the bounded heterogeneity condition and retain order-optimal convergence rates.

— Decentralized SGD under Heavy-Tailed Noise: Optimal Convergence Rates and the Role of Gradient Clipping  (2610.10527 - Armacki et al., 7 Oct 2026) in Section Conclusion

This, in turn, raises a further design question: should clipping be applied to local gradients or to the tracker variable, and what are the consequences of each choice?

— Decentralized SGD under Heavy-Tailed Noise: Optimal Convergence Rates and the Role of Gradient Clipping  (2610.10527 - Armacki et al., 7 Oct 2026) in Section Conclusion