Optimality of the communication complexity

Determine whether the nearly dimension-free communication complexity \(\widetilde{O}(\gamma^{-1/2}\delta^{-1}\epsilon^{-3})\) achieved for finding an \((\delta,\epsilon)\)-Goldstein stationary point in decentralized nonsmooth nonconvex stochastic optimization is nearly optimal.

Background

The paper develops a decentralized online-to-nonconvex conversion whose instantiation with accelerated decentralized follow-the-regularized-leader achieves $O(\delta^{-1}\epsilon^{-3})$ stochastic-gradient queries and $$\widetilde{O}(\gamma^{-1/2}\delta^{-1}\epsilon^{-3})$$ communication rounds. Although the sample complexity is optimal even in the non-distributed setting, the authors do not establish a matching lower bound or otherwise determine whether the communication dependence on the spectral gap, stationarity parameters, and other quantities is close to the best possible.