Dimension-free decentralized zeroth-order optimization

Generalize the decentralized online-to-nonconvex conversion for nonsmooth nonconvex stochastic optimization to the zeroth-order setting and determine whether it can achieve comparable improvements in sample and communication complexity.

Background

The proposed conversion uses first-order stochastic gradient oracles and removes the polynomial dependence on the problem dimension from the sample complexity, while leaving only logarithmic dimension dependence in the communication complexity. The paper notes that extending this mechanism to zeroth-order optimization is unresolved, despite randomized smoothing being a relevant technique in that setting. The open issue is whether a corresponding decentralized zeroth-order method can obtain similar complexity improvements.

References

Nonetheless, several questions remain open. First, it is unclear whether our communication complexity is nearly optimal. Second, it may be possible to generalize our conversion to the zeroth-order setting and achieve similar improvements.

— Dimension-Free Decentralized Nonsmooth Nonconvex Stochastic Optimization  (2610.05789 - Wan et al., 5 Oct 2026) in Section 5, Conclusion and Future Work