Formal lower bound for log‑d dependence in non‑convex iteration complexity

Establish a formal lower bound proving that at least one factor of log d in the iteration complexity is unavoidable for algorithms that find an ε–second‑order stationary point in non‑convex optimization, motivated by the principal component analysis problem where the best known runtimes for the power method and Lanczos method incur a log d factor.

Background

The paper proves that perturbed gradient descent finds an ε–second‑order stationary point in O(polylog(d)/ε2) iterations and argues that at least one log d factor may be inherent.

The authors point to principal component analysis, for which the best known runtimes of the power and Lanczos methods include a log d factor, as evidence suggesting such a lower bound might be necessary.

References

We believe that the dependence on at least one log d factor in the iteration complexity is unavoidable in the non-convex setting, as our result can be directly applied to the principal component analysis problem, for which the best known runtimes (for the power method or Lanczos method) incur a log d factor. Establishing this formally is still an open question however.

— How to Escape Saddle Points Efficiently  (1703.00887 - Jin et al., 2017) in Section: Main Result (paragraph after Theorem \ref{thm:main})