Near-Optimal Convex Optimization with Lazy Second-Order Oracles

Abstract: This paper studies the complexity of convex optimization using lazy second-order oracles (Doikov, Chayti, and Jaggi, ICML 2023), where an algorithm queries gradients every iteration and Hessians once per $m$ iterations. Under this setting, we show a lower bound of $Ω(m+ m^{1/7} ε<sup>{-2/7})$ on the number of total iterations to find an $ε$-solution using a novel block zero-chain construction. Then we propose a novel method that achieves a new upper bound of $$\tilde{\mathcal{O}}(m+ m^{1/7} ε<sup>{-2/7})$$, which significantly improves the prior one (Chen, Liu, Luo, and Zhang, COLT 2026) of $$\tilde{\mathcal{O}}(m+ m^{13/21} ε<sup>{-2/7})$$ and is tight up to logarithmic factors.