Obtaining Lower Query Complexities through Lightweight Zeroth-Order Proximal Gradient Algorithms (2410.02559v1)

Abstract: Zeroth-order (ZO) optimization is one key technique for machine learning problems where gradient calculation is expensive or impossible. Several variance reduced ZO proximal algorithms have been proposed to speed up ZO optimization for non-smooth problems, and all of them opted for the coordinated ZO estimator against the random ZO estimator when approximating the true gradient, since the former is more accurate. While the random ZO estimator introduces bigger error and makes convergence analysis more challenging compared to coordinated ZO estimator, it requires only $\mathcal{O}(1)$ computation, which is significantly less than $\mathcal{O}(d)$ computation of the coordinated ZO estimator, with $d$ being dimension of the problem space. To take advantage of the computationally efficient nature of the random ZO estimator, we first propose a ZO objective decrease (ZOOD) property which can incorporate two different types of errors in the upper bound of convergence rate. Next, we propose two generic reduction frameworks for ZO optimization which can automatically derive the convergence results for convex and non-convex problems respectively, as long as the convergence rate for the inner solver satisfies the ZOOD property. With the application of two reduction frameworks on our proposed ZOR-ProxSVRG and ZOR-ProxSAGA, two variance reduced ZO proximal algorithms with fully random ZO estimators, we improve the state-of-the-art function query complexities from $\mathcal{O}\left(\min{\frac{dn^{1/2}}{\epsilon^2}, \frac{d}{\epsilon^3}}\right)$ to $\tilde{\mathcal{O}}\left(\frac{n+d}{\epsilon^2}\right)$ under $d > n^{\frac{1}{2}}$ for non-convex problems, and from $\mathcal{O}\left(\frac{d}{\epsilon^2}\right)$ to $\tilde{\mathcal{O}}\left(n\log\frac{1}{\epsilon}+\frac{d}{\epsilon}\right)$ for convex problems.

• The paper presents variance-reduced ZO proximal gradient methods, ZOR-ProxSVRG and ZOR-ProxSAGA, that lower function query complexities using random zero-order estimators.
• The approach leverages the ZOO objective decrease (ZOOD) property to integrate smoothing errors and ensure robust convergence.
• Empirical validations on adversarial attacks and logistic regression showcase superior efficiency in both convex and non-convex optimization scenarios.

Essay: Lower Query Complexities in Zeroth-Order Proximal Gradient Algorithms

The paper "Obtaining Lower Query Complexities through Lightweight Zeroth-Order Proximal Gradient Algorithms" presents an advancement in the domain of zeroth-order (ZO) optimization, particularly focusing on reducing the function query complexities (FQC) associated with ZO proximal algorithms. Zeroth-order optimization is crucial for machine learning tasks where calculating gradients is challenging or infeasible.

Overview

The authors introduce novel lightweight variance-reduced ZO proximal gradient algorithms, specifically the ZOR-ProxSVRG and ZOR-ProxSAGA, which predominantly use random ZO estimators. These algorithms optimize both convex and non-convex functions with reduced computational overhead.

Key Innovations:

1. ZO Objective Decrease (ZOOD) Property: A unique property that includes errors from smoothing parameters and random ZO estimators to ensure convergence, distinguishing it from traditional first-order techniques.
2. Reduction Frameworks: Two reduction frameworks—AdaptRdct-C for convex problems and AdaptRdct-NC for non-convex problems—are introduced to lower query complexities significantly.
3. Superior Query Complexities: The proposed methods achieve improved query complexities from previous standards of $$\mathcal{O}\left(\min\{\frac{dn^{1/2}}{\epsilon^2}, \frac{d}{\epsilon^3}\}\right)$$ to $$\tilde{\mathcal{O}}\left(\frac{n+d}{\epsilon^2}\right)$$ for non-convex problems under $d > n^{1/2}$, and from $\mathcal{O}\left(\frac{d}{\epsilon^2}\right)$ to $$\tilde{\mathcal{O}}\left(n\log\frac{1}{\epsilon}+\frac{d}{\epsilon}\right)$$ for convex problems.

Technical Insights

The approaches leverage the computational efficiency of random ZO estimators, which necessitate only $\mathcal{O}(1)$ computation compared to the $\mathcal{O}(d)$ of coordinated alternatives. This choice allows the algorithms to effectively address black-box optimization tasks prevalent in machine learning applications such as adversarial attacks and reinforcement learning.

The technical underpinning involves incorporating two distinct error components into convergence analysis, facilitating application in high-dimensional simulation tasks where the true gradient is not readily accessible. The authors also meticulously derive conditions under which their ZO algorithms satisfy the ZOOD property, illustrating robustness and efficacy.

Experimental Validation

Empirical results demonstrate the algorithms' advantages across several domains:

• Adversarial Attack Generation: Experiments on well-known datasets (Cifar-10, fmnist, Mnist) validate the reduced FQC in crafting adversarial examples.
• Logistic Regression: Both convex and non-convex formulations reveal the superior query efficiency of the proposed methods, bolstering their applicability in real-world regularized learning problems.

Future Prospects and Implications

The work invites further exploration into adaptive techniques for various other ZO optimization tasks beyond proximal gradient settings. These advancements could be pivotal for scalable machine learning algorithms where gradient computations are prohibitive due to computational constraints or model architecture complexities.

Additionally, the generalization capabilities in optimizing over complex spaces suggest potential applications in domains like bioinformatics and autonomous control systems, where model interpretability often relies on non-differentiable objective functions.

Conclusion

In conclusion, the researchers provide compelling evidence that leveraging random ZO estimators in variance-reduced settings can substantially reduce the FQC, opening avenues for more efficient and flexible applications of ZO optimization in machine learning. This approach marks a significant stride in the optimization field, providing both theoretical insights and practical tools to tackle high-dimensional black-box problems effectively.