A simple and improved algorithm for noisy, convex, zeroth-order optimisation (2406.18672v1)

Abstract: In this paper, we study the problem of noisy, convex, zeroth order optimisation of a function $f$ over a bounded convex set $\bar{\mathcal X}\subset \mathbb{R}^d$. Given a budget $n$ of noisy queries to the function $f$ that can be allocated sequentially and adaptively, our aim is to construct an algorithm that returns a point $\hat x\in \bar{\mathcal X}$ such that $f(\hat x)$ is as small as possible. We provide a conceptually simple method inspired by the textbook center of gravity method, but adapted to the noisy and zeroth order setting. We prove that this method is such that the $f(\hat x) - \min_{x\in \bar{\mathcal X}} f(x)$ is of smaller order than $d^2/\sqrt{n}$ up to poly-logarithmic terms. We slightly improve upon existing literature, where to the best of our knowledge the best known rate is in [Lattimore, 2024] is of order $d^{2.5}/\sqrt{n}$, albeit for a more challenging problem. Our main contribution is however conceptual, as we believe that our algorithm and its analysis bring novel ideas and are significantly simpler than existing approaches.

• The paper introduces an innovative algorithm inspired by the center of gravity method to optimize functions using only noisy evaluations.
• The paper establishes regret bounds of order d²/√n up to polylog factors, demonstrating improved efficiency over existing approaches.
• The paper employs a smoothed functional approach that simplifies implementation and enhances robustness in practical noisy environments.

An Analytical Overview of a Novel Algorithm for Noisy, Convex, Zeroth-Order Optimization

The paper presents a novel approach to tackling the issue of noisy, convex, zeroth-order optimization of a function over a bounded convex set. The problem addressed is pivotal in optimization scenarios where derivative information is unavailable and function evaluations are subject to noise. This is often encountered in real-world applications such as hyperparameter tuning in machine learning, where gradient information can be expensive or infeasible to obtain.

Key Contributions

1. Algorithmic Innovation: The paper proposes an algorithm motivated by the center of gravity method, tailored for scenarios with noise and without derivative information. This adaptation is significant because traditional methods designed for noiseless scenarios or those relying on gradients struggle in settings where noise makes gradient estimation unreliable.
2. Improved Regret Bounds: A significant theoretical result in this work is the establishment of an upper bound on the regret of the proposed algorithm, shown to be of smaller order than the existing literature's bounds. Specifically, the regret is shown to be of order $d^2/\sqrt{n}$ up to polylogarithmic factors, where $d$ is the dimensionality of the space and $n$ is the budget of function evaluations. This slightly advances previous results, offering more efficient utilization of the evaluation budget.
3. Theoretical Simplicity: Despite addressing a complex problem, the proposed algorithm is conceptually simpler than existing methods. It employs a smoothed functional approach, leveraging proxies for the target function that can be approximated more accurately in noisy environments.

Theoretical and Practical Implications

The proposed algorithm's theoretical underpinning rests on estimating gradients through smoothed proxies rather than direct evaluations of $f$. This approach reduces the noise's impact on the optimization process, making the algorithm robust and efficient in practical noisy scenarios.

The algorithm’s simplicity and the clarity of its analysis are noteworthy because they lower the implementation barriers for practitioners needing efficient zeroth-order optimization tools. With regard to computational costs, the authors acknowledge that their algorithm involves recursive constructions which may involve higher computational overhead compared to non-adaptive methods. However, the logical structure and the efficiency in handling noise offer a compelling argument for its use in appropriate contexts, especially where noise is a significant factor.

Future Directions

This work opens several avenues for further research. One potential direction is refining the balance between algorithmic simplicity and theoretical efficiency, pushing further towards closing the gap between upper and lower bounds for minimax regret across different dimensions. Additionally, extending the current framework to accommodate different types of noise models or function classes, such as non-convex settings, could be a valuable expansion.

In stochastic and adversarial settings, the adaptability of the proposed algorithm suggests potential adaptations for managing cumulative regret, which remains a significant challenge in bandit settings.

Conclusion

Overall, the paper makes a meaningful contribution to optimization literature by proposing a simple yet effective algorithm for noisy, convex, zeroth-order settings. While the improvement in regret bounds is incremental, the conceptual simplicity and adaptability offered are of considerable value to both theorists and practitioners. This work could catalyze further developments in robust optimization strategies where derivative information is absent or unreliable.