On the Complexity of Bandit and Derivative-Free Stochastic Convex Optimization (1209.2388v3)

Abstract: The problem of stochastic convex optimization with bandit feedback (in the learning community) or without knowledge of gradients (in the optimization community) has received much attention in recent years, in the form of algorithms and performance upper bounds. However, much less is known about the inherent complexity of these problems, and there are few lower bounds in the literature, especially for nonlinear functions. In this paper, we investigate the attainable error/regret in the bandit and derivative-free settings, as a function of the dimension d and the available number of queries T. We provide a precise characterization of the attainable performance for strongly-convex and smooth functions, which also imply a non-trivial lower bound for more general problems. Moreover, we prove that in both the bandit and derivative-free setting, the required number of queries must scale at least quadratically with the dimension. Finally, we show that on the natural class of quadratic functions, it is possible to obtain a "fast" O(1/T) error rate in terms of T, under mild assumptions, even without having access to gradients. To the best of our knowledge, this is the first such rate in a derivative-free stochastic setting, and holds despite previous results which seem to imply the contrary.

• The paper derives a tight lower bound showing that for strongly-convex and smooth functions, error scales as Θ(√(d²/T)), highlighting key complexity challenges.
• It demonstrates a surprising O(1/T) error rate for quadratic functions in derivative-free settings, challenging previous assumptions in algorithm design.
• The study bridges theoretical bounds with practical algorithm strategies, providing benchmarks for future research on high-probability guarantees in non-gradient contexts.

Complexity of Bandit and Derivative-Free Stochastic Convex Optimization

The paper "On the Complexity of Bandit and Derivative-Free Stochastic Convex Optimization" by Ohad Shamir offers a comprehensive examination of the inherent complexity of stochastic convex optimization, particularly in scenarios where feedback is derived from bandit-like observations or in settings devoid of gradient information. Despite significant advancements in the development of algorithms with performance guarantees, a definitive understanding of the lower bounds of these problems remains elusive. The paper fills a gap by providing in-depth insights into the attainable error and regret characteristics within these optimization frameworks.

The primary focus of the paper is to characterize the optimization error as a function of the dimensionality $d$ and the number of queries $T$. The paper establishes that for strongly-convex and smooth functions, the attainable error or regret is denoted by $\Theta(\sqrt{d^2/T})$, which provides a new perspective on the problem complexity. Most notably, this result implies that the number of queries needed scales quadratically with the dimension, even within simpler optimization contexts. The paper further extends this analysis to quadratic functions, demonstrating a "fast" $O(1/T)$ error rate for derivative-free optimization, a significant finding given the absence of gradient information. This result is deemed novel, as previous understandings appeared to suggest that such a rate was unattainable in derivative-free settings.

The implications of these findings are profound in both theoretical and practical aspects. Practically, the results provide clear benchmarks for designing and evaluating optimization algorithms in derivative-free or bandit settings. Theoretically, the paper strengthens the foundations of understanding complexity in non-linear bandit problems, contributing a rare sharp characterization to this domain. Additionally, the paper examines other broad function classes and highlights gaps between optimization error and average regret, providing a necessary demarcation that is often obscured in linear settings.

Moreover, the paper explores specific algorithmic strategies like stochastic gradient descent adapted for derivative-free optimization to procure an $O(d^2/T)$ error bound under certain mild assumptions. The robustness of this algorithm is contrasted with lower bounds using innovative probabilistic and information-theoretic techniques that precisely indicate the difficulty of the problem under generic noise conditions.

Future research could extend the findings of this paper to explore bounds on error with high probability, rather than on expectation, paving the way for more robust theoretical guarantees. Additionally, despite discerning tight bounds for strongly-convex and smooth functions, the paper leaves open explorations into general convex function settings and strongly-convex non-smooth functions. These avenues could further refine the understanding of optimization complexity.

In summary, the paper represents a crucial step in quantifying the complexity of bandit and derivative-free stochastic convex optimization. It invites continued exploration into bridging theoretical bounds with practical algorithm design, fostering advancements in optimization methodologies that are versatile across various settings where derivative information is inaccessible.