Non-stochastic Best Arm Identification and Hyperparameter Optimization (1502.07943v1)

Abstract: Motivated by the task of hyperparameter optimization, we introduce the non-stochastic best-arm identification problem. Within the multi-armed bandit literature, the cumulative regret objective enjoys algorithms and analyses for both the non-stochastic and stochastic settings while to the best of our knowledge, the best-arm identification framework has only been considered in the stochastic setting. We introduce the non-stochastic setting under this framework, identify a known algorithm that is well-suited for this setting, and analyze its behavior. Next, by leveraging the iterative nature of standard machine learning algorithms, we cast hyperparameter optimization as an instance of non-stochastic best-arm identification, and empirically evaluate our proposed algorithm on this task. Our empirical results show that, by allocating more resources to promising hyperparameter settings, we typically achieve comparable test accuracies an order of magnitude faster than baseline methods.

• The paper extends best-arm identification from stochastic to non-stochastic settings, applying it to hyperparameter optimization.
• It adapts the Successive Halving algorithm for deterministic and adversarial loss scenarios, outperforming uniform allocation strategies.
• Empirical results in ridge regression and kernel SVMs demonstrate significant computational speedups, highlighting its practical efficacy.

Non-stochastic Best Arm Identification and Hyperparameter Optimization

The paper presents a rigorous examination of the non-stochastic best-arm identification problem within the context of multi-armed bandits, specifically applied to hyperparameter optimization. Unlike traditional approaches predominantly focused on stochastic settings, this work introduces and explores the non-stochastic variant where deterministic and possibly adversarial losses converge over time.

Key Contributions

• Non-stochastic Best-Arm Identification: The research extends best-arm identification from stochastic to non-stochastic settings. Existing bandit algorithms, like those minimizing cumulative regret, have already considered non-stochastic environments, but best-arm identification had not yet been fully addressed.
• Algorithm Selection and Adaptation: A known algorithm, Successive Halving, is identified as being particularly suitable for the non-stochastic framework. This selection leverages its ability to perform well under both fixed budget and certain non-stochastic conditions, making it robust across different settings.
• Performance Analysis: The paper provides a detailed theoretical analysis of Successive Halving, showing its capability to identify the best arm effectively under specific conditions of budget, arms, and convergence behavior. This is contrasted to the uniform allocation strategy, revealing both theoretical and practical advantages.

Empirical Evaluation and Results

Empirical evaluations leverage iterative learning methods, particularly focusing on hyperparameter optimization. By framing hyperparameter tuning as a non-stochastic best-arm problem, the authors demonstrate that allocating resources dynamically to more promising hyperparameters can achieve significant computational speedups, often by an order of magnitude, compared to baseline methods.

For example, in experimental setups involving ridge regression and kernel SVMs, the proposed approach outperformed traditional hyperparameter optimization methods in both iteration and wall-clock time, emphasizing its efficiency and applicability.

Implications and Future Research

The framework expands the applicability of bandit algorithms to problems where traditional i.i.d. assumptions are untenable, such as in hyperparameter optimization under non-smooth and volatile conditions. This potentially broadens the scope for similar applications in areas like feature selection and complex combinatorial optimizations lacking typical stochastic assurances.

Future research could explore:

• Dynamic Arm Selection: Adapting the pool of arms (hyperparameter configurations) dynamically rather than relying on a static selection could enhance efficiency further. Integrating principles from Bayesian optimization could offer a promising direction.
• Better Utilization of Arm-specific Convergence Rates: The paper discusses adaptivity to specific arm convergence rates. Developing methods that leverage knowledge of arm-specific behaviors could improve both theoretical bounds and practical outcomes.
• Pairwise Switching Costs: Considering costs associated with switching between arms could further improve the framework's applicability to high-dimensional or resource-constrained environments.

The non-stochastic setting presented in this paper stands as a robust and adaptable framework, illuminating new pathways in hyperparameter optimization and iterative algorithm analysis.