Powell-Style Model-Based Derivative-Free Optimization with Complexity Guarantees

Abstract: We propose variants of model-based trust region derivative-free algorithms that are closest to methods initially proposed and implemented by Powell. These methods rely on low degree polynomial interpolation and carefully maintain geometry of the interpolation sets. We are able to derive complexity bounds for these methods that make them theoretically competitive to other derivative-free methods. Applying these methods in randomly generated subspaces recovers what we believe to be nearly tight complexity. This paper builds on recent results where complexity of a much simplified version of Powell's methods was derived. Here, we extend the analysis to fully incorporate Powell's geometry handling approach and conduct an extensive numerical comparison of the model-based trust region methods connecting practical and theoretical performance. We also extend the analysis of subspace model-based trust region methods to the case of noisy function evaluations.