PDFO: A Cross-Platform Package for Powell's Derivative-Free Optimization Solvers

Abstract: The late Professor M. J. D. Powell devised five trust-region methods for derivative-free optimization, namely COBYLA, UOBYQA, NEWUOA, BOBYQA, and LINCOA. He carefully implemented them into publicly available solvers, renowned for their robustness and efficiency. However, the solvers were implemented in Fortran 77 and hence may not be easily accessible to some users. We introduce the PDFO package, which provides user-friendly Python and MATLAB interfaces to Powell's code. With PDFO, users of such languages can call Powell's Fortran solvers easily without dealing with the Fortran code. Moreover, PDFO includes bug fixes and improvements, which are particularly important for handling problems that suffer from ill-conditioning or failures of function evaluations. In addition to the PDFO package, we provide an overview of Powell's methods, sketching them from a uniform perspective, summarizing their main features, and highlighting the similarities and interconnections among them. We also present experiments on PDFO to demonstrate its stability under noise, tolerance of failures in function evaluations, and potential to solve certain hyperparameter optimization problems.