C-GLISp: Preference-Based Global Optimization under Unknown Constraints with Applications to Controller Calibration (2106.05639v3)

Abstract: Preference-based global optimization algorithms minimize an unknown objective function only based on whether the function is better, worse, or similar for given pairs of candidate optimization vectors. Such optimization problems arise in many real-life examples, such as finding the optimal calibration of the parameters of a control law. The calibrator can judge whether a particular combination of parameters leads to a better, worse, or similar closed-loop performance. Often, the search for the optimal parameters is also subject to unknown constraints. For example, the vector of calibration parameters must not lead to closed-loop instability. This paper extends an active preference learning algorithm introduced recently by the authors to handle unknown constraints. The proposed method, called C-GLISp, looks for an optimizer of the problem only based on preferences expressed on pairs of candidate vectors, and on whether a given vector is reported feasible and/or satisfactory. C-GLISp learns a surrogate of the underlying objective function based on the expressed preferences, and a surrogate of the probability that a sample is feasible and/or satisfactory based on whether each of the tested vectors was judged as such. The surrogate functions are used iteratively to propose a new candidate vector to test and judge. Numerical benchmarks and a semi-automated control calibration task demonstrate the effectiveness of C-GLISp, showing that it can reach near-optimal solutions within a small number of iterations.