On Constraint Qualifications for MPECs with Applications to Bilevel Hyperparameter Optimization for Machine Learning

Abstract: Constraint qualifications for a Mathematical Program with Equilibrium Constraints (MPEC) are essential for analyzing stationarity properties and establishing convergence results. In this paper, we explore several classical MPEC constraint qualifications and clarify the relationships among them. We subsequently examine the behavior of these constraint qualifications in the context of a specific MPEC derived from bilevel hyperparameter optimization (BHO) for L1-loss support vector classification. In particular, for such an MPEC, we provide a complete characterization of the well-known MPEC linear independence constraint qualification (MPEC-LICQ), therefore, establishing conditions under which it holds or fails for our BHO for support vector machines.