A Near-Optimal Algorithm for Convex Simple Bilevel Optimization under Weak Assumptions (2409.08948v1)

Abstract: Bilevel optimization provides a comprehensive framework that bridges single- and multi-objective optimization, encompassing various formulations, including standard nonlinear programs. This paper focuses on a specific class of bilevel optimization known as simple bilevel optimization. In these problems, the objective is to minimize a composite convex function over the optimal solution set of another composite convex minimization problem. By reformulating the simple bilevel problem as finding the left-most root of a nonlinear equation, we employ a bisection scheme to efficiently obtain a solution that is $\epsilon$-optimal for both the upper- and lower-level objectives. In each iteration, the bisection narrows down an interval by assessing the feasibility of a discriminating criterion. By introducing a novel dual approach and employing the Accelerated Proximal Gradient (APG) method, we demonstrate that each subproblem in the bisection scheme can be solved in ${\mathcal{O}}(\sqrt{(L_{g_1}+2D_z L_{f_1}+1)/\epsilon}|\log\epsilon|^2)$ oracle queries under weak assumptions. Here, $L_{f_1}$ and $L_{g_1}$ represent the Lipschitz constants of the gradients of the upper- and lower-level objectives' smooth components, and $D_z$ is the upper bound of the optimal multiplier of the subproblem. Considering the number of binary searches, the total complexity of our proposed method is ${\mathcal{O}}(\sqrt{(L_{g_1}+2D_z L_{f_1}+1)/\epsilon}|\log\epsilon|^3)$. Our method achieves near-optimal complexity results, comparable to those in unconstrained smooth or composite convex optimization when disregarding the logarithmic terms. Numerical experiments also demonstrate the superior performance of our method compared to the state-of-the-art.