Smooth Bilevel Programming

• Smooth bilevel programming is a hierarchical optimization framework where both upper and lower level functions are smooth, enabling standard nonlinear programming methods.
• It employs Wolfe duality to reformulate the bilevel structure into a single-level problem that restores constraint qualifications like MFCQ and alleviates degeneracy.
• Relaxation schemes and iterative algorithms in this approach enhance feasibility, improve computational efficiency, and outperform traditional MPEC reformulations.

Smooth bilevel programming refers to the class of bilevel optimization problems in which the involved functions (objectives and constraints at both levels) are smooth (at least continuously differentiable, often twice differentiable), and where solution methodologies seek to produce single-level, smooth reformulations suitable for the application of standard nonlinear programming (NLP) techniques. This field combines hierarchical decision making, variational analysis, and nonlinear optimization under nonconvex and degenerate constraints. The central challenge is obtaining reformulations and algorithms that retain or restore the regularity and tractability often lost due to intrinsic nonsmoothness or degeneracy in the hierarchical structure.

1. Fundamental Formulations and Degeneracy

Smooth bilevel programs arise in the following prototypical form:

$$(\mathrm{BP}) \quad \min_{x,y}\; F(x,y) \quad\text{s.t.}\; x \in X,\, y \in S(x)$$

where

$$S(x) = \arg\min_{y \in Y(x)} f(x,y),\quad Y(x) = \{ y : g(x,y) \leq 0,\, h(x,y) = 0 \}$$

with $F$ (upper-level objective) being $C^1$, and $f$, $g$, $h$ (lower-level objective and constraints) $C^2$ functions. The feasible region defined by $y \in S(x)$ is generally degenerate; even when all data are smooth, the feasible set is typically nonsmooth, and classical regularity conditions—especially the Mangasarian–Fromovitz Constraint Qualification (MFCQ)—fail at all feasible points for standard reformulations, such as the Mathematical Program with Equilibrium Constraints (MPEC) (Li et al., 2023).

2. Classical MPEC Reformulation and its Pathologies

The most prevalent approach to render (BP) a single-level problem is to use the KKT conditions of the lower level, yielding an MPEC:

$$\begin{aligned} (\mathrm{MPEC})\quad &\min_{x,y,u,v} F(x,y) \ &\text{s.t.}\;\; x \in X,\, h(x,y) = 0,\, g(x,y) \leq 0,\, u \geq 0,\ &\quad u^\top g(x,y) = 0,\ &\quad \nabla_y f(x,y) + \nabla_y g(x,y)u + \nabla_y h(x,y)v = 0 \end{aligned}$$

However, the complementarity conditions $$S(x) = \arg\min_{y \in Y(x)} f(x,y),\quad Y(x) = \{ y : g(x,y) \leq 0,\, h(x,y) = 0 \}$$0 always violate MFCQ: no direction decreases all active inequalities while remaining in the tangent space of the equalities, so classical NLP constraint qualifications are inapplicable. This precludes direct application of robust NLP theory (KKT, second-order conditions, sequential quadratic programming, etc.) and necessitates alternative reformulations (Li et al., 2023).

3. Smooth Reformulation via Wolfe Duality

A novel single-level reformulation leverages the Wolfe dual of the lower-level problem. For fixed $$S(x) = \arg\min_{y \in Y(x)} f(x,y),\quad Y(x) = \{ y : g(x,y) \leq 0,\, h(x,y) = 0 \}$$1, define the Lagrangian

$$S(x) = \arg\min_{y \in Y(x)} f(x,y),\quad Y(x) = \{ y : g(x,y) \leq 0,\, h(x,y) = 0 \}$$2

and consider the Wolfe dual program

$$S(x) = \arg\min_{y \in Y(x)} f(x,y),\quad Y(x) = \{ y : g(x,y) \leq 0,\, h(x,y) = 0 \}$$3

Under pseudoconvexity of $$S(x) = \arg\min_{y \in Y(x)} f(x,y),\quad Y(x) = \{ y : g(x,y) \leq 0,\, h(x,y) = 0 \}$$4 and Guignard CQ at a minimizer, strong duality holds: $$S(x) = \arg\min_{y \in Y(x)} f(x,y),\quad Y(x) = \{ y : g(x,y) \leq 0,\, h(x,y) = 0 \}$$5 Substituting this into the original (BP) yields the Wolfe-Dual Program (WDP) (Li et al., 2023): $$S(x) = \arg\min_{y \in Y(x)} f(x,y),\quad Y(x) = \{ y : g(x,y) \leq 0,\, h(x,y) = 0 \}$$6 The WDP is smooth except for the nonnegativity of $$S(x) = \arg\min_{y \in Y(x)} f(x,y),\quad Y(x) = \{ y : g(x,y) \leq 0,\, h(x,y) = 0 \}$$7, and all constraints are either equality or smooth inequality (with slack for $$S(x) = \arg\min_{y \in Y(x)} f(x,y),\quad Y(x) = \{ y : g(x,y) \leq 0,\, h(x,y) = 0 \}$$8).

4. Regularity and Constraint Qualification Restoration

Unlike MPEC reformulations, WDP typically restores MFCQ at feasible points. The crucial inequalities in WDP are (i) active only at optimality, and (ii) smooth, so the linear independence of active constraint gradients (and positive linear independence for inequality constraints) can generically be achieved. When the critical inequality $$S(x) = \arg\min_{y \in Y(x)} f(x,y),\quad Y(x) = \{ y : g(x,y) \leq 0,\, h(x,y) = 0 \}$$9 is inactive, the active constraint Jacobian is well-conditioned, allowing the use of standard first- and second-order optimality conditions and robust SQP algorithms (Li et al., 2023). Explicit examples demonstrate cases where WDP satisfies MFCQ while MPEC never does.

5. Relaxation and Algorithmic Schemes

The WDP can still encounter MFCQ failure if $F$0 is active at the solution. To address this, a relaxation scheme introduces a slack parameter $F$1: $F$2 For any fixed $F$3, the inequality is inactive at minimizers, ensuring MFCQ or the weaker CPLD (constant positive linear dependence), and making local theory and practical algorithms applicable. As $F$4, any accumulation point of WDP$F$5 minimizers solves the original WDP. Convergence is guaranteed by theorems in the Wolfe dual reformulation framework (Li et al., 2023).

A practical iterative algorithm (Algorithm 5.1) operates as follows:

• Solve the lower-level subproblem for $F$6.
• Initialize the relaxation problem at $F$7 and solve $F$8.
• Update $F$9 for $C^1$0, iterate until convergence. Numerical evidence shows this approach outperforms both direct MPEC and WDP in reliability and computational cost for a set of large randomly generated bilevel programs (Li et al., 2023).

6. Exact and Approximate Equivalence under Mild Assumptions

Global and local equivalence of the original (BP) and the WDP reformulation are established under mild regularity, specifically:

• Pseudoconvexity of the lower-level Lagrangian in $C^1$1.
• Guignard constraint qualification at lower-level optimum. Under these, the sets of global minimizers coincide (Theorem 3.1), and local minimizers map between (BP) and WDP in corresponding neighborhoods (Theorem 3.5) (Li et al., 2023). This ensures that convergence of the relaxation method to a solution of WDP suffices to yield a solution of the original bilevel program.

7. Practical Implication and Comparative Numerical Results

On extensive linear test sets (dimensions up to $C^1$2, $C^1$3), the direct WDP and MPEC approaches are similar in feasibility and objective quality, with WDP often faster when feasible. The relaxation-based WDP method, however, achieves higher feasibility (18/20 cases at tight tolerance vs 15/20 for MPEC relaxation), better or equal objective values in the majority of cases, and reduced CPU times. This positions the Wolfe-dual relaxation as a robust and efficient technique for large, smooth bilevel programs where constraint-qualification pathologies would otherwise preclude reliable single-level optimization (Li et al., 2023).

---

In summary, smooth bilevel programming via Wolfe duality provides a single-level, smooth reformulation amenable to robust NLP techniques. The restored regularity (MFCQ) at most feasible points, the existence of effective relaxation schemes, and provable equivalence under broad conditions constitute major advances over classical KKT-based representations. These developments allow practical, convergent algorithms for smooth bilevel problems that were previously intractable due to degeneracy in standard MPEC reformulations.