Bilevel Search Objective

• Bilevel search objective is a hierarchical optimization formulation that couples an upper-level decision with a nested lower-level response, resulting in a nonconvex, set-valued solution space.
• It is widely applied in hyperparameter tuning, neural architecture search, meta-learning, and multi-objective machine learning for modeling leader–follower dynamics.
• Algorithmic approaches include KKT-based reductions, first-order and surrogate methods, which tackle challenges like nondifferentiability and multiplicity of optimal lower-level solutions.

A bilevel search objective formalizes the optimization of a decision variable where feasibility and/or performance is tightly coupled to the optimal response of a nested, lower-level optimization problem. This arrangement arises in hierarchical decision-making, hyperparameter optimization, neural architecture search, meta-learning, multi-objective ML, game-theoretic models, planning, and other settings where one agent or process ("upper-leader") must anticipate or rely on the best response of a subordinate ("lower-follower") scenario. The bilevel search objective implicitly defines a solution set that is generally both highly nonconvex and constrained by the solution map of the inner-level problem, leading to unique analytical and algorithmic challenges (Pujara et al., 5 Nov 2025).

1. Canonical Mathematical Formulation

The general bilevel search objective is defined over coupled variables:

• $x \in X \subseteq \mathbb{R}^n$: upper-level (leader) variables,
• $y \in Y(x) \subseteq \mathbb{R}^m$: lower-level (follower) variables.

The objective for bilevel optimization is: $$\begin{aligned} &\min_{x \in X,\, y \in Y(x)}\; F(x,y) \ &\quad \textrm{s.t.} \quad y \in \arg\min_{y' \in Y(x)}\, f(x, y') \ &\phantom{\quad \textrm{s.t.}} G_p(x, y) \leq 0 \;\; \forall p, \;\; g_q(x, y) \leq 0 \;\; \forall q \end{aligned}$$ where $F$ is the upper-level objective, $f$ the lower-level objective, and $G_p$, $g_q$ encode upper- and lower-level constraints, respectively (Pujara et al., 5 Nov 2025). The inducible region consists of all $(x, y)$ such that $y$ is an optimal lower-level response.

For single-objective, unconstrained settings: $$\min_{x \in X} F(x, y^*(x)), \qquad y^*(x) \in \arg\min_{y \in Y(x)} f(x, y)$$ This compactly defines the hyper-objective $y \in Y(x) \subseteq \mathbb{R}^m$0 (Chen et al., 2023).

For multi-objective bilevel optimization: $y \in Y(x) \subseteq \mathbb{R}^m$1 with $y \in Y(x) \subseteq \mathbb{R}^m$2 vector-valued and $y \in Y(x) \subseteq \mathbb{R}^m$3 constituting the Pareto front of the LL problem (Wang et al., 2024, Wang et al., 2023).

2. Analytical Properties, Solution Concepts, and Notational Regimes

Bilevel search objectives are characterized by set-valued solution mappings $y \in Y(x) \subseteq \mathbb{R}^m$4 and can exhibit severe nonconvexity and nondifferentiability:

• Nonconvex feasible region: The response set $y \in Y(x) \subseteq \mathbb{R}^m$5 is often disconnected, and the inducible region is typically highly nonconvex.
• Nondifferentiability and multiple optima: When $y \in Y(x) \subseteq \mathbb{R}^m$6, $y \in Y(x) \subseteq \mathbb{R}^m$7 may be discontinuous. Leader–follower behavior must be specified:
  • Optimistic (strong): Leader assumes follower selects $y \in Y(x) \subseteq \mathbb{R}^m$8; most favorable.
  • Pessimistic (weak): Leader assumes $y \in Y(x) \subseteq \mathbb{R}^m$9; least favorable.
  • Extreme optimistic (in evolutionary literature): Follower may return partially feasible $$\begin{aligned} &\min_{x \in X,\, y \in Y(x)}\; F(x,y) \ &\quad \textrm{s.t.} \quad y \in \arg\min_{y' \in Y(x)}\, f(x, y') \ &\phantom{\quad \textrm{s.t.}} G_p(x, y) \leq 0 \;\; \forall p, \;\; g_q(x, y) \leq 0 \;\; \forall q \end{aligned}$$0 not strictly optimal but not worse on $$\begin{aligned} &\min_{x \in X,\, y \in Y(x)}\; F(x,y) \ &\quad \textrm{s.t.} \quad y \in \arg\min_{y' \in Y(x)}\, f(x, y') \ &\phantom{\quad \textrm{s.t.}} G_p(x, y) \leq 0 \;\; \forall p, \;\; g_q(x, y) \leq 0 \;\; \forall q \end{aligned}$$1 (Sharma, 2020).

In multi-objective contexts, both the UL and LL subproblems may be vector-valued; the feasible set is defined via lower-level and upper-level Pareto dorminance (Wang et al., 2023, Wang et al., 2024). Feasibility is tightly coupled to LL Pareto optimality.

3. Hyper-Objective, Hyper-Gradient, and Theoretical Barriers

The hyper-objective approach substitutes the LL optimum into the UL objective: $$\begin{aligned} &\min_{x \in X,\, y \in Y(x)}\; F(x,y) \ &\quad \textrm{s.t.} \quad y \in \arg\min_{y' \in Y(x)}\, f(x, y') \ &\phantom{\quad \textrm{s.t.}} G_p(x, y) \leq 0 \;\; \forall p, \;\; g_q(x, y) \leq 0 \;\; \forall q \end{aligned}$$2 Optimality or stationarity in $$\begin{aligned} &\min_{x \in X,\, y \in Y(x)}\; F(x,y) \ &\quad \textrm{s.t.} \quad y \in \arg\min_{y' \in Y(x)}\, f(x, y') \ &\phantom{\quad \textrm{s.t.}} G_p(x, y) \leq 0 \;\; \forall p, \;\; g_q(x, y) \leq 0 \;\; \forall q \end{aligned}$$3 hinges on the properties of $$\begin{aligned} &\min_{x \in X,\, y \in Y(x)}\; F(x,y) \ &\quad \textrm{s.t.} \quad y \in \arg\min_{y' \in Y(x)}\, f(x, y') \ &\phantom{\quad \textrm{s.t.}} G_p(x, y) \leq 0 \;\; \forall p, \;\; g_q(x, y) \leq 0 \;\; \forall q \end{aligned}$$4:

• Smooth regime, unique LL solution: If $$\begin{aligned} &\min_{x \in X,\, y \in Y(x)}\; F(x,y) \ &\quad \textrm{s.t.} \quad y \in \arg\min_{y' \in Y(x)}\, f(x, y') \ &\phantom{\quad \textrm{s.t.}} G_p(x, y) \leq 0 \;\; \forall p, \;\; g_q(x, y) \leq 0 \;\; \forall q \end{aligned}$$5 is strongly convex, $$\begin{aligned} &\min_{x \in X,\, y \in Y(x)}\; F(x,y) \ &\quad \textrm{s.t.} \quad y \in \arg\min_{y' \in Y(x)}\, f(x, y') \ &\phantom{\quad \textrm{s.t.}} G_p(x, y) \leq 0 \;\; \forall p, \;\; g_q(x, y) \leq 0 \;\; \forall q \end{aligned}$$6 is smooth in $$\begin{aligned} &\min_{x \in X,\, y \in Y(x)}\; F(x,y) \ &\quad \textrm{s.t.} \quad y \in \arg\min_{y' \in Y(x)}\, f(x, y') \ &\phantom{\quad \textrm{s.t.}} G_p(x, y) \leq 0 \;\; \forall p, \;\; g_q(x, y) \leq 0 \;\; \forall q \end{aligned}$$7 (implicit function theorem applies). The hyper-gradient is: $$\begin{aligned} &\min_{x \in X,\, y \in Y(x)}\; F(x,y) \ &\quad \textrm{s.t.} \quad y \in \arg\min_{y' \in Y(x)}\, f(x, y') \ &\phantom{\quad \textrm{s.t.}} G_p(x, y) \leq 0 \;\; \forall p, \;\; g_q(x, y) \leq 0 \;\; \forall q \end{aligned}$$8
• Nonconvex–convex regime: Only strict convexity in $$\begin{aligned} &\min_{x \in X,\, y \in Y(x)}\; F(x,y) \ &\quad \textrm{s.t.} \quad y \in \arg\min_{y' \in Y(x)}\, f(x, y') \ &\phantom{\quad \textrm{s.t.}} G_p(x, y) \leq 0 \;\; \forall p, \;\; g_q(x, y) \leq 0 \;\; \forall q \end{aligned}$$9 is assumed for $F$0. Recent hardness results demonstrate that for such cases, even finding a stationary point of $F$1 can be intractable for all zero-respecting, first-order algorithms. This is due to the inability of the algorithm to propagate gradient information through coordinates not yet "activated" at the LL (Chen et al., 2023).
• PL condition regime: When $F$2 satisfies a Polyak–Łojasiewicz (PL) condition, tractable rates are restored. Fully first-order algorithms achieve $F$3 (deterministic), $F$4 (partially stochastic), and $F$5 (fully stochastic) convergence for $F$6 (Chen et al., 2023).

4. Algorithmic Frameworks and Complexity

Bilevel search objectives require specialized algorithms that address LL feasibility and search nonconvex, set-valued induced regions. Key methodologies include:

Approach	Upper Level	Lower Level	Complexity/Rate	Applicability
KKT-based reduction	Single-level with MPCC	KKT system	Problem-dependent	LL convex, satisfies constraint qualifications (Pujara et al., 5 Nov 2025)
Dual/bisection, root-finding	Value function, root-finding	Equality constraint	$F$7	Convex–convex, composite (Jiang et al., 2024, Wang et al., 2024)
Fully first-order approximation (F²BA)	Gradient descent	PL or strongly convex	$F$8	Nonconvex–PL or strongly convex LL (Chen et al., 2023)
Stochastic approximation	SGD with inexact gradients	SGD with LL oracle	$F$9 (outer), $f$0 (inner)	Nonconvex, stochastic (Ghadimi et al., 2018)
Direct-search, derivative-free	Pattern/mads/poll search	Inexact LL oracle	$f$1	Black-box, smooth/nonsmooth (Diouane et al., 2023)
Surrogate/meta-models	Bayesian/ML surrogates	Nested/NNS/GP/NN	Sublinear regret	Black-box functions, costly LL (Chew et al., 4 Feb 2025)
Evolutionary (MOEA, Tabu, etc)	Pareto-based, crowding	Pareto MOO or scalarization	Empirical (problem-dependent)	Multi-objective, combinatorial (Wang et al., 2023, Chen et al., 2024, Wang et al., 2024)

First-order and surrogate-based methods exploit structure in the LL (convexity, smoothness, PL) to achieve near-optimal convergence. For black-box or very expensive LLs, Bayesian optimization and evolutionary approaches using surrogates are prevalent (Chew et al., 4 Feb 2025, Wang et al., 2023).

5. Multi-objective and Set-valued Bilevel Search

Multi-objective bilevel search objectives generalize the solution concept:

• For each $f$2, the LL problem admits a Pareto set $f$3 in place of a unique minimizer.
• The UL search must identify $f$4 and $f$5 such that $f$6 is Pareto-nondominated.
• This leads to a one-to-many search mapping and the need for surrogates, e.g., helper-variable neural networks parameterized by $f$7 to map $f$8 (Wang et al., 2024), and preference-based scalarizations selecting a unique $f$9 among LL Pareto solutions (Wang et al., 2023).
• Pareto set prediction and surrogate-based acceleration are essential due to the computational burden of evaluating all LL Pareto solutions for a given $G_p$0.

6. Best Practices, Limitations, and Current Research Frontiers

A rigorous approach to bilevel search objectives includes:

• Explicitly specifying follower behavior (optimistic vs. pessimistic) when LL solution is non-unique (Ustun et al., 2024, Sharma, 2020).
• Exploiting convexity, strong convexity, or PL geometry of the LL to enable tractable rates.
• Using surrogates or preference models in repeated or expensive LL settings.
• Validating all surrogates and approximation techniques, especially for multi-objective and combinatorial problems.

Significant challenges remain:

• The intractability of stationary point search under natural assumptions without additional structure, e.g., strict convexity is not always sufficient (Chen et al., 2023).
• Multi-objective LL regimes require explicit handling of Pareto sets and associated set-valued mapping difficulties (Wang et al., 2023).
• Black-box settings demand sample-efficient, robust algorithms (e.g., BILBO uses one-query per iteration policy with regret guarantees) (Chew et al., 4 Feb 2025).
• Automated hyperparameter search and ML meta-learning with bilevel structure must address instability from LL solution multiplicity; pessimistic formulations provide more robust generalization (Ustun et al., 2024).

7. Applications and Impact

Bilevel search objectives underpin core advances in:

• Neural architecture search using bilevel frameworks (e.g., BM-NAS, differentiable NAS) (Yin et al., 2021).
• Hyperparameter optimization, especially for robust learning under uncertainty or transfer settings (Ustun et al., 2024).
• Automated machine learning (AutoML) pipelines, where feature selection, transfer, and classification hyperparameters are jointly optimized in a multi-objective bilevel scheme (Chen et al., 2024).
• Multi-agent planning, hierarchical reinforcement learning, and cross-domain optimization, including planning with learned symbolic abstractions (Silver et al., 2022).

Recent theoretical advances delineate boundaries for algorithmic tractability, provide near-optimal complexity guarantees in the convex setting, and extend bilevel search methodology to black-box, time-varying, and multi-objective regimes (Chen et al., 2023, Jiang et al., 2024, Lin et al., 2023, Wang et al., 2024). This positions bilevel search objectives as a foundational paradigm for modern hierarchical and multi-level optimization across disciplines.