---
title: First-Order Bi-Level Optimization
url: https://www.emergentmind.com/topics/first-order-bi-level-optimization-foblo
type: topic
---

# First-Order Bi-Level Optimization

First-Order Bi-Level Optimization (FOBLO) denotes a family of methods for bilevel programs that rely on first-order or proximal information, rather than explicit Hessian inverses, full KKT-system solves, or exact implicit differentiation through the lower-level solution map. In the literature represented here, the term covers several distinct but related regimes: classical hierarchical convex selection problems, nonconvex–strongly-convex hyperparameter-style bilevel programs, constrained and coupled lower-level problems, online bilevel optimization, stochastic bilevel approximation, and domain-specific extensions in meta-learning, pruning, speech adaptation, and reinforcement learning [1702.03999], [2301.10945], [2406.10148], [2602.11665], [2601.16399]. A central unifying theme is that FOBLO replaces exact lower-level sensitivity computation by structural surrogates—fixed-point iterations, penalty formulations, primal-dual tracking, barrier smoothing, stochastic sampling, or first-order approximation—while preserving a meaningful link to the original bilevel objective.

## 1. Conceptual scope and problem classes

The canonical bilevel template in this literature is
$$
\min_x F(x,y^*(x))
\quad \text{s.t.}\quad
y^*(x)\in \arg\min_y G(x,y),
$$
but FOBLO is not restricted to this single form. One important branch studies hierarchical selection problems of the form
$$
\min_{x\in X^\ast}\omega(x),
\qquad
X^\ast=\arg\min_x\{f(x)+g(x)\},
$$
where the outer level selects a preferred point from a possibly nonunique lower-level solution set. The method in "A First Order Method for Solving Convex Bi-Level Optimization Problems" specializes Xu’s Sequential Averaging Method to this setting and emphasizes fixed-point selection over a convex composite lower problem rather than parameterized hypergradient computation [1702.03999].

A second branch studies optimistic bilevel optimization without the lower-level singleton assumption. "A Generic First-Order Algorithmic Framework for Bi-Level Programming Beyond Lower-Level Singleton" reformulates the objective as
$$
\varphi(x)=\inf_{y\in S(x)}F(x,y),
\qquad
S(x)=\arg\min_y f(x,y),
$$
and argues that lower-level descent alone can converge to a lower-level minimizer that is poor for the upper-level objective when \(S(x)\) is set-valued [2006.04045]. This perspective makes FOBLO not merely a matter of cheap hypergradients, but also of choosing the correct lower-level selector.

A third branch centers on modern hyperparameter-optimization-style formulations with a distinct upper variable and lower variable, often under strong convexity of the lower problem. "A Fully First-Order Method for Stochastic Bilevel Optimization" works in the unconstrained stochastic setting with unique \(y^*(x)\) induced by lower-level strong convexity [2301.10945], while "Accelerated Fully First-Order Methods for Bilevel and Minimax Optimization" studies nonconvex–strongly-convex bilevel problems and the minimax special case obtained by setting \(g(x,y)=-\bar f(x,y)\) [2405.00914].

Constraint structure further enlarges the taxonomy. "A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints" treats lower-level feasible sets of the form
$$
\mathcal Y(x)=\{y\in\mathcal Y:g^c(x,y)\le 0\},
$$
so the follower’s feasible region moves with the leader’s decision [2406.10148]. "A Single-Loop First-Order Algorithm for Linearly Constrained Bilevel Optimization" and "A Barrier-Metric First-Order Method for Linearly Constrained Bilevel Optimization" focus on linear inequality structure, respectively with coupled linear constraints \(Bx+Ay-b\le 0\) and a fixed polyhedral lower set \(Y=\{y:Ay\le b\}\) [2510.24710], [2605.11476]. "Fully First-Order Algorithms for Online Bilevel Optimization" adds temporal drift, replacing static optimality by local regret against time-varying bilevel objectives \(F_t(x)=f_t(x,y_t^\ast(x))\) [2602.11665].

This variety matters because FOBLO is not a single algorithmic recipe. It is a design space organized by lower-level geometry, constraint structure, stochasticity, and the meaning assigned to “first-order.”

## 2. What “first-order” means in FOBLO

In the strongest oracle sense, a fully first-order method uses only gradients of the upper and lower objectives, projected or proximal steps, and simple averaging or primal-dual updates. This is the meaning adopted by F\(^2\)SA, which uses only stochastic gradients of \(f\) and \(g\) and tracks two lower-level points \(y_k\) and \(z_k\) to approximate the gradient of a penalized surrogate \(L_\lambda^\ast(x)\), entirely avoiding Hessians, Jacobians, and Hessian-vector products [2301.10945]. The same oracle-level interpretation appears in RAF\(^2\)BA/PRAF\(^2\)BA, which optimize the surrogate
$$
\mathcal L_\lambda(x,y)
=
f(x,y)+\lambda\left(g(x,y)-\min_z g(x,z)\right)
$$
using only gradients of \(f\) and \(g\), plus accelerated gradient descent on lower subproblems [2405.00914].

In constrained settings, “first-order” often means that the method avoids differentiating through the lower-level KKT system, even when extra structure such as multipliers or barrier geometry enters the updates. BLOCC, for example, is “fully first-order” because it uses gradients of \(f\), \(g\), and \(g^c\), projected primal-dual steps, and no Hessian inverses or Hessian-vector products, even though the correct value-function sensitivity must include the multiplier term
$$
\nabla v(x)=\nabla_x g(x,y_g^\ast(x))
+
\langle \mu_g^\ast(x),\nabla_x g^c(x,y_g^\ast(x))\rangle
$$
in the coupled-constraint case [2406.10148]. The barrier-metric method is similarly first-order in the sense that it uses only \(\nabla_x f,\nabla_y f,\nabla_x g,\nabla_y g\), together with the explicit logarithmic barrier Hessian induced by known linear constraints rather than second derivatives of \(f\) or \(g\) [2605.11476].

A looser meaning appears in approximate hypergradient methods. "A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learning" introduces a “first-order approximation” inside a forward-mode-like recurrence through SGLD, but the method still uses Hessian-vector products such as
$$
\frac{\partial^2 f}{\partial \theta^2}v,\qquad
\frac{\partial^2 \log p}{\partial \theta^2}v,\qquad
\frac{\partial^2 \log p}{\partial \lambda\,\partial\theta}v.
$$
The paper therefore distinguishes computational first-order character from strict derivative-order purity [2410.10417]. A related warning appears in "Debiasing a First-order Heuristic for Approximate Bi-level Optimization": the popular FOM heuristic has constant memory because it drops the second-derivative terms in the exact ABLO hypergradient, but that simplification induces bias and can prevent convergence to a stationary point of the ABLO objective [2106.02487].

This distinction suggests that FOBLO is best understood as a spectrum. At one end are methods that are first-order in oracle usage; at the other are scalable first-order approximations that still retain some second-order structure internally.

## 3. Principal algorithmic paradigms

FOBLO methods differ mainly in how they replace exact lower-level sensitivity.

| Paradigm | Representative methods | Core mechanism |
|---|---|---|
| Fixed-point and averaging | BiG-SAM, BDA | Select or aggregate lower-level solutions using nonexpansive maps or combined upper/lower descent |
| Penalty and primal-dual reformulation | F\(^2\)SA, BLOCC, SFLCB, penalty methods | Replace the bilevel constraint by lower-level optimality-gap penalties and optimize a surrogate |
| Smoothing and geometry control | Moreau smoothing, barrier smoothing, stochastic Gibbs smoothing | Regularize the lower or outer problem to restore differentiability or robustness |

BiG-SAM is the cleanest fixed-point example. It rewrites the lower solution set as \(\operatorname{Fix}(T_t)\), where
$$
T_t(x)=\operatorname{prox}_{tg}(x-t\nabla f(x)),
$$
and averages a lower proximal-gradient step with a contraction generated by the upper objective, producing a single-loop first-order scheme for convex hierarchical selection [1702.03999]. BDA generalizes the idea of biased lower-level motion: for fixed \(x\), it updates
$$
y_{k+1}
=
y_k-\bigl(\alpha_k s_u\nabla_y F(x,y_k)+(1-\alpha_k)s_l\nabla_y f(x,y_k)\bigr),
$$
so the lower iterate is simultaneously driven toward lower-level optimality and upper-level preference. This is the paper’s mechanism for handling non-singleton lower solutions without assuming \(S(x)\) is single-valued [2006.04045].

Penalty reformulation is the dominant modern fully first-order strategy. In F\(^2\)SA, the surrogate \(L_\lambda^\ast(x)\) approximates the true bilevel objective and yields an \(x\)-gradient that depends only on \(\nabla_x f(x,y)\), \(\nabla_x g(x,y)\), and \(\nabla_x g(x,z)\), where \(y\) tracks the minimizer of the penalized lower problem and \(z\) tracks the true lower optimum [2301.10945]. "On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximation" extends this idea to smooth constrained, generally nonconvex bilevel problems by defining
$$
\psi_\sigma(x)
=
\frac{1}{\sigma}\left(
\min_{y\in Y}(\sigma f(x,y)+g(x,y))-\min_{z\in Y}g(x,z)
\right),
$$
and then optimizing a smoothed min-max representation with proximal-envelope subproblems [2309.01753]. BLOCC uses a primal-dual-assisted penalty formulation for coupled constraints rather than joint projection onto \(\{(x,y):g^c(x,y)\le 0\}\), while SFLCB introduces slack variables and an augmented Lagrangian to obtain a single-loop primal-dual gradient method for linearly constrained lower levels [2406.10148], [2510.24710].

Smoothing plays several distinct roles. Moreau smoothing allows BiG-SAM to handle nonsmooth strongly convex outer objectives through \(\operatorname{prox}_{s\omega}\) and the Moreau envelope \(M_{s\omega}\) [1702.03999]. Barrier smoothing replaces a polyhedrally constrained lower problem by the interior objective \(\psi_\mu(x,y)=g(x,y)+\mu\phi(y)\), after which a proxy-gradient method operates in local Dikin geometry rather than Euclidean geometry [2605.11476]. HPO-SGLD changes the underlying target even more radically by replacing the deterministic inner minimizer with a Gibbs distribution
$$
p_\tau(\theta\mid\lambda)
=
\frac{\exp(-\mathcal L_T(\lambda,\theta)/\tau)}{Z_\tau(\lambda)},
$$
so the outer problem becomes optimization of an expected validation loss under the induced inner distribution [2410.10417].

A final strand is approximation and debiasing. FOM in ABLO drops second-derivative terms entirely; UFOM restores unbiasedness by occasionally computing an exact correction with Bernoulli randomization, preserving constant memory in the inner-loop length \(r\) [2106.02487]. In speech adaptation, MAdaPT-FOBLO adopts a Reptile-style parameter-difference approximation for a heterogeneous-loss bilevel objective with self-supervised inner adaptation and supervised outer evaluation [2512.21204].

## 4. Theory: stationarity, approximation, and complexity

Theoretical guarantees in FOBLO are highly setting-dependent, and the performance criteria vary accordingly. In convex hierarchical selection, BiG-SAM proves convergence to the unique bilevel solution and an explicit \(O(1/k)\) bound in the lower-level objective value,
$$
\varphi(y^k)-\varphi(x_{mn}^\ast)\le
\frac{2C_{x_{mn}^\ast}^2(J+2)}{(k+1)(1-\beta)t},
$$
but the paper is explicit that this rate is not stated for the outer objective gap or a full bilevel optimality gap [1702.03999]. In non-singleton optimistic bilevel optimization, BDA proves convergence of minimizers of the surrogate value functions \(\varphi_K\) to minimizers of the optimistic value function \(\varphi\), rather than a standard hypergradient stationarity statement [2006.04045].

For stochastic unconstrained bilevel optimization, F\(^2\)SA gives finite-time convergence to an \(\epsilon\)-stationary point of the true bilevel objective after \(\widetilde O(\epsilon^{-7/2})\) iterations when both levels are noisy, \(\widetilde O(\epsilon^{-5/2})\) when only the upper level is noisy, and \(\widetilde O(\epsilon^{-3/2})\) in the deterministic setting. Its momentum-assisted variant improves the stochastic bounds to \(\widetilde O(\epsilon^{-5/2})\) and \(\widetilde O(\epsilon^{-2})\), while retaining \(\widetilde O(\epsilon^{-3/2})\) deterministically [2301.10945]. RAF\(^2\)BA/PRAF\(^2\)BA further show that accelerated fully first-order complexity is possible in nonconvex–strongly-convex bilevel optimization, achieving \(\widetilde O(\kappa^{3.25}\epsilon^{-1.75})\) gradient-query complexity for approximate first-order and second-order stationarity, while the minimax specialization recovers \(\widetilde O(\kappa^{1.75}\epsilon^{-1.75})\) [2405.00914].

Penalty and smoothing methods hinge on approximation theorems. The nonconvex penalty paper proves that \(\psi_\sigma\) is \(O(\sigma)\)-close to the original optimistic hyper-objective \(\psi\) in both value and derivative under proximal error-bound and local regularity assumptions, enabling the choice \(\sigma=O(\epsilon)\) when solving the surrogate [2309.01753]. The barrier-smoothed constrained method proves
$$
|F_\mu(x)-F(x)|\le C_F\sqrt{\mu},
$$
then establishes stationarity rates for the smoothed objective \(F_\mu\): \(\widetilde O(K^{-2/3})\) deterministically and \(\widetilde O(K^{-2/5})\) with upper-level-only stochastic noise [2605.11476]. SFLCB likewise quantifies closeness between \(\Phi_\delta\) and the original constrained hyper-objective \(\Phi\), including \(\|\nabla\Phi(x)-\nabla\Phi_\delta(x)\|\le O(\delta)\) under additional regularity, and improves prior double-loop complexity from \(O(\epsilon^{-3}\log(\epsilon^{-1}))\) to \(O(\epsilon^{-3})\) in its linearly constrained setting [2510.24710].

Coupled constraints introduce additional structural conditions. BLOCC proves convergence to approximate stationary points of its penalized reformulation using the projected gradient mapping \(G_\eta(x_t)\), with total complexity \(\widetilde O(\epsilon^{-5/2})\) in the general nonlinear-constraint case and \(\widetilde O(\epsilon^{-3/2})\) when the lower-level constraints are affine in \(y\) and the dual function becomes globally strongly concave [2406.10148]. Online FOBLO replaces stationarity of a single objective by local regret:
$$
\mathrm{Reg}_F(T)
=
\sum_{t=1}^T
\left\|
\mathcal G_{\mathcal X}(x_t,\nabla F_t(x_t),\gamma)
\right\|^2.
$$
F\(^2\)OBO achieves \(O(1+V_T+H_{2,T})\) regret, and AF\(^2\)OBO removes the dependence on lower-solution drift from the regret bound, attaining \(O(\sqrt T+V_T)\) [2602.11665].

A different theoretical caution comes from the ABLO debiasing paper. It proves an explicit FOM bias bound and constructs a family of examples where FOM-based SGD does not converge to a stationary point of the ABLO objective; UFOM restores unbiasedness while preserving constant memory in \(r\) [2106.02487]. This is one of the clearest demonstrations that not all “first-order” bilevel heuristics optimize the intended objective.

## 5. Applications and empirical domains

FOBLO has been used across a strikingly broad range of applications. Hyperparameter optimization and meta-learning are recurrent motifs: BDA evaluates on data hyper-cleaning and few-shot learning benchmarks [2006.04045]; F\(^2\)SA studies MNIST data hyper-cleaning and reports competitive or superior practical behavior relative to second-order baselines [2301.10945]; HPO-SGLD targets hyperparameter optimization, loss-function learning, MAML-style few-shot learning, invariance learning, and scales to 87M hyperparameters in the ViT-B-16 L1-regularizer setting [2410.10417]. Online bilevel optimization extends these concerns to drifting environments [2602.11665].

Constrained real-world planning problems are another major domain. BLOCC is motivated by lower-level coupled inequalities and is evaluated on support vector machine hyperparameter selection and transportation network design with real data from the city of Seville, avoiding joint projection onto coupled feasible sets that render prior approaches impractical at scale [2406.10148]. SFLCB studies both SVM hyperparameter optimization and transportation network design, reporting faster convergence or better upper-level utility than its baselines in those linearly constrained settings [2510.24710]. The barrier-metric method targets fixed polyhedral lower constraints, which suggests applications involving allocations, flows, simplex or box constraints, and equilibrium-like lower problems, although its guarantees are stated for the barrier-smoothed surrogate rather than directly for the nonsmoothed bilevel objective [2605.11476].

Several papers show how FOBLO ideas migrate into domain-specific learning systems. "A Regularized Actor-Critic Algorithm for Bi-Level Reinforcement Learning" formulates reward-parameterized reinforcement learning as a bilevel problem and proposes a single-loop, first-order, penalty-based actor-critic method with attenuating entropy regularization, proving convergence to a stationary point of the original unregularized bilevel RL objective and illustrating the framework on a GridWorld goal-position problem and happy tweet generation through RLHF [2601.16399]. "SpidR-Adapt" casts low-resource speech adaptation as a meta-learning bilevel problem and uses a Reptile-style FOBLO approximation inside MAdaPT; empirically it reports rapid gains in phonemic discriminability and spoken language modeling, including improvement over in-domain language models after training on less than 1h of target-language audio and over \(100\times\) more data-efficient than standard training [2512.21204]. Model pruning is also cast as a bilevel problem in BiP, where the paper argues that a bi-linear pruning structure makes the resulting bilevel problem as easy as first-order optimization and reports 2–7 times speedup over IMP at matched accuracy and sparsity [2210.04092].

Empirical results also illuminate the cost-structure claims that motivate FOBLO. BiG-SAM generally reaches stopping criteria much faster than the MNG baseline on inverse problems such as phillips, baart, and foxgood, which the paper attributes to MNG’s extra half-space optimization [1702.03999]. HPO-SGLD is reported to be more stable and less sensitive than IFT-based baselines in noisy or multimodal inner problems [2410.10417]. These findings do not establish a uniform empirical ordering across all settings, but they do show that first-order surrogate design can offset weaker worst-case rates by reducing per-iteration cost and approximation fragility.

## 6. Limitations, misconceptions, and open directions

A common misconception is that FOBLO is synonymous with differentiating through a unique lower-level argmin map. The literature here shows otherwise. Some of the earliest and clearest first-order bilevel methods solve hierarchical selection problems by fixed-point averaging rather than implicit differentiation [1702.03999]. Other works treat set-valued lower solutions directly and demonstrate that LL-only descent can be fundamentally wrong without a lower-level singleton assumption [2006.04045]. Constraint-sensitive methods show that once the feasible set depends on the upper variable, the correct first-order sensitivity may include dual multiplier terms, and naive unconstrained formulas can point in the wrong direction [2406.10148].

A second misconception is that any first-order approximation is harmless. The ABLO debiasing paper proves that the FOM heuristic can remain arbitrarily far from stationarity of the ABLO objective, which makes bias analysis indispensable for approximate FOBLO [2106.02487]. HPO-SGLD reinforces the taxonomic ambiguity: it is first-order-like in memory and oracle scaling, but not purely first-order in the strict derivative-order sense because it still uses Hessian-vector products [2410.10417]. The term “first-order” therefore must always be read against the paper’s oracle model.

The dominant limitations remain structural. Many strongest results still require lower-level strong convexity, PL-type or proximal error-bound conditions, LICQ, strict complementarity, bounded multipliers, fixed or linearly parameterized constraints, or full-row-rank conditions on lower-level constraint matrices [2301.10945], [2309.01753], [2406.10148], [2510.24710]. The barrier-metric method does not handle \(x\)-dependent lower constraints, and its guarantees are for the smoothed surrogate \(F_\mu\) rather than directly for the original nonsmoothed objective [2605.11476]. The bilevel RL method assumes finite state-action spaces, tabular softmax policies in theory, a special regularization-dependent PL condition, and yields a slow \(\widetilde O(\epsilon^{-10})\) sample complexity for the original unregularized problem [2601.16399].

Open directions are correspondingly clear in the cited work. BLOCC identifies relaxing lower-level strong convexity, removing local dual curvature assumptions, and designing more practical single-loop variants for general coupled constraints as natural next steps [2406.10148]. The RL paper notes that extending the analysis to \(x\)-dependent transition kernels is nontrivial because \(\nabla_x J_\tau(x,\pi)\) becomes harder to derive and estimate [2601.16399]. The broader literature represented here suggests a continuing tension between three goals: wider modeling scope, oracle purity, and sharp complexity. FOBLO advances whenever a method pushes one of these frontiers without collapsing the others.

Source: https://www.emergentmind.com/topics/first-order-bi-level-optimization-foblo