Stochastic Dynamic Barrier Perturbed Gradient Methods for Nonconvex Simple Bilevel Optimization

Abstract: We study stochastic simple bilevel optimization with smooth, possibly nonconvex upper- and lower-level objectives accessed only through stochastic gradient oracles. A key challenge is that the dual multiplier induced by the lower-level constraint may become unbounded near lower-level stationary points, invalidating bounded-dual analyses and destabilizing stochastic gradient estimates. To address this, we propose \emph{Stochastic Dynamic Barrier Perturbed Gradient} (SDBPG), a single-loop method that adaptively perturbs the dual formulation to regularize this degeneracy. The perturbation stabilizes the multiplier and yields controlled bias and variance even near the lower-level stationarity region. Under a mild rare-visit assumption, SDBPG finds an $(ε_f,ε_g)$-stationary point in $\mathcal{O}(\max{ε_f<sup>{-2},ε_g<sup>{-2}})$ iterations, with sample gradient complexities $\mathcal{O}(ε<sup>{-4})$ and $\mathcal{O}(ε<sup>{-6})$ for the upper- and lower-level objectives where $ε=\max{ε_f,ε_g}$. We further develop PR-SDBPG, a penalty-regularized variant that eliminates the rare-visit assumption, and VR-PR-SDBPG, which improves the resulting sample complexities entirely through variance reduction. To our knowledge, these are the first explicit $(ε_f,ε_g)$-stationarity guarantees for stochastic nonconvex-nonconvex simple bilevel optimization.

• The paper introduces a Stochastic Dynamic Barrier Perturbed Gradient (SDBPG) method that stabilizes dual variables to control bias and variance in nonconvex bilevel optimization.
• It provides explicit joint stationarity guarantees with detailed sample complexity bounds, overcoming limitations of classical methods in nonconvex settings.
• Empirical results on neural network unlearning benchmarks demonstrate superior sample and time efficiency compared to previous deterministic and projection-based approaches.

Stochastic Dynamic Barrier Perturbed Gradient Methods for Nonconvex Simple Bilevel Optimization

Problem Statement and Theoretical Challenges

Stochastic simple bilevel optimization constitutes a critical framework for emerging large-scale hierarchical decision processes, where the upper-level and lower-level objectives are accessed via stochastic gradient oracles and may both be fully nonconvex. The target problem formulation is

$$\min_{\mathbf{x} \in \mathbb{R}^n} \; f(\mathbf{x}) = \mathbb{E}\big[\tilde f(\mathbf{x}, \xi_f)\big] \quad \text{subject to} \quad \mathbf{x} \in \mathcal{X}_g^* := \arg\min_{\mathbf{z} \in \mathbb{R}^n} \; g(\mathbf{z}) = \mathbb{E}\big[\tilde g(\mathbf{z}, \xi_g)\big].$$

This structure, with implicit feasible set defined by the solution to a generally nonconvex lower-level problem, excludes projection operations and makes the use of standard primal-dual, Tikhonov regularization, and value-function reformulation techniques ineffective or intractable.

Existing deterministic methods (e.g., DBGD) introduce a dynamic dual multiplier to generate update directions mixing upper- and lower-level gradients, but in the nonconvex stochastic context, this multiplier $\lambda_k \propto 1/\|\nabla g(\mathbf{x}_k)\|$ becomes unbounded near stationary points of the lower level, invalidating classic bounded-dual analysis, and causing stochastic gradient directions to be both biased and arbitrarily high variance.

No prior work has offered explicit $(\epsilon_f, \epsilon_g)$-stationarity guarantees (i.e., joint control of both optimality metrics) with sample complexity bounds in this general stochastic nonconvex–nonconvex simple bilevel setting.

Algorithmic Advances

SDBPG: Stochastic DBGD with Dual Perturbation

The Stochastic Dynamic Barrier Perturbed Gradient method (SDBPG) introduces a crucial perturbation, regularizing the denominator of the dynamic dual variable with a sequence $\gamma_k$. This stabilization transforms the dual variable:

$$\lambda_{\gamma, k} = \max\left\{ \beta_k - \frac{\nabla g(\mathbf{x}_k)^\top \nabla f(\mathbf{x}_k)}{\|\nabla g(\mathbf{x}_k)\|^2 + \gamma_k},\; 0 \right\}$$

and, in the stochastic regime, constructs

$$\tilde\lambda_{\gamma, k} = \max\left\{\beta_k - \frac{\nabla \tilde g_k^\top \nabla\tilde f_k}{\|\nabla \tilde g_k\|^2+\gamma_k}, 0\right\}$$

using the same minibatch for both the numerator and denominator, which induces correlation and hence bias but tames the variance.

A single-loop update,

$$\mathbf{x}_{k+1} = \mathbf{x}_k - \eta_k \bigl( \nabla \tilde f_k + \tilde\lambda_{\gamma,k} \nabla \tilde g_k \bigr),$$

is then performed. The regularizer $\gamma_k$ allows explicit control of the bias-variance trade-off, enabling $(\epsilon_f, \epsilon_g)$-stationarity in $\mathcal{O}(\epsilon^{-4})$ (upper) and $\lambda_k \propto 1/\|\nabla g(\mathbf{x}_k)\|$0 (lower) gradient evaluations, provided a rare-visit assumption to problematic "bad regions" is maintained.

Figure 1: Sample and time efficiency of SDBPG (blue), PR-SDBPG (green), VR-PR-SDBPG (orange), and deterministic DDBPG (red) on TOFU (top) and MUSE-News (bottom).

PR-SDBPG and VR-PR-SDBPG: Penalty-Regularized and Variance-Reduced Forms

To eliminate the rare-visit assumption (which, while mild, can be hard to verify or enforce in practice), the penalty-regularized SDBPG (PR-SDBPG) reformulates the subproblem entirely:

$\lambda_k \propto 1/\|\nabla g(\mathbf{x}_k)\|$1

The corresponding multiplier is always bounded, and mini-batch sizes can be fixed. This approach guarantees $\lambda_k \propto 1/\|\nabla g(\mathbf{x}_k)\|$2-stationarity unconditionally, at elevated sample complexities: $\lambda_k \propto 1/\|\nabla g(\mathbf{x}_k)\|$3 and $\lambda_k \propto 1/\|\nabla g(\mathbf{x}_k)\|$4 for the upper and lower levels.

A further enhancement, VR-PR-SDBPG, applies STORM-type variance reduction to all stochastic estimators, reducing the sample complexities to $\lambda_k \propto 1/\|\nabla g(\mathbf{x}_k)\|$5 and $\lambda_k \propto 1/\|\nabla g(\mathbf{x}_k)\|$6.

Nonasymptotic Analysis: Joint Stationarity and Complexity

A significant theoretical contribution is a direct joint Lipschitz analysis of the stochastic update direction, treating the dual-multiplier and lower-level gradient as a coupled, noise-dependent map. This avoids artificial independence assumptions and yields tight nonasymptotic bounds for both bias and second moment. The authors derive explicit iteration and sample complexity results for each method:

Algorithm	Upper (f) Sample Complexity	Lower (g) Sample Complexity	Additional Assumptions
SDBPG	$\lambda_k \propto 1/\|\nabla g(\mathbf{x}_k)\|$7	$\lambda_k \propto 1/\|\nabla g(\mathbf{x}_k)\|$8	Rare-visit assumption
PR-SDBPG	$\lambda_k \propto 1/\|\nabla g(\mathbf{x}_k)\|$9	$(\epsilon_f, \epsilon_g)$0	None
VR-PR-SDBPG	$(\epsilon_f, \epsilon_g)$1	$(\epsilon_f, \epsilon_g)$2	None, variance reduction

The analysis also demonstrates that multi-loop inexact stochastic primal-dual alternatives are strictly inferior in overall sample complexity.

Empirical Validation

Experiments are conducted on nonconvex bilevel tasks in LLM unlearning with both TOFU and MUSE-News benchmarks, optimizing neural network parameters via LoRA on instruction-tuned LLMs. All metrics are normalized and compared as a function of processed example (not iteration count) and wall-clock time to reflect practical efficiency in large-scale settings:

• SDBPG consistently outperforms both deterministic (DDBPG) and penalty-regularized methods in sample and time efficiency, rapidly reducing both upper and lower stationarity measures.
• VR-PR-SDBPG closes the gap with SDBPG, confirming the theoretical benefit of variance reduction without recourse to the rare-visit assumption.

Figure 1: SDBPG, VR-PR-SDBPG, and PR-SDBPG achieve better normalized stationarity rates per sample and per time unit than DDBPG for both retain ($(\epsilon_f, \epsilon_g)$3) and joint ($(\epsilon_f, \epsilon_g)$4) criteria.

Implications and Future Directions

This work establishes the first single-loop stochastic methods to obtain explicit joint $(\epsilon_f, \epsilon_g)$5-stationarity in fully nonconvex simple bilevel settings with theoretical sample complexity guarantees, opening the door to practical optimization of large-scale, nonconvex bilevel problems, such as large-model unlearning, meta-learning, and nonconvex hyperparameter tuning. The stabilization mechanism for dual variables via perturbation or penalty, and variance-reduction techniques, are both shown to be essential for overcoming the instability and bias induced by stochastic gradients and degeneracy near lower-level stationarity.

Further research directions include tightening the bounds for the (penalty-)regularized algorithm to close the remaining log-factor gap; extending the approach to general bilevel or compositional objectives with cross-level coupling; and exploring adaptive, data-driven schedules for the penalty and perturbation parameters to minimize sample complexity in practice.

Conclusion

The proposed SDBPG, PR-SDBPG, and VR-PR-SDBPG methods provide a comprehensive set of single-loop, first-order stochastic algorithms for nonconvex simple bilevel optimization, each with explicit nonasymptotic guarantees on both optimality measures and sample complexities. The analysis establishes essential new tools for handling the instability of stochastic dynamic dual multipliers, and the experiments corroborate strong practical improvements in the core application of stochastic unlearning tasks in neural networks (2607.10957).