Line-Search SQP Methods in Optimization

• Line-search SQP is a class of Newton-like methods that linearize constraints and solve quadratic models to generate search directions.
• It employs globalization strategies like merit functions, Armijo tests, and filters to ensure convergence from remote starting points.
• Variants extend the approach to noisy, stochastic, and derivative-free settings, with modern implementations in frameworks like OpenSQP and Uno.

Line-search sequential quadratic programming (SQP) is a class of Newton-like methods for constrained nonlinear optimization in which each iteration computes a direction by solving a quadratic programming subproblem built from a local quadratic model and linearized constraints, then selects a step length through a one-dimensional globalization procedure. In the standard nonlinear program

$$\min_{x\in\mathbb R^n} f(x)\quad \text{s.t.}\quad h(x)=0,\;\; g(x)\le 0,$$

the target first-order conditions are the Karush–Kuhn–Tucker (KKT) conditions, and line-search SQP augments the basic SQP direction computation with a merit-function, filter, or funnel acceptance rule so that feasibility and stationarity are controlled from remote starting points while retaining fast local behavior near a solution (Zhang et al., 2 Apr 2026, Berahas et al., 2024).

1. Canonical formulation and quadratic subproblems

At the core of line-search SQP is the replacement of the nonlinear program by a sequence of quadratic models with linearized constraints. A standard mixed-constraint SQP subproblem at an iterate $x_t$ has the form

$$\begin{aligned} \min_{\Delta x\in\mathbb R^n}\quad & \langle \nabla f(x_t),\Delta x\rangle+\frac12 \Delta x^T H_t\Delta x \ \text{s.t.}\quad & h(x_t)+J_h(x_t)\Delta x=0,\qquad g(x_t)+J_g(x_t)\Delta x\le 0, \end{aligned}$$

while equality-constrained first-order variants replace $H_t$ by a scaled identity, such as $\frac1\eta I$, and proximal variants use $L_t I$ (Zhang et al., 2 Apr 2026, Bai et al., 2018).

This construction may be viewed as a local approximation of Newton’s method applied to the KKT system. In equality-constrained settings, the quadratic subproblem can be written through the saddle-point system

$$\begin{bmatrix} H_k & J_k^T\ J_k & 0 \end{bmatrix} \begin{bmatrix} d_k\ y_k \end{bmatrix} = -\begin{bmatrix} g_k\ c_k \end{bmatrix},$$

and line-search SQP uses the resulting direction $d_k$ while reserving the line search for step-length selection rather than direction computation (Berahas et al., 2020).

The choice of curvature model $H_k$ or $W_k$ is one of the main axes along which implementations differ. Some methods use the exact Hessian of the Lagrangian; some use positive-definite quasi-Newton approximations such as damped BFGS; others deliberately use regularized or first-order surrogates to lower cost or preserve well-posedness. OpenSQP, for example, uses a full-memory positive-definite BFGS approximation and resets the Hessian approximation to the identity matrix if numerical roundoff makes it indefinite (Joshy et al., 5 Dec 2025).

A recurring practical difficulty is inconsistency of the linearized constraints. Classical and modern variants address this in several ways: by solving augmented QPs with relaxation variables, by adding elastic variables, by slack-variable reformulations, or by switching to restoration phases. These mechanisms do not change the SQP paradigm; they modify the quadratic subproblem so that direction computation remains meaningful even when the current linearization is infeasible (Joshy et al., 5 Dec 2025, Kiessling et al., 2024).

2. Merit functions, Armijo tests, filters, and funnel globalization

The defining feature of line-search SQP is its globalization layer. For equality constraints, a classical choice is the exact $x_t$0 penalty merit function

$x_t$1

with sufficient decrease enforced by an Armijo condition of the form

$x_t$2

For mixed equality–inequality problems, a closely related exact-penalty merit function is

$x_t$3

and an Armijo backtracking test accepts a step when

$x_t$4

where $x_t$5 is a directional-derivative surrogate computed from the reduced model (Berahas et al., 2024, Zhang et al., 2 Apr 2026).

Other line-search SQP methods replace the nonsmooth $x_t$6 penalty with a smooth augmented Lagrangian. OpenSQP uses

$x_t$7

as the one-dimensional merit function for step selection and enforces the strong Wolfe conditions through MINPACK’s line search. If the strong-Wolfe search fails, it falls back to an inexact line search using an $x_t$8-penalty merit function (Joshy et al., 5 Dec 2025).

Filter and funnel methods provide an alternative globalization logic. A filter stores pairs in the $x_t$9 plane and accepts points that improve either feasibility or objective relative to all stored entries. A funnel method instead maintains a single nonincreasing upper bound

$$\begin{aligned} \min_{\Delta x\in\mathbb R^n}\quad & \langle \nabla f(x_t),\Delta x\rangle+\frac12 \Delta x^T H_t\Delta x \ \text{s.t.}\quad & h(x_t)+J_h(x_t)\Delta x=0,\qquad g(x_t)+J_g(x_t)\Delta x\le 0, \end{aligned}$$0

on constraint violation. In the unified funnel restoration framework, an accepted trial point must first lie inside the funnel; it is then classified either as an $$\begin{aligned} \min_{\Delta x\in\mathbb R^n}\quad & \langle \nabla f(x_t),\Delta x\rangle+\frac12 \Delta x^T H_t\Delta x \ \text{s.t.}\quad & h(x_t)+J_h(x_t)\Delta x=0,\qquad g(x_t)+J_g(x_t)\Delta x\le 0, \end{aligned}$$1-type iteration, based on predicted and actual objective decrease, or as an $$\begin{aligned} \min_{\Delta x\in\mathbb R^n}\quad & \langle \nabla f(x_t),\Delta x\rangle+\frac12 \Delta x^T H_t\Delta x \ \text{s.t.}\quad & h(x_t)+J_h(x_t)\Delta x=0,\qquad g(x_t)+J_g(x_t)\Delta x\le 0, \end{aligned}$$2-type iteration, based on sufficient reduction in infeasibility. If the SQP QP is infeasible, or if backtracking drives the step below $$\begin{aligned} \min_{\Delta x\in\mathbb R^n}\quad & \langle \nabla f(x_t),\Delta x\rangle+\frac12 \Delta x^T H_t\Delta x \ \text{s.t.}\quad & h(x_t)+J_h(x_t)\Delta x=0,\qquad g(x_t)+J_g(x_t)\Delta x\le 0, \end{aligned}$$3, the method switches to a restoration phase that temporarily solves a feasibility problem (Kiessling et al., 2024).

These globalization devices all serve the same structural role. They determine whether the quadratic-model step is safe to accept, how much feasibility pressure is imposed, and how aggressively the method can approach the fast local regime.

3. Local geometry, Maratos effect, and convergence rates

A central theoretical theme in line-search SQP is that the SQP step separates into tangential and normal components. For equality-constrained problems with feasible set

$$\begin{aligned} \min_{\Delta x\in\mathbb R^n}\quad & \langle \nabla f(x_t),\Delta x\rangle+\frac12 \Delta x^T H_t\Delta x \ \text{s.t.}\quad & h(x_t)+J_h(x_t)\Delta x=0,\qquad g(x_t)+J_g(x_t)\Delta x\le 0, \end{aligned}$$4

the first-order SQP update admits the closed form

$$\begin{aligned} \min_{\Delta x\in\mathbb R^n}\quad & \langle \nabla f(x_t),\Delta x\rangle+\frac12 \Delta x^T H_t\Delta x \ \text{s.t.}\quad & h(x_t)+J_h(x_t)\Delta x=0,\qquad g(x_t)+J_g(x_t)\Delta x\le 0, \end{aligned}$$5

where $$\begin{aligned} \min_{\Delta x\in\mathbb R^n}\quad & \langle \nabla f(x_t),\Delta x\rangle+\frac12 \Delta x^T H_t\Delta x \ \text{s.t.}\quad & h(x_t)+J_h(x_t)\Delta x=0,\qquad g(x_t)+J_g(x_t)\Delta x\le 0, \end{aligned}$$6 is the Euclidean gradient projected onto the tangent space and $$\begin{aligned} \min_{\Delta x\in\mathbb R^n}\quad & \langle \nabla f(x_t),\Delta x\rangle+\frac12 \Delta x^T H_t\Delta x \ \text{s.t.}\quad & h(x_t)+J_h(x_t)\Delta x=0,\qquad g(x_t)+J_g(x_t)\Delta x\le 0, \end{aligned}$$7 is the normal correction that cancels the linearized constraint residual. Near the manifold, this makes first-order SQP behave almost exactly like Riemannian gradient descent, with local linear convergence governed by the condition number $$\begin{aligned} \min_{\Delta x\in\mathbb R^n}\quad & \langle \nabla f(x_t),\Delta x\rangle+\frac12 \Delta x^T H_t\Delta x \ \text{s.t.}\quad & h(x_t)+J_h(x_t)\Delta x=0,\qquad g(x_t)+J_g(x_t)\Delta x\le 0, \end{aligned}$$8 of the Riemannian Hessian (Bai et al., 2018).

The main obstacle to clean Newton-like local behavior in classical line-search SQP is the Maratos effect. Even when the SQP direction is excellent, a standard first-order Armijo test may reject the unit step because nonlinear constraints introduce second-order mismatch between predicted and actual merit decrease. A modified line-search strategy addresses this by adding second-order terms involving both the objective Hessian and the constraint Hessians:

$$\begin{aligned} \min_{\Delta x\in\mathbb R^n}\quad & \langle \nabla f(x_t),\Delta x\rangle+\frac12 \Delta x^T H_t\Delta x \ \text{s.t.}\quad & h(x_t)+J_h(x_t)\Delta x=0,\qquad g(x_t)+J_g(x_t)\Delta x\le 0, \end{aligned}$$9

Under the stated assumptions, this yields global convergence, eventual unit-step acceptance, and asymptotic superlinear convergence, thereby restoring the familiar two-phase behavior of globally convergent Newton-type methods (Berahas et al., 2024).

A related remedy is explicit second-order correction. An earlier improved SQP algorithm combines the QP direction with directions obtained from systems of linear equations and computes a higher-order correction by solving another SLE with the same coefficient matrix. The method proves global and superlinear convergence without strict complementarity and uses the correction specifically to overcome the Maratos effect (Guo et al., 2012).

On manifolds, once the line search eventually accepts full steps and the curvature model is the exact Lagrangian Hessian, the local regime becomes genuinely Newtonian. The Riemannian sequential quadratic optimization method updates by

$H_t$0

uses an $H_t$1 penalty merit function with Armijo backtracking, and establishes local quadratic convergence under LICQ, SOSC, strict complementarity, exact Hessians, and eventual unit steps (Obara et al., 2020).

4. Noisy, stochastic, derivative-free, and black-box variants

Line-search SQP has been extended well beyond the deterministic smooth setting by modifying either the line-search condition, the quadratic model, or both.

Setting	Line-search mechanism	Reported behavior
Noisy equality constraints	Relaxed Armijo with $H_t$2 (Oztoprak et al., 2021)	Convergence to a noise-determined neighborhood; the relaxed line search always succeeds (Oztoprak et al., 2021)
Noisy inequality constraints	Relaxed QP feasibility plus relaxed backtracking with $H_t$3 (Oztoprak et al., 15 Apr 2026)	Infinitely many visits to feasibility/optimality critical regions, with $H_t$4 or $H_t$5 accuracy under regularity (Oztoprak et al., 15 Apr 2026)
Stochastic equality constraints	Adaptive stochastic line search on a differentiable exact augmented Lagrangian (Na et al., 2021)	Global almost sure convergence for non-adaptive and adaptive variants (Na et al., 2021)
Stochastic upper-$H_t$6 objectives	$H_t$7 merit plus bounded line-search parameters and adaptive sampling (Wang et al., 2023)	Subsequential convergence in expectation; KKT accumulation points with probability $H_t$8 under stated conditions (Wang et al., 2023)
Zeroth-order constrained optimization	Armijo line search on an exact-penalty merit after random-subspace SQP steps (Zhang et al., 2 Apr 2026)	High-probability convergence to first-order KKT points in an averaged sense (Zhang et al., 2 Apr 2026)
Bayesian black-box optimization	One-dimensional constrained Thompson sampling on $H_t$9 (Brunzema et al., 3 Feb 2026)	SQP-like local search under model uncertainty rather than classical Armijo/Wolfe globalization (Brunzema et al., 3 Feb 2026)

In noisy settings, the main pathology is false rejection of productive steps. Relaxing the Armijo test by a margin proportional to the known noise level prevents failure of the line search when function values fluctuate inside a noise band. For equality-constrained SQP with noisy function and derivative information, this yields iterates that eventually enter a critical region and return to it infinitely often; the size of that region worsens with noise levels and Jacobian conditioning (Oztoprak et al., 2021).

The inequality-constrained noise-tolerant method rSQP adds a second relaxation: it first solves a feasibility-relaxation problem to compute the smallest relaxation $\frac1\eta I$0 making the noisy linearized constraints consistent, then solves the QP with that same relaxation. Its line-search rule subtracts $\frac1\eta I$1 from the required model decrease. The analysis proves repeated entry into feasibility and optimality critical regions, with stronger $\frac1\eta I$2 bounds when multiplier regularity holds (Oztoprak et al., 15 Apr 2026).

Stochastic line-search SQP changes the source of inexactness. One branch uses a differentiable exact augmented Lagrangian

$\frac1\eta I$3

together with stochastic line search and adaptive sampling of derivatives. Another branch exploits upper-$\frac1\eta I$4 structure, where the nonsmooth objective admits the inequality

$\frac1\eta I$5

so a convex quadratic SQP subproblem can still be formed and globalized by an $\frac1\eta I$6 merit function (Na et al., 2021, Wang et al., 2023).

Derivative-free and black-box variants preserve the line-search-SQP template while replacing exact derivatives. Random-subspace zeroth-order SQP estimates projected gradients and Jacobians by two-point differences along the columns of a random orthonormal basis, solves a reduced SQP subproblem in dimension $\frac1\eta I$7, and uses Armijo backtracking on an exact-penalty merit function. BayeSQP goes further by replacing deterministic line search with a budgeted constrained Thompson-sampling selection on the one-dimensional segment $\frac1\eta I$8 (Zhang et al., 2 Apr 2026, Brunzema et al., 3 Feb 2026).

5. Structural generalizations

Line-search SQP has proved adaptable to problem structures that are not covered by the classical Euclidean nonlinear program.

On Riemannian manifolds, sequential quadratic optimization replaces the Euclidean step $\frac1\eta I$9 by a retraction $L_t I$0, solves the quadratic subproblem in the tangent space, and uses the manifold analogue of the exact $L_t I$1 penalty function

$L_t I$2

The method is globally convergent to KKT points and locally quadratically convergent under the exact-Hessian and unit-step assumptions (Obara et al., 2020).

For two-block nonconvex optimization with linear equality, inequality, and box constraints, monotone splitting SQP reformulates the inequalities with slack variables, decomposes the QP subproblem by an ADMM-style splitting, and applies Armijo line search to an augmented Lagrangian merit function. The resulting algorithm proves that every accumulation point is a KKT point, and the slack-variable block admits an explicit projection formula that does not increase computational cost (Jian et al., 2023).

In dynamic games and generalized Nash equilibrium computation, the underlying optimization target is no longer a single scalar objective. The SQP step is therefore built from the coupled KKT system of all agents, and globalization is driven by a merit function based on the norm of stacked stationarity residuals plus an $L_t I$3 penalty on shared-constraint violation:

$L_t I$4

A watchdog-style non-monotone line search and a decaying regularization scheme are used to stabilize the QP step; local linear convergence is proved, and racing experiments report improved solver robustness and success rates relative to comparison methods (Zhu et al., 2022).

For nonconvex sum-of-squares programs in nonlinear control, sequential quadratic sum-of-squares programming replaces standard nonlinear constraints by SOS cone constraints in coefficient space. Each SQP subproblem is a convex quadratic SOS program with linearized cone constraints, globalization is handled by a filter line-search, and feasibility restoration is itself an SOS problem. The implementation also includes second-order correction to mitigate the Maratos effect, and benchmarks report substantial reductions in iteration counts and computation time relative to coordinate-descent SOS heuristics (Olucak et al., 2 Feb 2026).

6. Implementations, software frameworks, and empirical practice

Modern line-search SQP research places substantial emphasis on modular software. OpenSQP implements a major-iteration SQP framework in Python with primal, dual, and slack updates, strong-Wolfe line search on a smooth augmented Lagrangian merit function, full-memory damped BFGS Hessian approximation, augmented-QP repair for infeasible linearizations, and fallback $L_t I$5-penalty inexact search. Its architecture makes merit functions, line-search procedures, Hessian approximations, and QP solvers independently swappable, and its standard configuration is reported to be competitive with SLSQP, SNOPT, and IPOPT on CUTEst (Joshy et al., 5 Dec 2025).

The open-source solver Uno implements a broader funnel-restoration framework that can globalize SQP by either line search or trust region. On 278 small CUTEst instances, funnel methods slightly outperform filter methods overall in terms of constraint and Hessian evaluations, while trust-region methods outperform line-search methods on the tested subset because the trust-region variants can exploit indefinite QPs and negative curvature whereas the line-search variants regularize the Hessian until positive definite (Kiessling et al., 2024).

A separate practical direction replaces expensive exact function evaluations by cheap models with certified error bounds. A generalized $L_t I$6-merit line-search SQP method with tunable-accuracy function approximations preserves the same first-order global convergence properties as classical $L_t I$7-merit SQP while using only objective and constraint models together with model-error bounds. In the Boussinesq boundary-control application, these models are projection-based reduced-order models of the PDE, and at $L_t I$8 the FOM-only method requires 105 expensive evaluations whereas the ROM-based method requires 7 (Grundvig et al., 8 Jul 2025).

Taken together, this literature suggests that line-search SQP is not a single algorithm but a family of constrained Newton-type methods unified by a one-dimensional acceptance mechanism. The subproblem may be full-space, reduced-space, random-subspace, tangent-space, split, or cone-constrained; the globalization device may be Armijo, strong Wolfe, non-monotone watchdog, filter, funnel, restoration, or probabilistic segment search. What remains invariant is the central SQP idea: linearize the constraints, solve a quadratic model for a candidate direction, and use the line-search layer to reconcile local curvature exploitation with global control of feasibility and stationarity.