NCL Method for Nonlinear Optimization

• NCL Method is a nonlinear optimization strategy that employs an augmented Lagrangian framework with slack variables to guarantee full rank constraints.
• It decomposes challenging problems into subproblems solved efficiently with warm-started interior-point or Newton-type solvers.
• Empirical results demonstrate robust convergence and improved performance on large-scale, degenerate problems, including GPU-accelerated applications.

The Nonlinearly Constrained Augmented Lagrangian (NCL) Method is a broad class of optimization algorithms for solving smooth, and more generally nonsmooth or nonconvex, nonlinear programming problems with nonlinear equality and/or inequality constraints. NCL methods are characterized by an outer augmented Lagrangian structure—regularizing constraint violations via a penalty and dual term—while solving a sequence of subproblems formulated to maintain favorable algebraic properties and convergence even when standard constraint qualifications such as LICQ fail. Modern NCL algorithms support a hierarchy of subproblem solvers (from first- to second-order, deterministic and randomized), and are robust to degeneracies, large-scale problem structure, and can be implemented efficiently on modern hardware including GPU architectures (Ma et al., 2021, Montoison et al., 7 Oct 2025, Pacaud et al., 15 Oct 2025, Hong et al., 2023).

1. Problem Classes and Formulation

NCL methods target general nonlinear optimization problems of the form

$$\min_{x\in\mathbb{R}^n} \;\; \varphi(x) \;\; \text{subject to} \;\; c(x) = 0, \; \ell \leq x \leq u,$$

with $\varphi: \mathbb{R}^n \to \mathbb{R}$ and $c: \mathbb{R}^n \to \mathbb{R}^m$ at least twice differentiable, and possibly extensions to nonsmooth or composite objectives via separable or indicator terms (Ma et al., 2021, Sahin et al., 2019, Zhu et al., 2020). Arbitrary nonlinear inequalities $a(x) \geq 0$ are recast as equalities by introducing explicit slack variables.

Crucially, the NCL framework imposes no requirement of linear independence among the constraint gradients (i.e., non-satisfaction of LICQ is allowed), a key distinction from classical SQP, filter- or IP-based approaches. This property enables robust handling of degeneracy and convergence even when constraints are not regular at solutions (Ma et al., 2021, Montoison et al., 7 Oct 2025).

2. Augmented Lagrangian Construction and NCL Subproblems

The canonical NCL augmented Lagrangian is

$$L(x, \lambda, \mu) = \varphi(x) + \lambda^T c(x) + \tfrac{1}{2} \mu \|c(x)\|^2$$

or, with sign convention $y = -\lambda$, $$L(x, y, \rho) = \varphi(x) - y^T c(x) + \frac{\rho}{2}\|c(x)\|^2$$.

Distinctively, NCL introduces explicit slack variables $r \in \mathbb{R}^m$, such that for outer iteration $k$ the subproblem is

$$\min_{x, r} \;\; \varphi(x) + y_k^T r + (\rho_k/2)\|r\|^2 \;\; \text{subject to} \;\; c(x) + r = 0,\;\; \ell \leq x \leq u.$$

This formulation ensures that, for each subproblem, the constraint Jacobian with respect to $\varphi: \mathbb{R}^n \to \mathbb{R}$0, namely $\varphi: \mathbb{R}^n \to \mathbb{R}$1, is always full row rank, guaranteeing that the subproblem itself satisfies LICQ regardless of the original problem's regularity (Ma et al., 2021, Montoison et al., 7 Oct 2025). As $\varphi: \mathbb{R}^n \to \mathbb{R}$2, the sequence $\varphi: \mathbb{R}^n \to \mathbb{R}$3 drives feasibility in the original constraints.

Each subproblem NC$\varphi: \mathbb{R}^n \to \mathbb{R}$4 is typically solved inexactly and efficiently using a warm-started interior-point or Newton-type solver, with the previous iterate's solution as the initial guess (Ma et al., 2021). Penalty and dual parameter management is performed adaptively, as detailed in Section 4.

3. Outer Iteration: Multiplier and Penalty Updates

The NCL outer loop maintains (i) primal sequence $\varphi: \mathbb{R}^n \to \mathbb{R}$5, (ii) dual variables $\varphi: \mathbb{R}^n \to \mathbb{R}$6, (iii) penalty sequence $\varphi: \mathbb{R}^n \to \mathbb{R}$7, and (iv) decreasing feasibility and optimality tolerances $\varphi: \mathbb{R}^n \to \mathbb{R}$8.

After solving NC$\varphi: \mathbb{R}^n \to \mathbb{R}$9, with $c: \mathbb{R}^n \to \mathbb{R}^m$0 retrieved from the subproblem solver:

• If $c: \mathbb{R}^n \to \mathbb{R}^m$1 (feasibility improved), update

$c: \mathbb{R}^n \to \mathbb{R}^m$2

• Otherwise, keep $c: \mathbb{R}^n \to \mathbb{R}^m$3 and tolerances unchanged, but increase

$c: \mathbb{R}^n \to \mathbb{R}^m$4

The algorithm terminates if $c: \mathbb{R}^n \to \mathbb{R}^m$5 (desired feasibility) or infeasibility is flagged when $c: \mathbb{R}^n \to \mathbb{R}^m$6. Warm starting, especially for interior-point solvers, is implemented by initializing the subproblem with previous optimal primal-dual variables (Ma et al., 2021).

4. Algorithmic Robustness: Degeneracy and Relation to LANCELOT

The NCL subproblem’s slack reformulation and penalty update echo the early LANCELOT method [Conn, Gould, Toint: 1991–92], but NCL operates with a small number (typically $c: \mathbb{R}^n \to \mathbb{R}^m$7) of outer iterations, each leveraging efficient warm-started nonlinear programming solvers.

The reformulation of $c: \mathbb{R}^n \to \mathbb{R}^m$8 as $c: \mathbb{R}^n \to \mathbb{R}^m$9 ensures that the Jacobian $a(x) \geq 0$0 is always full row rank, even when the original $a(x) \geq 0$1 is rank deficient. As a consequence, the interior-point or other Newton-based subproblem solvers are shielded from degeneracies that hinder standard optimization techniques (Ma et al., 2021, Montoison et al., 7 Oct 2025). This property is particularly advantageous on large-scale, degenerate, or ill-conditioned problems, as shown in practical implementations on tax policy models and large CUTEst instances.

5. High-Level Pseudocode and Implementation

A high-level pseudocode capturing the core of Algorithm NCL is as follows:

1. Initialize $a(x) \geq 0$2, $a(x) \geq 0$3, feasibility and optimality tolerances $a(x) \geq 0$4.
2. For $a(x) \geq 0$5 until convergence: a. Form and solve NC$a(x) \geq 0$6 in $a(x) \geq 0$7 to (approximate) KKT optimality with current tolerances. b. Retrieve $a(x) \geq 0$8. c. Update multipliers and penalty according to the feasibility rule above. d. Check stopping criteria.

In practical software, e.g. NCL.jl (Julia), this is encapsulated in a wrapper around a general nonlinear programming interface, so parameter and slack updates are highly localized, and outer iterations invoke IPOPT, KNITRO, or other solvers with warm-start support (Ma et al., 2021).

6. Numerical Results and Performance Characteristics

Empirical testing of NCL on high-dimensional tax models (570,000 constraints, 1,500 variables), nonlinear least-squares, and diverse CUTEst test instances demonstrates:

• Consistent converge in $a(x) \geq 0$910 major NCL iterations, reducing solver times to minutes or less on large-scale problems.
• Superior robustness: on testing, NCL/KNITRO solved 147 out of 166 large CUTEst problems (vs. 126 for KNITRO alone). For nonlinear least-squares, NCL (with a single NC$$L(x, \lambda, \mu) = \varphi(x) + \lambda^T c(x) + \tfrac{1}{2} \mu \|c(x)\|^2$$0 solve) solved 119/127 problems (vs. 101 for native methods).
• Competitive performance: Despite Hessian evaluation, NCL’s subproblem structure, warm starting, and adequacy for modern IPM solvers yield run-times comparable to or better than direct approaches (Ma et al., 2021).

A summary table from the implementation context:

Test Suite	NCL/KNITRO Solved	KNITRO Alone Solved
Large CUTEst NLPs	147/166	126/166
Nonlinear least-squares	119/127	101/127

These results illustrate that, by decomposing a challenging (possibly degenerate) original problem into a streamlined sequence of subproblems with LICQ guaranteed, NCL improves both the reliability and numerical efficiency of large-scale nonlinear optimization.

7. Extensions and Impact

The NCL paradigm has been extended and implemented on modern platforms, notably on GPU architectures in MadNCL (Montoison et al., 7 Oct 2025, Pacaud et al., 15 Oct 2025). Here, the regularizing effect of the quadratic penalty in the subproblem allows for the construction of stabilized or condensed KKT systems solvable by efficient, pivoting-free LDL$$L(x, \lambda, \mu) = \varphi(x) + \lambda^T c(x) + \tfrac{1}{2} \mu \|c(x)\|^2$$1 factorizations (e.g., NVIDIA cuDSS). Fused NCL-inside-IPM implementations enable rapid solution of extremely large-scale, degenerate, or MPCC-constrained problems, often achieving 10–18$$L(x, \lambda, \mu) = \varphi(x) + \lambda^T c(x) + \tfrac{1}{2} \mu \|c(x)\|^2$$2 speedups over CPU-based solvers and handling problem classes (e.g., with complementarity constraints) that defeat standard IPM and SQP routines (Montoison et al., 7 Oct 2025, Pacaud et al., 15 Oct 2025).

In summary, the Nonlinearly Constrained Augmented Lagrangian (NCL) method is a versatile, theoretically robust, and empirically effective architecture for nonlinear constrained optimization, suitable for a spectrum of smooth and degenerate problems, with scalable implementations available for large-scale and high-dimensional applications (Ma et al., 2021, Montoison et al., 7 Oct 2025).