Algorithm NCL for Constrained Optimization

• Algorithm NCL is a method that reformulates classical augmented Lagrangian techniques by introducing slack and regularization variables to maintain explicit nonlinear constraints.
• It robustly handles smooth, degenerate problems—including those with complementarity constraints—by ensuring full rank Jacobians and stable KKT systems.
• Implementations in Julia and on GPUs demonstrate improved scalability and performance, especially in large-scale power flow and MPCC applications.

Algorithm NCL, usually expanded as Nonlinearly Constrained augmented Lagrangian, is an augmented Lagrangian method for smooth constrained optimization that reformulates classical LANCELOT-style bound-constrained augmented-Lagrangian subproblems as a short sequence of nonlinear programs with explicit nonlinear constraints and auxiliary regularization variables. In the formulation emphasized in the Julia, GPU, and AC-SCOPF papers, NCL targets general smooth nonlinear programs, including cases where the original constraints do not satisfy the LICQ, and extends naturally to degenerate mathematical programs with complementarity constraints (MPCCs) arising in security-constrained AC optimal power flow (Ma et al., 2021, Montoison et al., 7 Oct 2025, Pacaud et al., 15 Oct 2025).

1. Definition, lineage, and scope

The basic optimization model treated by NCL is

$$\min_{x\in\mathbb{R}^n}\ \phi(x)\quad \text{s.t.}\quad c(x)=0,\ \ell \le x \le u,$$

with smooth objective $\phi$, smooth constraints $c$, and simple bounds. Inequality constraints are handled by introducing slack variables into $x$ (Ma et al., 2021).

NCL is explicitly presented as a reformulation of the LANCELOT augmented-Lagrangian method. Classical LANCELOT uses a sequence of bound-constrained augmented-Lagrangian subproblems, whereas NCL introduces extra variables and retains nonlinear constraints in each subproblem. This yields a sequence of nonlinearly constrained subproblems that can be solved by interior-point methods such as IPOPT and KNITRO, with warm starts between subproblems (Ma et al., 2021).

A concise chronology of the optimization literature around Algorithm NCL is as follows.

Paper	Contribution	Context
"A Julia implementation of Algorithm NCL for constrained optimization" (Ma et al., 2021)	Reformulates LANCELOT in Julia and solves NCL subproblems with IPOPT and KNITRO	General smooth constrained optimization, including non-LICQ models
"MadNCL: A GPU Implementation of Algorithm NCL for Large-Scale, Degenerate Nonlinear Programs" (Montoison et al., 7 Oct 2025)	Develops a GPU implementation with loop fusion, extrapolation, and stabilized or condensed KKT systems	Large-scale and degenerate NLPs, including MPCC constraints
"An Augmented Lagrangian Method on GPU for Security-Constrained AC Optimal Power Flow" (Pacaud et al., 15 Oct 2025)	Applies NCL as the core engine for corrective AC-SCOPF with complementarity constraints	AC-SCOPF and MPCC reformulations on GPU

A central structural fact is that NCL is designed for cases where the original problem may be degenerate. In the Julia formulation, the Jacobian of the NCL subproblem constraints is $$\begin{pmatrix}J(x) & I\end{pmatrix}$$, which always has full row rank because of the identity block. Thus the NCL subproblems satisfy LICQ even if the original problem does not (Ma et al., 2021). The later GPU work generalizes this viewpoint to large-scale and degenerate NLPs, including MPCC-type degeneracy (Montoison et al., 7 Oct 2025).

2. Augmented-Lagrangian subproblems and outer iteration

The defining NCL subproblem introduces a slack or regularization variable $r\in\mathbb{R}^m$ and solves

$$\min_{x,r}\ \phi(x)+y_k^\top r+\frac{\rho_k}{2}\|r\|^2 \quad\text{s.t.}\quad c(x)+r=0,\ \ell\le x\le u.$$

Here $y_k$ is the current multiplier estimate and $\rho_k>0$ is the penalty parameter (Montoison et al., 7 Oct 2025, Ma et al., 2021).

This is equivalent to the classical augmented-Lagrangian form because substituting $r=-c(x)$ recovers

$\phi$0

The distinction is algorithmic rather than variational: NCL keeps a nonlinear constraint in the subproblem instead of collapsing everything into a bound-constrained objective (Ma et al., 2021).

The outer iteration follows the usual augmented-Lagrangian pattern. In the general MadNCL presentation, if the residual is small enough, the multiplier update is performed in classical augmented-Lagrangian fashion; otherwise the penalty parameter is increased. For the generic NLP form, the update is written as

$\phi$1

In the MPCC/AC-SCOPF formulation the same logic appears in the relaxed variables $\phi$2 and $\phi$3, schematically as

$\phi$4

with $\phi$5 and $\phi$6 measuring complementarity relaxation (Montoison et al., 7 Oct 2025, Pacaud et al., 15 Oct 2025).

A persistent misconception is that NCL produces unconstrained augmented-Lagrangian subproblems. The AC-SCOPF paper states the opposite explicitly: NCL is “nonlinearly constrained” because it does not form completely unconstrained augmented-Lagrangian subproblems; each subproblem still has nonlinear constraints, but these constraints are relaxed by extra variables and penalized (Pacaud et al., 15 Oct 2025).

NCL also has a built-in infeasibility mechanism. In the MPCC formulation, if the original problem is infeasible, the penalty parameter diverges and the iterates converge to a stationary point of a feasibility problem that minimizes $\phi$7. This yields a “minimal violation” mode rather than requiring an ad hoc $\phi$8 feasibility penalty (Pacaud et al., 15 Oct 2025).

3. KKT structure, regularization, and degeneracy handling

The computational core of NCL is the Newton system for the primal-dual KKT equations of each subproblem. For the generic NLP version, MadNCL writes the barrier system as $\phi$9 in the variables $c$0, with barrier parameter $c$1, and computes Newton directions by solving the Jacobian system of $c$2 (Montoison et al., 7 Oct 2025).

After eliminating bound-multiplier variables, one obtains a regularized augmented KKT system whose stabilized form is

$c$3

where $c$4 is the barrier-regularized Hessian block, $c$5 is a diagonal regularization, and $c$6 with $c$7 (Montoison et al., 7 Oct 2025). Eliminating the dual step gives a condensed SPD system, denoted $c$8 in the paper, whose positive definiteness is equivalent to the desired inertia of the full KKT matrix (Montoison et al., 7 Oct 2025).

In the AC-SCOPF MPCC formulation, the Newton system is expressed as

$c$9

where $x$0 collects Hessian and barrier terms, $x$1 is the Jacobian of the constraints, and $x$2 contains the dual regularization induced by the augmented-Lagrangian term. The decisive feature is that $x$3 contains $x$4, so the dual block is nonzero and positive definite for positive $x$5. The paper identifies this dual regularization as the main driver of improved conditioning (Pacaud et al., 15 Oct 2025).

The degeneracy mechanisms addressed by NCL are described explicitly in the AC-SCOPF paper. First, when strict complementarity fails, the Jacobian block $x$6 may lose rank because rows associated with complementarity pairs tend toward zero; the nonzero $x$7 block keeps the saddle-point system nonsingular. Second, bilinear complementarity terms can increase the indefiniteness of $x$8, but they do not destroy positive definiteness of the reduced Hessian near the solution (Pacaud et al., 15 Oct 2025).

The generic GPU paper states the same structural point differently: the added variables $x$9 make the constraint Jacobian $$\begin{pmatrix}J(x) & I\end{pmatrix}$$0 full row rank regardless of degeneracy in $$\begin{pmatrix}J(x) & I\end{pmatrix}$$1, and the quadratic penalty on $$\begin{pmatrix}J(x) & I\end{pmatrix}$$2 yields a positive definite dual regularization in the Schur complement. This is why NCL is described as more robust than pure IPM or SQP in degenerate settings (Montoison et al., 7 Oct 2025).

4. MPCC formulation and AC-SCOPF specialization

The most detailed contemporary application of Algorithm NCL is corrective AC security-constrained optimal power flow in polar form. In that setting the AC-SCOPF is written as

$$\begin{pmatrix}J(x) & I\end{pmatrix}$$3

subject to base-case AC equations and inequalities, contingency-wise recourse equations and inequalities, and complementarity constraints

$$\begin{pmatrix}J(x) & I\end{pmatrix}$$4

where the complementarity models two corrective mechanisms: droop/AGC clipping of active power and PV/PQ switching via reactive limits (Pacaud et al., 15 Oct 2025).

The AC-SCOPF with complementarity is cast as an MPCC in “vertical form,” and then converted into an equivalent NLP-MPCC by replacing complementarity with a bilinear inequality

$$\begin{pmatrix}J(x) & I\end{pmatrix}$$5

combined with nonnegativity of $$\begin{pmatrix}J(x) & I\end{pmatrix}$$6. The paper emphasizes that this formulation is degenerate because the feasible set has empty relative interior and standard constraint qualifications fail (Pacaud et al., 15 Oct 2025).

NCL addresses this by introducing two sets of relaxation variables: $$\begin{pmatrix}J(x) & I\end{pmatrix}$$7 for the aggregated nonlinear constraints $$\begin{pmatrix}J(x) & I\end{pmatrix}$$8, and $$\begin{pmatrix}J(x) & I\end{pmatrix}$$9 for the complementarity bilinear inequalities. The inner subproblem is

$r\in\mathbb{R}^m$0

subject to

$r\in\mathbb{R}^m$1

The paper notes that $r\in\mathbb{R}^m$2 plays the role of a Scholtes-type relaxation parameter, but is endogenous: it is driven to zero by the augmented-Lagrangian penalty rather than chosen manually (Pacaud et al., 15 Oct 2025).

This specialization is also where NCL’s infeasibility behavior becomes operational. For contingency screening, solving the induced feasibility problem with NCL provides the scalar measure $r\in\mathbb{R}^m$3, which quantifies how infeasible a contingency is (Pacaud et al., 15 Oct 2025).

On the theoretical side, the AC-SCOPF paper states that under MPCC-LICQ and boundedness of $r\in\mathbb{R}^m$4, NCL converges to a strongly stationary solution of the MPCC, rather than merely a weakly stationary one (Pacaud et al., 15 Oct 2025).

5. Implementations, solver stacks, and GPU realization

The 2021 Julia implementation, distributed as NCL.jl, is built on the JuliaSmoothOptimizers stack. The principal abstractions are AbstractNLPModel, AmplModel, NCLModel, and NCLSolve, with solver backends provided by NLPModelsIpopt.jl and NLPModelsKnitro.jl (Ma et al., 2021). The implementation automatically converts a general smooth constrained model into an NCLModel, adds the $r\in\mathbb{R}^m$5-variables, updates $r\in\mathbb{R}^m$6 across outer iterations, and warm-starts each NC$r\in\mathbb{R}^m$7 subproblem. Warm-start policies are solver-specific: for IPOPT, warm_start_init_point = yes is activated for $r\in\mathbb{R}^m$8, and for KNITRO barrier parameters such as bar_initmu are decreased as outer iterations progress (Ma et al., 2021).

MadNCL generalizes this architecture to GPUs. It uses MadNLP as the primal-dual interior-point engine for each subproblem, ExaModels for GPU-resident model and derivative evaluation, and NVIDIA cuDSS for sparse $r\in\mathbb{R}^m$9 or Cholesky factorizations with static pivoting (Montoison et al., 7 Oct 2025). Two KKT formulations are implemented: a stabilized indefinite system $$\min_{x,r}\ \phi(x)+y_k^\top r+\frac{\rho_k}{2}\|r\|^2 \quad\text{s.t.}\quad c(x)+r=0,\ \ell\le x\le u.$$0 and a condensed SPD system $$\min_{x,r}\ \phi(x)+y_k^\top r+\frac{\rho_k}{2}\|r\|^2 \quad\text{s.t.}\quad c(x)+r=0,\ \ell\le x\le u.$$1. The stabilized form is solved with sparse $$\min_{x,r}\ \phi(x)+y_k^\top r+\frac{\rho_k}{2}\|r\|^2 \quad\text{s.t.}\quad c(x)+r=0,\ \ell\le x\le u.$$2 and a small pivot epsilon, while the condensed form is solved by sparse Cholesky (Montoison et al., 7 Oct 2025).

A further implementation change in MadNCL is loop fusion. Rather than fully solving each augmented-Lagrangian subproblem before updating multipliers or penalty parameters, MadNCL fuses outer ALM updates and inner IPM iterations into a single loop and supplements them with an extrapolation step. The extrapolation is accepted when the residual reduction satisfies

$$\min_{x,r}\ \phi(x)+y_k^\top r+\frac{\rho_k}{2}\|r\|^2 \quad\text{s.t.}\quad c(x)+r=0,\ \ell\le x\le u.$$3

which reduces the number of expensive full subproblem solves (Montoison et al., 7 Oct 2025).

In the AC-SCOPF implementation, the GPU-oriented advantages are especially pronounced. The paper attributes the performance of MadNCL to the combination of regularized Newton systems from NCL, pivoting-free factorization via cuDSS, and GPU-based derivative evaluation via ExaModels (Pacaud et al., 15 Oct 2025).

6. Numerical behavior, strengths, and limitations

The numerical record reported across the three papers is consistent on two points: NCL is primarily a robustness mechanism for degeneracy and infeasibility, and more recent implementations turn that robustness into scalable GPU performance.

In the Julia CUTEst study, plain KNITRO solved 126 of 166 constrained problems, whereas NCL/KNITRO solved 147. On nonlinear least-squares problems from CUTEst, the NCL reformulation solved 119 of 127 problems, compared with 101 for KNITRO’s dedicated nonlinear least-squares mode (Ma et al., 2021). On the tax-policy models that motivated the implementation, NCL with warm-started interior-point subsolvers was able to solve instances with extremely many nearly active nonlinear inequalities, whereas direct IPOPT attempts were reported to time out on NEOS (Ma et al., 2021).

The GPU paper extends this picture to much larger models. On the largest PGLIB OPF case 78484_epigrids, MadNCL–$$\min_{x,r}\ \phi(x)+y_k^\top r+\frac{\rho_k}{2}\|r\|^2 \quad\text{s.t.}\quad c(x)+r=0,\ \ell\le x\le u.$$4+cuDSS converged in about 54 seconds, compared with about 354 seconds for MadNLP+MA27 on CPU, and the paper reports GPU speedups up to about $$\min_{x,r}\ \phi(x)+y_k^\top r+\frac{\rho_k}{2}\|r\|^2 \quad\text{s.t.}\quad c(x)+r=0,\ \ell\le x\le u.$$5 versus MA27 (Montoison et al., 7 Oct 2025). On SCOPF MPCC instances, Ipopt and MadNLP are reported to fail or become stuck in feasibility restoration, while MadNCL solves all tested instances to tolerance $$\min_{x,r}\ \phi(x)+y_k^\top r+\frac{\rho_k}{2}\|r\|^2 \quad\text{s.t.}\quad c(x)+r=0,\ \ell\le x\le u.$$6 (Montoison et al., 7 Oct 2025).

The AC-SCOPF application sharpens the scale comparison. For the ACTIVSg500 network with up to 256 contingencies, corresponding to nearly 1 million variables, MadNCL-GPU solves the AC-SCOPF in under 3 minutes, while Knitro requires several hours. The abstract states more specifically that MadNCL can solve AC-SCOPF with 500 buses and 256 contingencies fully on the GPU in less than 3 minutes, whereas Knitro takes more than 3 hours to find an equivalent solution (Pacaud et al., 15 Oct 2025). For contingency screening with fixed base-case control, MadNCL-CPU is reported to be roughly $$\min_{x,r}\ \phi(x)+y_k^\top r+\frac{\rho_k}{2}\|r\|^2 \quad\text{s.t.}\quad c(x)+r=0,\ \ell\le x\le u.$$7 faster than Knitro for networks with more than 300 buses (Pacaud et al., 15 Oct 2025).

The principal strengths emphasized in the literature are therefore threefold. First, NCL handles difficult and degenerate problems, including infeasible cases and MPCC models with complementarity constraints. Second, the augmented-Lagrangian term regularizes the Newton systems in a way that preserves nonsingularity and controlled inertia. Third, that regularity makes sparse factorizations without dynamic pivoting feasible, which is decisive for GPU efficiency (Pacaud et al., 15 Oct 2025, Montoison et al., 7 Oct 2025).

The limitations are also explicit. In the AC-SCOPF paper, very large and heavily degenerate problems can suffer from ill-conditioning induced mainly by the PV/PQ complementarity reformulation, leading to many inertia corrections and line-search rejections, substantially larger iteration counts, and occasional failure to converge (Pacaud et al., 15 Oct 2025). In the generic GPU paper, the condensed system $$\min_{x,r}\ \phi(x)+y_k^\top r+\frac{\rho_k}{2}\|r\|^2 \quad\text{s.t.}\quad c(x)+r=0,\ \ell\le x\le u.$$8 is described as less robust than the stabilized system $$\min_{x,r}\ \phi(x)+y_k^\top r+\frac{\rho_k}{2}\|r\|^2 \quad\text{s.t.}\quad c(x)+r=0,\ \ell\le x\le u.$$9 on the largest SCOPF cases, and the rocket COPS instance is singled out as a case where degeneracy lies in the reduced Hessian, outside the scope of the augmented-Lagrangian regularization used there (Montoison et al., 7 Oct 2025). The Julia paper likewise notes that NCL is not uniformly faster than plain KNITRO on easy general problems; its advantage is concentrated on hard or degenerate instances (Ma et al., 2021).

Algorithm NCL is therefore best understood not merely as an augmented-Lagrangian variant, but as a specific reformulation strategy: by introducing explicit relaxation variables and preserving nonlinear constraints inside each subproblem, it converts difficult degenerate optimization problems into subproblems with better Jacobian rank properties and more regular KKT systems. In contemporary implementations, that structural regularity is what simultaneously supports stronger degeneracy robustness and efficient execution on modern GPU linear algebra stacks (Ma et al., 2021, Montoison et al., 7 Oct 2025, Pacaud et al., 15 Oct 2025).