---
title: 'MadNCL: Robust Augmented-Lagrangian Solver'
url: https://www.emergentmind.com/topics/madncl
type: topic
---

# MadNCL: Robust Augmented-Lagrangian Solver

MadNCL denotes implementations of Algorithm NCL, a nonlinearly constrained augmented-Lagrangian method for smooth constrained optimization. In the 2021 Julia work, the implementation is called **NCL.jl**, and the paper’s “MadNCL” wording refers to that Julia codebase; in 2025, **MadNCL** designates a GPU implementation of Algorithm NCL and, in a closely related power-systems study, a GPU-accelerated solver for security-constrained AC optimal power flow. Across these usages, the common core is the solution of large-scale nonlinear programs—especially degenerate ones in which standard constraint qualifications such as LICQ or MFCQ may fail—by reformulating augmented-Lagrangian subproblems with auxiliary residual variables and solving the resulting nonlinearly constrained subproblems by interior-point methods [2101.02164; 2510.05885; 2510.13333].

## 1. Naming, lineage, and scope

The name **MadNCL** is attached to a family of NCL-based implementations rather than to a single invariant software artifact. The 2021 paper presents a Julia implementation of Algorithm NCL under the package name **NCL.jl**, with the “MadNCL” wording referring to that implementation. The 2025 GPU paper uses **MadNCL** for a GPU implementation of Algorithm NCL for large-scale, degenerate nonlinear programs. The 2025 AC-SCOPF paper presents **MadNCL.jl** as a GPU-accelerated augmented-Lagrangian solver specialized to security-constrained AC optimal power flow in polar form [2101.02164; 2510.05885; 2510.13333].

| Paper | Implementation/name | Reported focus |
|---|---|---|
| [2101.02164] | NCL.jl | Julia implementation of Algorithm NCL for smooth constrained optimization, including LICQ-violating cases |
| [2510.05885] | MadNCL | GPU implementation of Algorithm NCL for large-scale, degenerate nonlinear programs |
| [2510.13333] | MadNCL.jl | GPU-accelerated NCL solver for large-scale corrective AC-SCOPF with complementarity constraints |

Algorithmically, the lineage runs through the classic **LANCELOT** augmented-Lagrangian framework. The 2021 formulation states that Algorithm NCL is equivalent to the LANCELOT augmented Lagrangian method, but recasts it as a short sequence of nonlinearly constrained subproblems that can be handled efficiently by **IPOPT** and **KNITRO** with warm starts [2101.02164]. The later GPU work preserves this augmented-Lagrangian structure while redesigning the KKT linear algebra for sparse GPU execution [2510.05885].

A common misconception is to treat MadNCL as merely an interior-point solver. The cited works instead present it as an augmented-Lagrangian method whose subproblems are solved by interior-point techniques. This distinction is central because the robustness claims are tied to the outer augmented-Lagrangian mechanism, not to replacing one IPM by another [2510.05885].

## 2. Core optimization model and NCL reformulation

The baseline problem class is the smooth constrained program
\[
\min_x \ \phi(x) \qquad \text{subject to} \qquad c(x)=0,\ \ell \le x \le u,
\]
where \(\phi\) and \(c\) are twice continuously differentiable, and first and second derivatives are available [2101.02164; 2510.05885]. The key pathological regime is constraint degeneracy. In the 2021 presentation, if \(J(x)\) is the constraint Jacobian, LICQ requires \(\operatorname{rank}(J(x))=m\) at the solution; many standard NLP solvers have difficulty when this fails [2101.02164].

The classical augmented-Lagrangian subproblem minimizes
\[
\phi(x) - y_k^T c(x) + \tfrac12 \rho_k \|c(x)\|^2
\]
over the bounds \(\ell \le x \le u\). Algorithm NCL introduces an auxiliary variable \(r\equiv -c(x)\) and instead solves
\[
\min_{x,r}\ \phi(x) + y_k^T r + \tfrac12 \rho_k \|r\|^2
\quad \text{subject to} \quad
c(x)+r=0,\ \ell \le x \le u.
\]
This reformulation is mathematically equivalent to the classical ALM subproblem, but numerically advantageous because the subproblem constraint Jacobian becomes
\[
[J(x)\ \ I],
\]
which is structurally full row rank even if the original Jacobian \(J(x)\) is rank deficient [2101.02164; 2510.05885].

That structural property explains the method’s role in degenerate optimization. Interior-point solvers such as IPOPT and KNITRO can exploit the regularized subproblem more stably than when applied directly to the original model. The 2025 GPU paper makes the same point in different language: the added variables \(r\) regularize the constraint system and make the constraints linearly independent, while the quadratic penalty acts as a dual regularizer [2510.05885].

This suggests that the defining idea of MadNCL is not the use of GPUs, Julia, or a specific linear solver, but the NCL transformation itself: a residual-variable augmentation that restores a favorable constraint structure without changing the underlying optimization problem.

## 3. Major-iteration policy, stopping rules, and warm starts

In the 2021 implementation, each major iteration solves an NCL subproblem approximately and returns
\[
(x_k^*, r_k^*, y_k^*, z_k^*),
\]
where \(z_k^*\) are bound multipliers [2101.02164]. The major-iteration logic is driven by feasibility. With
\[
\eta_0 = 10,\quad \omega_0 = 10,\quad \rho_0 = 100,\quad \mu_0 = 0.1,
\]
the implementation uses \(\eta_k\) as feasibility tolerance, \(\omega_k\) as optimality tolerance, \(\rho_k\) as penalty parameter, and \(\mu_0\) as the initial barrier parameter for IPOPT or KNITRO [2101.02164].

If
\[
\|r_k^*\| \le \max(\eta_k,\eta^*),
\]
the method updates the multipliers and tightens the subproblem tolerances:
\[
y_{k+1} = y_k - \rho_k r_k^*, \qquad
\eta_{k+1}=\eta_k/10,\qquad
\omega_{k+1}=\omega_k/10,\qquad
\rho_{k+1}=\rho_k.
\]
Otherwise it keeps \(y_k\) fixed and increases the penalty:
\[
y_{k+1}=y_k,\qquad
\eta_{k+1}=\eta_k,\qquad
\omega_{k+1}=\omega_k,\qquad
\rho_{k+1}=\min(10\rho_k,\rho^*),
\]
with \(\rho^*=10^{12}\) in the implementation [2101.02164]. A natural stopping test is
\[
\|r_k^*\|_\infty \le \eta^*,
\]
since \(r=0\) makes the subproblem equivalent to the original constrained problem [2101.02164].

The literature uses two sign conventions for the multiplier update. The 2021 paper writes \(y_{k+1}=y_k-\rho_k r_k^*\), while the 2025 GPU paper gives the equivalent convention \(y_{k+1}\gets y_k+\rho_k r_{k+1}\) because its outer update is written in terms of \(c(x_{k+1})\) rather than \(r\) [2101.02164; 2510.05885]. This is a notational difference, not an algorithmic one.

Warm-starting is treated as essential rather than optional. For subproblem \(NC_k\), the 2021 solver is initialized from the previous solution \((x_k^*, r_k^*, y_k^*, z_k^*)\). For **IPOPT**, the implementation uses `warm_start_init_point=yes` for \(k\ge 2\), with `mu_init` reduced through the sequence \(10^{-4},10^{-5},10^{-6},10^{-7},10^{-8}\). For **KNITRO**, it uses `algorithm=1` for \(k=1\), and for \(k\ge 2\) employs `bar_directinterval=0`, `bar_initpt=2`, `bar_murule=1`, together with analogous reductions of `bar_initmu` and `bar_slackboundpush` [2101.02164].

The practical consequences are explicit in the reported experiments: there are typically only about **10 major iterations**; warm starts make later subproblems cheaper; and gradually tightening \(\eta_k,\omega_k\) from \(10^{-2}\) to \(10^{-6}\) saves work [2101.02164].

## 4. GPU reformulation and KKT system design

The 2025 GPU implementation extends the NCL idea from a robust algorithmic framework into a GPU-oriented solver architecture. MadNCL fuses the outer ALM loop and the inner IPM loop, rather than treating them as separate phases, and adds an extrapolation step intended to improve efficiency and convergence stability [2510.05885].

With primal-dual variables \(w=(x,r,y,z_l,z_u)\), the inner iterations solve the smooth system
\[
F(w,\rho_k,y_k,\mu_k)=0,
\]
where the components consist of the stationarity equations, the residual equation \(c(x)+r=0\), and the barrier-perturbed complementarity conditions for the bound constraints [2510.05885]. Newton’s method is applied to
\[
\nabla_w F(w,\rho_k,y_k,\mu_k)\,\Delta w=-F(w,\rho_k,y_k,\mu_k),
\]
with a filter line search. Near convergence, MadNCL attempts an extrapolation step
\[
\nabla_w F_k(w_k)d_k+F_k(w_k)=0,\qquad w_k^+=w_k+\alpha_k d_k,
\]
accepted when
\[
\|F_k(w_k^+)\|_\infty\le \theta\|F_k(w_k)\|_\infty+\varepsilon_k,
\qquad
\varepsilon_k=10\alpha_k^{0.2}\mu_k.
\]
The paper attributes the fast asymptotic behavior and superlinear convergence in the final phase to this mechanism [2510.05885].

A central contribution is the comparison of several KKT formulations. Starting from the full Newton system, eliminating the bound multipliers yields the classical symmetric augmented KKT system
\[
K_2= \begin{bmatrix}
\hat H+\delta_k I & 0 & J^\top\\
0 & (\rho_k+\delta_k)I & I\\
J & I & 0
\end{bmatrix},
\]
with \(\hat H=H+\Sigma\) [2510.05885]. For GPU execution, the paper introduces two alternatives:

1. **Stabilized KKT system** \(K_{2r}\),
\[
K_{2r}= \begin{bmatrix}
\hat H+\delta_k I & J^\top\\
J & -\theta_k I
\end{bmatrix},
\qquad
\theta_k=\hat\rho_k^{-1},\ \hat\rho_k=\rho_k+\delta_k,
\]
obtained by eliminating \(\Delta r\).

2. **Condensed KKT system** \(K_{1s}\),
\[
(\hat H+\delta_k I+\hat\rho_k J^\top J)\Delta x
=J^\top(y_k-\hat\rho_k c)-\bigl(\nabla\phi(x)- (X-L)^{-1}\mu_k e+(U-X)^{-1}\mu_k e\bigr),
\]
which is SPD after regularization [2510.05885].

The inertia analysis links these systems:
\[
\operatorname{Inertia}(K_2)=(n+m,m,0)
\iff
\operatorname{Inertia}(K_{2r})=(n,m,0)
\iff
\operatorname{Inertia}(K_{1s})=(n,0,0).
\]
Thus, correct inertia of the stabilized form or positive definiteness of the condensed form guarantees a valid descent direction for the filter line search [2510.05885].

On the GPU, linear algebra is performed with sparse \(\mathrm{LDL}^\top\) factorization using static pivoting in **NVIDIA cuDSS**. For \(K_{2r}\), the implementation uses cuDSS with a small pivot epsilon, e.g. \(10^{-10}\), and then recovers the original-system solution by iterative refinement, typically Richardson iterations. For \(K_{1s}\), cuDSS uses default settings because the matrix is positive definite after regularization [2510.05885]. MadNCL uses **MadNLP** as the inner IPM solver and implements \(K_{2r}\) and \(K_{1s}\) as specialized `AbstractKKTSystem` variants, keeping allocations and computations on the GPU [2510.05885].

The numerical interpretation is deliberately nuanced. The stabilized \(K_{2r}\) formulation is described as more robust, whereas the condensed \(K_{1s}\) formulation can be faster when conditioning is favorable. The condensed system can also become too ill-conditioned for OPF-like problems as \(\rho_k\) grows, so MadNCL does not claim a universally dominant GPU linearization [2510.05885].

## 5. Empirical behavior on general nonlinear programming benchmarks

The 2021 study evaluates the Julia implementation on tax policy models, nonlinear least-squares problems, and large constrained problems from **CUTEst**. The tax policy models are particularly significant because they may not satisfy LICQ at the solution and involve many active constraints. The test family ranges from 1D to 5D, with the largest instance having \(n=864\) variables and \(m=186193\) nonlinear constraints [2101.02164].

On these tax models, the reported pattern is that standard solvers can struggle badly, while NCL with **KNITRO** or **IPOPT** solves the problems robustly. The logs show early subproblems solved at low accuracy, monotonic decrease of \(r\), decrease of \(\|\nabla L\|\), and later subproblems that become much cheaper because of warm starts. Direct IPOPT can time out or fail on large instances, and NCL/KNITRO often outperforms direct KNITRO on these difficult models [2101.02164].

For nonlinear least squares, the paper rewrites
\[
\min_x \ \tfrac12 \|c(x)\|^2 \quad \text{s.t. } \ell\le x\le u
\]
as the feasibility problem
\[
\min_x 0 \quad \text{s.t.}\quad c(x)=0,\ \ell\le x\le u.
\]
With \(y_0=0\), \(\rho_0=1\), and a single major iteration, the first NCL subproblem essentially reproduces the least-squares objective structure [2101.02164]. On **127 CUTEst NLS problems**, KNITRO’s native nonlinear least-squares method solved **101**, whereas NCL solved **119**. Performance profiles show fewer residual evaluations, fewer Jacobian evaluations, lower runtime, and better overall robustness, despite NCL evaluating Hessians [2101.02164].

On **166** constrained CUTEst problems with at least 100 variables and nonlinear constraints, **KNITRO** solved **147** and **IPOPT** solved **126**. The paper emphasizes that NCL is not always the fastest on general problems, but succeeds on some cases where plain IPOPT or KNITRO fail [2101.02164]. This is an important corrective to the view of MadNCL as a general-purpose speed leader: in the general NLP setting, its principal identity is as a robust method for hard constrained problems, especially when degeneracy or LICQ failure is present.

The 2025 GPU paper broadens the empirical domain to large CUTEst problems, optimal power flow, COPS benchmarks, and degenerate SCOPF/MPCC cases. On the large OPF instance `78484epigrids`, MadNCL with \(K_{2r}\) and cuDSS solves the problem in about **54 seconds**, roughly an **18x speedup** relative to a CPU implementation, and about **17x faster** in the linear solver alone [2510.05885]. On the COPS set, both MadNCL-\(K_{2r}\) and MadNCL-\(K_{1s}\) perform well; on OPF problems with very sparse Jacobians, MadNCL-\(K_{2r}\) is usually reliable and faster than CPU baselines, but still generally slower than the fastest GPU IPM variants such as MadNLP with LiftedKKT [2510.05885]. The paper therefore treats MadNCL as extending GPU optimization to difficult degenerate problems rather than supplanting every GPU IPM on every workload.

## 6. Specialization to AC-SCOPF and MPCCs

The AC-SCOPF work presents MadNCL as a solver for **security-constrained AC optimal power flow** in polar form, with complementarity constraints modeling corrective actions such as **AGC / droop control** and **PV/PQ bus switching** [2510.13333]. The problem is written as
\[
\begin{aligned}
\min_{x, u} \; & f(x_0, u_0) \\
\text{s.t.} \quad & g_0(x_0, u_0) = 0 , \quad h_0(x_0, u_0) \leq 0 , \\
& \forall k \in \{1,\cdots,K\} : \\
& g_k(u_0, x_k, u_k) = 0 , \quad h_k(x_k, u_k) \leq 0 , \\
& 0 \leq G(x_k, u_k) \perp H(x_k, u_k) \geq 0 .
\end{aligned}
\tag{SCOPF}
\]
This is described as a very large nonlinear program with thousands of buses, hundreds of contingencies, and millions of variables and constraints in large cases [2510.13333].

After slack variables are introduced, the MPCC is written in vertical form as
\[
\min_{w \in \mathbb{R}^n} \; \phi(w)
\quad \text{s.t.} \quad
c(w)=0,\; w_0 \geq 0,\; 0 \leq w_1 \perp w_2 \geq 0.
\]
It can also be written with the product constraint
\[
(w_1,w_2)\ge 0,\qquad W_1W_2e \le 0.
\]
The paper explicitly states that this problem is degenerate: the feasible set has empty relative interior, MFCQ fails, and multipliers may become unbounded [2510.13333]. This is the setting in which MadNCL’s robustness claims are most direct.

At outer iteration \(n\), NCL solves
\[
\begin{aligned}
\min_{w, r, t} \; & \mathcal{L}_\rho(w, r, t, \lambda^{(n)}, \nu_0^{(n)}) \\
\text{s.t.} \quad & c(w) - r = 0 , \\
& W_1 W_2 e \leq t , \\
& (w_0, w_1, w_2) \geq 0 ,
\end{aligned}
\]
with
\[
\mathcal{L}_\rho(w, r, t, \lambda^{(n)}, \nu_0^{(n)})
=
\phi(w) + (\lambda^{(n)})^\top r + (\nu_0^{(n)})^\top t
+ \frac{\rho^{(n)}}{2}\left(\|r\|^2+\|t\|^2\right).
\]
Here \(r\) relaxes the equality constraints and \(t\) relaxes the complementarity product inequality [2510.13333]. If the original problem is infeasible, the associated feasibility problem minimizes \(\|r\|^2+\|t\|^2\), which the paper identifies as particularly useful for contingency screening [2510.13333].

The linear algebra reflects the same augmented-Lagrangian regularization principle seen in the general GPU paper. The barrier-perturbed Newton system is written as
\[
\begin{bmatrix} A & B^\top \\ B & -C \end{bmatrix}
\begin{bmatrix} \Delta w \\ \Delta y \end{bmatrix}
=
\begin{bmatrix} r_1 \\ r_2 \end{bmatrix},
\]
and the paper stresses that the \((2,2)\) block \(C\) is non-zero because of the AL regularization. This stabilizes the Newton matrix when rows of \(B\) lose rank, enabling a **pivoting-free factorization**, specifically a **signed Cholesky factorization**, compatible with **NVIDIA cuDSS** [2510.13333].

The implementation uses **Julia 1.11**, **MadNLP** for the inner IPM solves, **ExaModels** for derivative evaluation, **HSL MA57** on CPU, **NVIDIA cuDSS** on GPU, a **filter line-search** globalization, **Fiacco-McCormick** barrier updates, `tol = 1e-6`, and \(\mu_{\min}=10^{-7}\) [2510.13333]. Warm starts are also used across outer NCL iterations [2510.13333].

The reported performance is substantial. For one-contingency feasibility screening, average time per contingency is **0.01 s** versus **0.5 s** on `118ieee`, **0.2 s** versus **5.5 s** on `300ieee`, **0.3 s** versus **5.4 s** on `ACTIVSg500`, **5.0 s** versus **75.4 s** on `1354pegase`, and **14.1 s** versus **238.4 s** on `2869pegase`, comparing MadNCL to Knitro [2510.13333]. On `ACTIVSg500` with \(K=256\), yielding nearly **1 million variables**, the paper reports **11,136 s** for Knitro, **1,120 s** for MadNCL-CPU, and **171 s** for MadNCL-GPU; it states that MadNCL solves the case fully on the GPU in less than **3 minutes**, whereas Knitro takes more than **3 hours** to find an equivalent solution [2510.13333].

The limitations are also explicit. As problem size increases, degeneracy in the Newton system becomes more severe; the paper notes ill-conditioning, especially from the PV/PQ reformulation when \(\underline{q}_g\) and \(\overline{q}_g\) are close; MadNCL may require many iterations on larger instances and does not converge consistently on the largest problems [2510.13333]. This reinforces the broader interpretation of MadNCL: it is a robustness-oriented augmented-Lagrangian framework whose strongest results appear on infeasible, complementarity-constrained, and otherwise degenerate nonlinear programs, but whose performance remains problem dependent.

Source: https://www.emergentmind.com/topics/madncl