Disciplined Nonlinear Programming

Abstract: We introduce disciplined nonlinear programming (DNLP), a syntax for specifying nonlinear programming problems. DNLP is inspired by disciplined convex programming (DCP) and allows smooth functions to be freely mixed with nonsmooth convex and concave functions, with rules governing how the nonsmooth functions can be used. Problems expressed in DNLP form can be automatically canonicalized to a standard nonlinear programming (NLP) form and passed to a suitable NLP solver. As in DCP, the canonicalization relaxes nonsmooth convex and concave functions in a lossless way, allowing them to be handled by NLP solvers that require smooth functions. In addition to extending NLP to include useful nondifferentiable convex and concave functions, transforming the original problem to an equivalent NLP form offers several advantages, including simpler problem initialization. We describe the language and our open-source implementation of DNLP as an extension of CVXPY, a parser for DCP.

• The paper introduces a novel canonicalization framework, DNLP, that converts mixed smooth and nonsmooth models into standard smooth NLPs to boost solver robustness.
• It employs a curated atom library and expression grammar to categorize functions as smooth, NS-convex, or NS-concave, ensuring accurate model specifications.
• The methodology is implemented via an extension of CVXPY and validated through diverse applications in geometry, signal processing, finance, and energy.

Disciplined Nonlinear Programming: Language, Canonicalization, and Applications

Motivation and Background

Nonlinear programming (NLP) encompasses a broad class of optimization problems with nonconvex, smooth (differentiable), and nonsmooth components, central in domains such as chemical engineering, topology optimization, control, and finance. While powerful, the lack of general global optimality and the sensitivity of solver behavior to problem formulation, initialization, and nondifferentiabilities pose persistent challenges.

Traditional modeling languages for NLP (e.g., AMPL, GAMS, Pyomo, JuMP, CasADi) provide interfaces to solvers but typically treat user-specified formulations as black boxes. This impedes exploitation of structural properties, often resulting in fragile solver performance, especially for models combining smooth and nonsmooth components. By contrast, in disciplined convex programming (DCP), rigorous syntactic rules and canonicalizations both enable reliable solver behavior and guarantee convexity.

This paper introduces Disciplined Nonlinear Programming (DNLP)—a syntactic framework and implementation for specifying a broad class of NLPs with well-defined handling of both smooth and nonsmooth (convex/concave) terms, unifying and extending ideas from DCP to the nonlinear regime (2606.02896).

The Disciplined Nonlinear Programming (DNLP) Framework

Atom Library and Expression Grammar

DNLP is constructed on a curated atom library, classifying functions as either:

• Smooth: twice continuously differentiable on their domain (e.g., $\sin(x)$, $\exp(x)$, $\log(x)$ for $x > 0$, $\sqrt{x}$ for $x \geq 0$, affine, quadratic forms);
• Nonsmooth Convex (NS-convex): convex but nondifferentiable at some points (e.g., $|x|$, $\max_i x_i$, $\ell_1$ and $\ell_2$ norms, Huber, ordered means/largest);
• Nonsmooth Concave (NS-concave): concave but nondifferentiable (e.g., $\exp(x)$0, sum of smallest components).

Atoms are further annotated with sign and monotonicity properties, including sign-dependent monotonicity and affine/nonmonotone exceptions.

Expression Types: Smooth, L-Convex, L-Concave

DNLP expressions are recursively defined. The categorization is:

• Smooth Expression: Atom and all arguments are smooth expressions.
• L-Convex (Linearizable Convex) Expression: Atom is smooth or NS-convex; arguments are either smooth, L-convex with monotonicity matching nondecreasing slots in the atom, or L-concave with monotonicity matching nonincreasing slots.
• L-Concave (Linearizable Concave) Expression: Analogous, with appropriate monotonicity conditions.

A key insight is that smoothness is inherited upward (the composition of smooth functions is smooth), and the L-convex/L-concave composition rules mirror DCP’s convex promotion rules but with smooth atoms substituting for affine ones.

Problem Class and DNLP Compliance

A DNLP-compliant problem has objectives and constraints restricted as follows:

• Objective: Minimize L-convex or maximize L-concave expressions.
• Constraint: Equalities only between smooth expressions. Inequalities of the form L-convex $\exp(x)$1 L-concave, or L-concave $\exp(x)$2 L-convex.

Any conforming problem admits a lossless transformation to a standard smooth NLP.

Canonicalization: Syntax-Driven Transformation

The primary contribution is a canonicalization pipeline: DNLP problems are transformed, via parsing and tree rewriting, to smooth NLPs that can be solved by standard solvers (e.g., IPOPT, Knitro).

Smooth Atoms

For smooth atoms with a restricted domain, argument variables are introduced and explicit constraints enforce domain membership. This facilitates feasible initialization and improves solver robustness.

Nonsmooth Atoms

For each NS-convex or NS-concave atom, epigraph/hypograph variable introductions and canonical constraints are added, which are then smoothly reformulated (e.g., epigraph representations for $\exp(x)$3, $\exp(x)$4, $\exp(x)$5, etc., via standard algebraic manipulations). LICQ is preserved, and relaxation is lossless: solution sets and optimal values are not altered by the transformation.

Initialization is systematically handled. In contrast to typical modeling language behavior (where origin-based or arbitrary initialization yields failures for restricted-domain atoms), DNLP's canonicalization enables straightforward feasible initialization.

Implementation

DNLP is implemented as an extension of CVXPY, retaining CVXPY syntax but supporting a much broader collection of nonlinear, nonconvex, smooth, and nonsmooth atoms, as well as advanced features for initialization and randomized restart (\texttt{best_of}). The backend supports both open-source (IPOPT, Uno) and commercial solvers (Copt, Knitro), and explicitly propagates and caches derivative sparsity structure.

Numerical Examples and Applications

The expressivity and flexibility of DNLP are demonstrated via an extensive suite of representative applications in geometry, signal processing, finance, energy, and statistics.

Geometry and Planning

The framework elegantly encodes path planning problems with obstacle avoidance (using quadratic and norm-based constraints), global circle packing optimized under nonconvex constraints, and nonlinear optimal control (e.g., car trajectory with kinematic bounds).

Figure 1: Shortest path connecting two points while avoiding circular obstacles.

Figure 2: Circle packing.

Figure 3: The best circle packing found over 500 random initializations (left), and a histogram of the coverages obtained across all initializations (right).

Figure 4: Car trajectory. Top. Position and orientation of the car. Middle. Speed and steering angle. Bottom. Acceleration and steering rate.

Signal Processing

DNLP enables sparse signal recovery via nonconvex penalties (e.g., sum of square roots of absolute values), achieving higher recovery probabilities than convex relaxations, as well as nonconvex phase retrieval and nonlinear source localization.

Figure 5: Location estimation from range measurements. Left. Range measurements without noise. Right. Range measurements with noise.

Figure 6: Probability of successful signal recovery. Left. Approach based on nonconvex optimization. Right. Approach based on convex optimization.

Figure 7: Phase retrieval.

Finance

Risk-budgeted portfolio construction problems, SVI model calibration for option surfaces, and flexible risk-neutral density estimation are specified and solved using DNLP’s high-level syntax and canonicalization, supporting out-of-sample cross-validation and complex nonconvex forms.

Figure 8: Sector risk contributions of the risk-budgeted portfolio.

Figure 9: SVI model calibration. The red crosses show the observed implied volatilities, and the blue curve shows the fitted SVI model.

Figure 10: The percentage of held-out options for which the model price falls within the bid-ask spread, across all cross-sections.

Power and Statistics

Applications include AC optimal power flow for electrical networks and high-dimensional parameter estimation (e.g., nonnegative matrix factorization, robust regression with Huber loss, and factor analysis via constrained log-determinant minimization).

Figure 11: Optimal power flow.

Figure 12: Nonnegative matrix factorization for decomposing images into parts.

Figure 13: Fitting an exponential decay model with outliers.

Numerical Insights and Strong Claims

• DNLP-compliant formulations provide substantial increases in solver robustness for nonsmooth NLPs, compared to black-box approaches.
• In multiple instances (e.g., sparse recovery, circle packing, SVI calibration, portfolio design), the framework corrects for formulation-induced solver failures and further admits structured initializations, which are essential for solver success in high-dimensional, nonconvex landscapes.
• The canonical lossless transformation guarantees preservation of key properties such as the linear independence constraint qualification (LICQ).

Theoretical and Practical Implications

The introduction of DNLP fills a crucial gap between DCP and generic NLP modeling, enabling:

• Reliable, structured specification of broad classes of optimization models mixing smoothness, convexity, and concavity constraints.
• Automatic, correct canonicalization—yielding standardized, reliable NLP solver interfaces and removing sources of avoidable failures due to poor modeling or initialization.
• Easy extensibility and integration with emerging NLP solver technology, and potential applicability in (nonconvex) deep learning pipelines where part of the loss, constraints, or regularization are nonsmooth but convex (e.g., structured sparsity, robust statistics).

The framework can guide future trends in automatic program synthesis, meta-modeling, and even symbolic AI pipelines, where the sound handling and transformation of composite nonlinear expressions are critical for robustness and interpretability.

Conclusion

Disciplined Nonlinear Programming bridges the DCP paradigm and practical NLP modeling, offering a syntactically constrained, formally justified, and highly expressive system for large-scale, nonconvex, and nonsmooth optimization. This approach eliminates many common pitfalls of black-box modeling workflows and positions itself as a methodological standard for future nonlinear programming systems, both in academic optimization and in applied computational science.