Abs-Normal Form

• Abs-normal form is a canonical algebraic representation for continuous piecewise-affine and piecewise-smooth functions using auxiliary switch variables.
• It reformulates complex nonsmooth problems into complementarity systems, reducing combinatorial complexity and enabling efficient numerical solutions.
• Practical implementations use forward substitution and specialized solvers, offering robust performance in root-finding and optimization for high-dimensional systems.

The abs-normal form provides a canonical algebraic representation for continuous piecewise-affine (PA) and piecewise-smooth (PS) functions, especially those constructed via compositions of affine functions and scalar absolute value operators. This modeling tool enables systematic analysis, efficient algorithmic treatment of kinked functions, and direct connections to complementarity formulations that underpin modern algorithms for nonsmooth equations and optimization. Abs-normal form encodes all nonsmoothness via a fixed set of auxiliary “switch” variables, reducing the combinatorial overhead typical of generic nonsmooth systems and giving rise to natural complementarity and mixed-integer reformulations appropriate for both theoretical and practical treatment of nonsmooth problems (Zhang et al., 30 Jan 2025, Hegerhorst-Schultchen et al., 2020, Griewank et al., 2017, Hegerhorst-Schultchen et al., 2020).

1. Formal Definition and Construction

Let $f:\mathbb{R}^n\to\mathbb{R}^m$ be a continuous PA mapping. There exists $s\geq 0$ and constants $Z\in\mathbb{R}^{s\times n}$, $L\in\mathbb{R}^{s\times s}$ (strictly lower triangular), $J\in\mathbb{R}^{m\times n}$, $Y\in\mathbb{R}^{m\times s}$, $c\in\mathbb{R}^s$, $b\in\mathbb{R}^m$ such that

$$\begin{aligned} z &= c + Zx + L|z|,\ f(x) &= b + Jx + Y|z|, \end{aligned}$$

where $z\in\mathbb{R}^s$ is the auxiliary switch vector and $s\geq 0$0 denotes the componentwise absolute value. The strictly lower triangular structure of $s\geq 0$1 enforces a forward-substitution dependency, permitting unique evaluation for every $s\geq 0$2 by inductively solving for $s\geq 0$3 as a function of $s\geq 0$4 and $s\geq 0$5 for $s\geq 0$6.

For general PS functions with nonsmoothness arising only via $s\geq 0$7, an automatic differentiation–like forward sweep yields the abs-normal form as a local piecewise-linearization with quadratic approximation error (Griewank et al., 2017). The procedure consists of labeling each $s\geq 0$8 in the computational graph, writing the corresponding $s\geq 0$9 recursion, and expressing the output using the identified $Z\in\mathbb{R}^{s\times n}$0 variables.

2. Connections to Complementarity and Reformulations

The abs-normal form structured representation is crucial for reformulating root-finding and optimization problems into complementarity systems. For $Z\in\mathbb{R}^{s\times n}$1 in abs-normal form,

• Root-finding reduces to a mixed linear complementarity problem (MLCP), with possible further reduction to a linear complementarity problem (LCP) under invertibility conditions on transformed matrix $Z\in\mathbb{R}^{s\times n}$2 (Zhang et al., 30 Jan 2025, Griewank et al., 2017).
• Optimization for the scalar case ($Z\in\mathbb{R}^{s\times n}$3) is represented as a linear program with complementarity constraints (LPCC). Big-M reformulations are used to apply MILP technology (Zhang et al., 30 Jan 2025).

Complementarity-based approaches enable leveraging mature LCP and MPCC solution methods, such as the PATH solver for LCPs and advanced branch-and-bound or MIP solvers for LPCCs.

Problem Type	Abs-Normal Reformulation	Complementarity Formulation
Root-finding	$Z\in\mathbb{R}^{s\times n}$4	MLCP / LCP (in $Z\in\mathbb{R}^{s\times n}$5 variables)
Optimization	$Z\in\mathbb{R}^{s\times n}$6 with $Z\in\mathbb{R}^{s\times n}$7 in ANF	LPCC / Big-M MILP

If $Z\in\mathbb{R}^{s\times n}$8 and $Z\in\mathbb{R}^{s\times n}$9 is invertible, elimination of $L\in\mathbb{R}^{s\times s}$0 yields an LCP entirely in $L\in\mathbb{R}^{s\times s}$1: $L\in\mathbb{R}^{s\times s}$2 where $L\in\mathbb{R}^{s\times s}$3 and $L\in\mathbb{R}^{s\times s}$4 are explicitly constructed from ANF data (Zhang et al., 30 Jan 2025, Griewank et al., 2017). This reformulation is central to tractable nonsmooth solvability.

3. Theoretical Properties and Constraint Qualifications

The abs-normal representation supports a powerful theoretical framework paralleling that for MPCCs (Mathematical Programs with Complementarity Constraints) (Hegerhorst-Schultchen et al., 2020, Hegerhorst-Schultchen et al., 2020). Key concepts include:

• Kink Qualifications (KQ):
  • Linear Independence Kink Qualification (LIKQ): Ensures full-rank conditions for first-order theory, equivalent to MPCC–LICQ.
  • Interior Direction Kink Qualification (IDKQ): Generalizes MFCQ for nonsmooth settings, equivalent to MPCC–MFCQ.
• Tangent Cones and Branches: At feasible points, tangent and linearized cones (incorporating the effect of “signature vectors” indicating the local sign structure of switches) play the same role as in classical NLP but adapted to nonsmoothness.
• Stationarity Notions:
  • Mordukhovich (M-) Stationarity: Strong first-order necessary conditions, preserved through abs-normal ↔ MPCC correspondence.
  • Bouligand (B-) Stationarity: Holds under very weak CQs and ensures all local minimizers are stationary for every branch problem.

Moreover, second-order necessary and sufficient conditions (involving critical cones and reduced Hessians) are congruent between ANF and MPCC representations, relying on signature structure and full-rank conditions (Hegerhorst-Schultchen et al., 2020).

A popular slack reformulation converts inequalities into equalities with additional abs variables, facilitating fully equality-constrained problems but preserving only the strongest (LICQ/LIKQ) regularity. IDKQ/MFCQ may not be preserved in this translation, and nonuniqueness of slacks can complicate certain algorithms (Hegerhorst-Schultchen et al., 2020).

4. Numerical Methods and Algorithms

Solvers for abs-normal systems include direct and iterative approaches. For root-finding, once the LCP (or MLCP) is assembled from the given abs-normal data, standard complementarity solvers can be applied. For optimization, LPCC and big-M MILP formulations admit tractable solution by advanced mixed-integer solvers (Zhang et al., 30 Jan 2025). Semismooth Newton methods, Bokhoven's modulus iteration, signed Gaussian elimination, and block-Seidel–style iterations are also developed, with convergence guaranteed under spectral properties of the Schur complement matrix $L\in\mathbb{R}^{s\times s}$5 (Griewank et al., 2017).

Method	Applicability	Convergence Condition
Semismooth Newton (OPL/CPL)	General PL/ANF	nonsingularity of $L\in\mathbb{R}^{s\times s}$6 (all branches), or $L\in\mathbb{R}^{s\times s}$7 for CPL
Bokhoven's modulus	CPL	$L\in\mathbb{R}^{s\times s}$8
Signed Gaussian elimination	CPL	$L\in\mathbb{R}^{s\times s}$9
Block-Seidel	ANF	$J\in\mathbb{R}^{m\times n}$0

For large-scale systems, LCP solution times scale sublinearly with dimension: in numerical examples, PATHSolver.jl solves random LCPs with $J\in\mathbb{R}^{m\times n}$1 in under 1s (Zhang et al., 30 Jan 2025). The big-M MILP approach for large optimization instances is empirically 2–3× faster than direct LPCC for $J\in\mathbb{R}^{m\times n}$2 (Zhang et al., 30 Jan 2025).

5. Structural and Comparative Properties

The abs-normal form encodes all kinked behaviors via explicit $J\in\mathbb{R}^{m\times n}$3 variables, allowing every continuous PA function to have such a representation. Compared to general nonsmooth frameworks, the abs-normal approach achieves the following:

• Compactness and Explicitness: All nonsmoothness is localized to finitely many $J\in\mathbb{R}^{m\times n}$4 variables, enabling exact (not combinatorial) treatment of kinks, branches, and stationary points (Griewank et al., 2017).
• Theoretically Complete Link to MPCCs: All central notions in MPCC theory (CQs, stationarity, optimality) carry over verbatim to abs-normal NLPs (Hegerhorst-Schultchen et al., 2020, Hegerhorst-Schultchen et al., 2020).
• Reformulation Equivalence: ANF-based constrained NLPs and their MPCC analogs are homeomorphic in their feasible sets under the standard $J\in\mathbb{R}^{m\times n}$5 variable change.

Abs-normal form’s strict lower-triangular coupling (via $J\in\mathbb{R}^{m\times n}$6) ensures unique solvability in forward evaluation and in constructing the transformed matrices for complementarity reformulations.

6. Applications and Computational Practice

Abs-normal form is applied in modeling discrete-continuous processes (such as contact mechanics), neural network activations (notably ReLU), and more generally, in any setting requiring robust tractable nonsmooth analysis (Zhang et al., 30 Jan 2025). Its utility is particularly pronounced in the following contexts:

• Automatic differentiation for PS functions: Enabling AD-based piecewise-linear local models with quadratic error (Griewank et al., 2017).
• Robust root-finding for PA systems: LCP/MLCP reduction yields reliable existence and uniqueness theorems depending on matrix properties ($J\in\mathbb{R}^{m\times n}$7 is a Q-matrix for existence, P-matrix for uniqueness) (Zhang et al., 30 Jan 2025).
• Global optimization of nonsmooth objectives: Systems can be tested for coercivity via auxiliary MLCP. Big-M MILP methods provide practical solution capability for high-dimensional problems (Zhang et al., 30 Jan 2025).
• Constraint logic and combinatorial modeling: Explicit signature and branch structure facilitate finite enumeration or path-following, especially in settings where nonsmoothness is combinatorially induced (e.g., in switching networks).

7. Illustrative Examples and Software

Practical illustration is provided by numerical examples where abs-normal form systems of dimension up to 500 are solved efficiently, and optimization problems involving deeply nested absolute values are tractably reformulated and solved (Zhang et al., 30 Jan 2025). Tools developed in Julia (notably the repository https://github.com/kamilkhanlab/abs-normal) implement all central routines, including construction of abs-normal data, assembly of complementarity systems, and solver invocation for root-finding and optimization.

Typical example summaries:

• 2D MLCP and LCP: unique root found in milliseconds.
• 3D scalar optimization: global minimum found in $J\in\mathbb{R}^{m\times n}$8 to $J\in\mathbb{R}^{m\times n}$9 seconds.
• Scalability: LCP path solver achieves sub-second times up to $Y\in\mathbb{R}^{m\times s}$0.
• For $Y\in\mathbb{R}^{m\times s}$1, LCP+PATH approach outperforms older methods by 2–5×, and big-M MILP is 2–3× faster than direct LPCC (Zhang et al., 30 Jan 2025).

These examples affirm the practical performance benefits and numerical tractability of the abs-normal representation for both root-finding and nonsmooth global optimization tasks.