Graphical Derivative Newton Algorithm

• The Graphical Derivative Newton Algorithm is a method that replaces classical derivatives with graphical derivatives to capture first-order tangent behavior in nonsmooth or set-valued contexts.
• It employs an inclusion-based update scheme that accommodates piecewise linearizations and secant models, enabling robust convergence even with non-differentiable functions.
• Convergence is achieved under metric regularity with guarantees of superlinear to quadratic rates, making it effective for large-scale, structured optimization problems.

A graphical derivative-based Newton algorithm is a class of iterative methods for solving nonlinear equations and optimization problems, which generalizes the classical Newton method by replacing the standard derivative or Hessian with the graphical derivative—a variational-analytic object capturing first-order tangent behavior even for nonsmooth, set-valued, or piecewise-differentiable maps. This approach enables robust Newton-type convergence in settings where classical differentiability fails, including nonsmooth and nonconvex problems, and is supported by several frameworks in optimization and numerical analysis.

1. The Graphical Derivative and Its Role

The graphical derivative, for a set-valued mapping $F: \mathbb{R}^n \rightrightarrows \mathbb{R}^m$, is defined at a point $(\bar{x}, \bar{y}) \in \mathrm{gph} F$ by

$$DF(\bar{x}, \bar{y})(u) := \{\, v \in \mathbb{R}^m \mid (u,v) \in T_{\mathrm{gph} F}(\bar{x}, \bar{y}) \,\}$$

where $T_{\mathrm{gph} F}$ is the Bouligand tangent cone to the graph of $F$ (Hoheisel et al., 2010, Mordukhovich et al., 2020, Garrido et al., 2024). For single-valued, smooth $F$, the graphical derivative reduces to the classical Jacobian action: $DF(\bar{x})(u) = \nabla F(\bar{x}) u$.

In nonsmooth or set-valued contexts, the graphical derivative describes the limiting first-order behavior of $F$ around $\bar{x}$, thereby providing a framework for defining and analyzing Newton-type iterations even when classical derivatives do not exist.

2. Algorithmic Schemes Based on Graphical Derivatives

A generic graphical derivative-based Newton iteration for solving $H(x) = 0$, with $H$ continuous and possibly nonsmooth, is given by

$$- H(x^k) \in DH(x^k)(d^k), \quad x^{k+1} = x^k + d^k$$

where $DH(x^k)$ denotes the graphical derivative of $H$ at $x^k$ (Hoheisel et al., 2010, Mordukhovich et al., 2020). This inclusion replaces the linear Newton step from the classical algorithm by a set-valued mapping constructed from the tangent cone to the graph of $H$, enabling the extension of Newton-type methods to non-Lipschitz, non-differentiable, or piecewise-differentiable functions.

For piecewise differentiable or algorithmic functions, an explicit piecewise linearization may be constructed using two anchor points $x_{k-1}$, $x_k$, yielding the secant model $L_k(x)$. The update is then

$$x_{k+1} = \operatorname{argmin}_{L_k(x) = 0} \| x - m_k \| \quad \text{where } m_k = \tfrac{1}{2}(x_{k-1}+x_k)$$

as in two-point secant-based algorithms (Griewank et al., 2017).

Graphical derivative-based Newton methods also extend naturally to continuous-time dynamical systems for nonsmooth and nonconvex optimization, where the Newton flow is governed by the inclusion

$$0 \in \partial \varphi_1(x(t)) + \partial \varphi_2(x(t)) + DF(x(t))(\dot{x}(t))$$

with $\varphi_1$ primal lower-regular and $\varphi_2$ locally Lipschitz (Garrido et al., 2024).

3. Convergence Theory

The convergence properties of graphical derivative-based Newton algorithms are established under metric regularity and suitable local tangent approximation properties. Key results include:

• Metric regularity of $H$ ensures well-posedness of the Newton step, i.e., local solvability of the inclusion with $d^k$ (Hoheisel et al., 2010).
• Local superlinear convergence is shown when the family $\{ DH(x) \}$ forms a Newton approximation scheme (i.e., first-order tangent error is little-o of the displacement), even without semismoothness (Hoheisel et al., 2010).
• For unconstrained minimization of $f$ with Lipschitz gradient and tilt stability, the Newton direction is computed via the graphical derivative of $\nabla f$, and under twice epi-differentiability, local Q-superlinear convergence occurs (Mordukhovich et al., 2020).
• In the piecewise linear setting, tangent-model updates yield quadratic convergence, while secant-model updates achieve order $\varphi = (1+\sqrt{5})/2$ (the golden ratio) (Griewank et al., 2017, Griewank et al., 2017).
• For dynamical systems, energetic solutions of the Newton-like flow converge globally under strong metric subregularity or Kurdyka–Łojasiewicz properties, and may exhibit exponential, polynomial, or even finite-time convergence depending on the regularity of the target functional (Garrido et al., 2024).

A comparison of convergence assumptions and guarantees is presented in the following table (Hoheisel et al., 2010, Mordukhovich et al., 2020):

Method	Key Assumptions	Local Rate
Graphical-derivative	Metric regularity, tangent-approx. (H1, H2)	Superlinear
Semismooth Newton	Local Lipschitz, semismoothness	Superlinear (Jacobian)
Classical Newton	Differentiability, nonsingular derivative	Quadratic
Piecewise secant-Newton	Coherent orientation, local bilinear error	Quadratic/φ-order

4. Practical Implementation and Algorithmic Variants

Implementation issues center on the computation of the graphical derivative or its associated piecewise-linearization.

• In practical scenarios with functions implemented as straight-line programs involving smooth and nonsmooth primitives (e.g., abs, max), the secant model can be constructed algorithmically via forward-mode differentiation, propagating pairs of values and symbolic increments (Griewank et al., 2017).
• Numerical stability is enhanced by transferring to a midpoint/radius representation to eliminate catastrophic cancellation in divided differences (Griewank et al., 2017).
• Solving the Newton inclusion or the piecewise-linear system may require enumerating active regions or using convex programming (for polyhedral graphical derivatives) (Hoheisel et al., 2010, Griewank et al., 2017).
• Graphical Newton frameworks for structured problems (e.g., computational graphs or DAGs encoding the objective and constraints) exploit sparsity to reduce the complexity of Newton step computation from O(N³) to O(|graph|+tw³), where tw is the tree-width of the moralized graph (Srinivasan et al., 2015). Reverse- and forward-mode algorithmic differentiation combine with sparse KKT solvers or message-passing on tree-decompositions to efficiently solve large-scale systems.

Pseudocode for a general graphical-derivative Newton-type iteration is as follows (Hoheisel et al., 2010, Griewank et al., 2017):

k = 0 while norm(H(xk)) > tol: # Compute/set-valued linearization: d^k such that -H(xk) ∈ DH(xk)(d^k) d_k = graphical_newton_direction(xk, H) xk = xk + d_k k += 1

Here, graphical_newton_direction may require either explicit variational analysis, active-set enumeration, or, in high-dimensional sparse problems, a message-passing-based sparse solver (Srinivasan et al., 2015, Griewank et al., 2017).

5. Extensions and Related Algorithms

The graphical derivative-based Newton algorithm subsumes or generalizes several well-known methods:

• Semismooth Newton: If the function is locally Lipschitz and semismooth (in the Clarke sense), then the Clarke generalized Jacobian may be used in the Newton update, which is a specific selection from the graphical derivative (Hoheisel et al., 2010).
• B-differentiable Newton: If the directional derivative exists and is nonsingular, Pang’s Newton approach is recovered as a special case (Hoheisel et al., 2010).
• Cubic and higher-order Newton: Using a higher degree interpolant in place of the tangent leads to generalized-derivative or “graphical” Newton steps of order $p+1$ (Barbosa et al., 5 Jan 2026). In the scalar case, the cubic Newton step achieves quartic convergence and may reduce iteration counts by more than half compared to the classical Newton method.

For nonsmooth optimization, the approach carries over to Moreau envelope regularizations of prox-regular functions and tilt-stable minimizers, with the Newton direction found by solving a quadratic program in the critical cone of the feasible set at each iteration (Mordukhovich et al., 2020).

6. Representative Applications and Illustrative Examples

Graphical derivative-based Newton methods provide solution frameworks in:

• Nonsmooth roots and inclusions: Solving $F(x)=0$ for continuous or piecewise-differentiable $F$, with convergence in settings not accessible to classical Newton or semismooth Newton (Hoheisel et al., 2010).
• Nonsmooth, nonconvex minimization: Fast identification of tilt-stable minimizers and saddle points in large-scale nonsmooth optimization benchmarks (Mordukhovich et al., 2020, Garrido et al., 2024).
• Polynomiography and fractal visualizations: Iterative graphical-derivative-based Newton schemes visualize complex polynomial root basins, with rich behavior not present in the classical Newton method (Karaca et al., 2017).
• Computational graph problems: Graphical Newton leverages algorithmic differentiation and message passing for Newton step computation in large DAGs with complex dependencies, particularly in sparse optimal control and machine learning models (Srinivasan et al., 2015).

Table: Algorithm Properties and Typical Use-Cases

Method/Class	Typical Applications	Core Property/Advantage
Piecewise secant-Newton	Piecewise differentiable, programmatic F	Order φ convergence, no 2nd diff.
Graphical-derivative Newton	General nonsmooth/nonconvex systems	Superlinear, mild assumptions
Moreau envelope Newton	Nonsmooth, prox-regular optimization	Tilt stability, superlinear rate
Sparse Graphical Newton	Large-scale, structured objectives	O(
Higher-order graphical Newton	Scalar equations, analytic derivatives	Quartic/local convergence

7. Theoretical and Practical Significance

Graphical derivative-based Newton algorithms extend Newtonian iterative methods to nonsmooth and set-valued analysis, underpinned by metric regularity and tangent approximation principles rather than classical differentiability. This theoretical generality enables convergence in a broad range of settings:

• Problems with only primal lower-regularity (PLR) or prox-regularity, rather than full convexity or smoothness (Garrido et al., 2024, Mordukhovich et al., 2020).
• Nonsmooth and piecewise linear functions, e.g., those arising in complementarity systems, variational inequalities, or complex optimization landscapes (Griewank et al., 2017, Hoheisel et al., 2010).
• Computationally structured scenarios where problem sparsity and dependency structure allow complexity reductions from cubic to linear plus tree-width-cubic terms (Srinivasan et al., 2015).
• Algorithmic optimization requiring guaranteed convergence or sharp error estimates without reliance on higher-order classical derivatives (Hoheisel et al., 2010, Karaca et al., 2017).

A plausible implication is that further advances in algorithmic differentiation and graphical derivative computation—especially in high-dimensional, structured regimes—will continue to improve the tractability and robustness of Newton-type solvers for complex nonsmooth problems.