Second-Order Generalized Differentiation

• Second-order generalized differentiation is the framework that extends classical second derivatives to nonsmooth and constrained contexts by using varied derivative constructs such as coderivatives and constrained projections.
• It plays a vital role in optimality theory, stability analysis, and Newton-type algorithms by providing precise curvature conditions and effective second-order test formulations.
• Computational techniques like automatic differentiation and complex-step methods are employed to efficiently obtain and exploit second-order information in simulation, optimization, and transport calculus.

Second-order generalized differentiation is the body of theory that extends classical second derivatives beyond the $C^2$ finite-dimensional setting. In contemporary analysis it appears in several non-equivalent but structurally related forms: coderivative-based generalized Hessians for nonsmooth functions, constrained second derivatives obtained by repeated projection onto a constraint manifold, mixed second-order constructions in Hilbert spaces, second-order differentiation formulas along Wasserstein geodesics on $RCD^*(K,N)$ spaces, recursive local fractional derivatives that extract successive local scaling exponents, and algorithmic schemes that differentiate entire update maps to obtain Hessians and consistent tangents (Mordukhovich et al., 2015, Khanh et al., 2023, Gal, 2012, Wei et al., 2016, Gigli et al., 2018, Shen et al., 2024). In the smooth case these constructions reduce to the classical Hessian or its geodesic analogue; outside that regime they provide the curvature information needed for optimality theory, stability analysis, transport calculus, and higher-order sensitivity computation (Khanh et al., 2021, Gigli et al., 2018).

1. Principal frameworks

A single universal definition does not govern the subject. Different geometries require different second-order objects, and the choice is typically dictated by the structure of the first-order theory.

Setting	Representative second-order object	Typical role
Variational analysis	coderivative of the subgradient mapping	generalized Hessian, optimality, stability
Constrained calculus	constrained derivative applied twice	classification of constrained stationary points
Hilbert-space nonsmooth analysis	lower Dini derivative, mixed graphical derivative, mixed proximal subdifferential	quadratic growth and strict local minimality
Metric-measure geometry	Hessian tested along $W_2$-geodesics	second-order calculus on $RCD$ spaces
Local fractional analysis	recursive local fractional derivative	extraction of successive fractional orders
Computational differentiation	nested AD, CSFD-AD, Vibrato+AD	Hessians, consistent tangents, high-order sensitivities

Representative formulations of these six viewpoints are developed in the explicit CPWL and prox-regular settings (Mordukhovich et al., 2015, Khanh et al., 2023), in constrained and Hilbert-space optimization (Gal, 2012, Wei et al., 2016), in transport geometry on $RCD^*(K,N)$ spaces (Gigli et al., 2018), in recursive local fractional calculus (Kolwankar, 2013), and in simulation, constitutive modeling, and Monte Carlo sensitivity analysis (Blühdorn et al., 2020, Shen et al., 2024, Pagès et al., 2016).

2. Variational-analytic generalized Hessians

In finite-dimensional variational analysis, the central second-order object is the Mordukhovich second-order subdifferential

$$\partial^2 p(\bar x,\bar v)(u):=D^*(\partial p)(\bar x,\bar v)(u),$$

defined as the coderivative of the limiting subdifferential mapping. The 2023 optimality theory for prox-regular functions also distinguishes the combined second-order subdifferential

$$\breve\partial^2 p(\bar x,\bar v)(u):=\hat D^*(\partial p)(\bar x,\bar v)(u)$$

and the regular second-order subdifferential

$$\widehat{\partial}^2 p(\bar x,\bar v)(u):=\hat D^*(\hat\partial p)(\bar x,\bar v)(u),$$

with

$$\widehat{\partial}^2 p(\bar x,\bar v)(u)\subset \breve\partial^2 p(\bar x,\bar v)(u)\subset \partial^2 p(\bar x,\bar v)(u).$$

If $p$ is $RCD^*(K,N)$0, all of them reduce to the classical Hessian $RCD^*(K,N)$1 (Khanh et al., 2023).

For convex piecewise linear functions, second-order generalized differentiation becomes completely explicit. A CPWL function has a polyhedral epigraph, active index sets, and a subdifferential represented by a convex hull part plus a cone part. The paper "Generalized differentiation of piecewise linear functions in second-order variational analysis" proves that the second-order subdifferential of an arbitrary CPWL function can be calculated entirely in terms of this primitive data, and it derives an exact second-order sum rule for the decomposition into a maximum function and an indicator of a polyhedral domain (Mordukhovich et al., 2015). The same polyhedral structure underlies the CPWL analysis of subdifferential variational systems, where the critical cone is

$RCD^*(K,N)$2

and both $RCD^*(K,N)$3 and $RCD^*(K,N)$4 admit exact data-based formulas (Mordukhovich et al., 2016).

A common misconception is that any generalized Hessian-like object is interchangeable in optimality theory. The prox-regular theory shows otherwise: for a local minimizer $RCD^*(K,N)$5, the pointbased necessary condition is

$RCD^*(K,N)$6

and the analogous statement with the limiting/basic second-order subdifferential is false in general. In that sense, the combined construction is the correct necessity object, while the limiting construction is more naturally tied to sufficiency, tilt stability, and coderivative calculus (Khanh et al., 2023).

3. Optimality, stability, and Newton-type algorithms

Second-order generalized differentiation is central to modern nonsmooth optimality theory. For prox-regular objectives, local minimality yields first-order stationarity $RCD^*(K,N)$7 and the second-order inequality above; for a strong local minimizer with modulus $RCD^*(K,N)$8, the necessary condition strengthens to

$RCD^*(K,N)$9

In constrained problems of the form $W_2$0, these conditions reduce to Lagrangian-Hessian inequalities on critical cones under the stated qualification assumptions, and in $W_2$1 nonlinear programming they collapse to the classical second-order tests (Khanh et al., 2023).

The theory of critical and noncritical multipliers extends this variational picture to generalized KKT systems

$W_2$2

A multiplier $W_2$3 is critical if there exists a nonzero $W_2$4 solving a generalized second-order KKT system driven by $W_2$5. In the CPWL case this can be rewritten through the critical cone and a primal-dual linear system involving $W_2$6. The paper proves that noncriticality is equivalent to a semi-isolated calmness estimate and to an error bound for a canonically perturbed system; full stability rules out critical multipliers; and in CPWL composite optimization, robust isolated calmness is equivalent to noncriticality together with multiplier uniqueness and a second-order sufficient condition (Mordukhovich et al., 2016).

These generalized Hessians also define Newton directions. For $W_2$7 optimization, the generalized damped Newton method replaces

$W_2$8

by the inclusion

$W_2$9

followed by Armijo backtracking. Positive definiteness of $RCD$0 implies that any nonzero solution $RCD$1 is a descent direction. Under positivity on a bounded level set, the iterates converge to a tilt-stable local minimizer; under semismoothness and the stated step-acceptance conditions, convergence becomes $RCD$2-superlinear and full steps are eventually accepted (Khanh et al., 2021).

4. Constraints and infinite-dimensional extensions

In constrained differentiation, the first-order derivative is obtained by projecting the full derivative onto the tangent space of the constraint. For a functional $RCD$3 subject to $RCD$4, one special choice yields the orthogonal projection

$RCD$5

with $RCD$6. The second-order derivative is then defined by applying the constrained derivative operator twice. The resulting constrained Hessian is the proper general-purpose object for stationary-point analysis: its quadratic form on admissible variations yields necessary and sufficient conditions for constrained minima and maxima, and its eigenvalues classify the stationary point once the zero mode in the constraint direction is excluded (Gal, 2012).

This successive-differentiation viewpoint addresses an ambiguity in direct higher-order constrained projections. The essential conclusion is that one does not need to invent a separate canonical second-order constrained derivative from scratch; repeated application of the first-order constrained derivative already provides the correct curvature object for stationary-point classification (Gal, 2012).

In Hilbert spaces, finite-dimensional contingent geometry is no longer sufficient. The paper "Second-order Optimality Conditions by Generalized Derivatives and Applications in Hilbert Spaces" introduces three second-order constructions: the second-order lower Dini-directional derivative $RCD$7, the mixed graphical derivative $RCD$8, and the mixed proximal subdifferential $RCD$9. The mixed contingent cone uses strong convergence in the primal component and weak convergence in the dual component, and in finite dimensions the mixed and ordinary constructions coincide. Under paraconcavity, continuity, twice epi-differentiability, and the stated $RCD^*(K,N)$0 assumptions, the three second-order optimality conditions become equivalent. The same framework characterizes strict local minimizers of order two through a uniform lower bound of $RCD^*(K,N)$1 on the unit sphere (Wei et al., 2016).

5. Geodesic, metric-measure, and fractional formulations

Second-order generalized differentiation also arises in nonsmooth geometry. On finite-dimensional $RCD^*(K,N)$2 and $RCD^*(K,N)$3 spaces, the second-order differentiation formula along Wasserstein geodesics states that if $RCD^*(K,N)$4 is the unique $RCD^*(K,N)$5-geodesic with bounded densities and compact supports and $RCD^*(K,N)$6, then

$RCD^*(K,N)$7

Equivalent formulations hold along optimal geodesic test plans in $RCD^*(K,N)$8 (Gigli et al., 2018, Gigli et al., 2018).

The proof strategy is itself a second-order regularization theory. Direct differentiation of the geodesic flow is unavailable in the nonsmooth setting, and a naive viscous approximation is not sufficient because the transport potential must remain $RCD^*(K,N)$9-concave and shock-free. The successful approximation is entropic interpolation from the Schrödinger problem, with Schrödinger potentials satisfying viscous Hamilton–Jacobi equations, uniform $$\partial^2 p(\bar x,\bar v)(u):=D^*(\partial p)(\bar x,\bar v)(u),$$0 density bounds, local equi-Lipschitz control, and weighted $$\partial^2 p(\bar x,\bar v)(u):=D^*(\partial p)(\bar x,\bar v)(u),$$1 bounds on Hessians and Laplacians. The decisive step is the vanishing of the acceleration term in the limit. The compact $$\partial^2 p(\bar x,\bar v)(u):=D^*(\partial p)(\bar x,\bar v)(u),$$2 result was established first and then extended to the full finite-dimensional noncompact setting; the finite-dimensional assumption is essential because the argument uses Li–Yau type estimates (Gigli et al., 2017, Gigli et al., 2018).

A different higher-order idea appears in local fractional calculus. The original local fractional derivative is limited by a critical order

$$\partial^2 p(\bar x,\bar v)(u):=D^*(\partial p)(\bar x,\bar v)(u),$$3

The recursive local fractional derivative removes not only $$\partial^2 p(\bar x,\bar v)(u):=D^*(\partial p)(\bar x,\bar v)(u),$$4 but all previously detected lower-order fractional Taylor terms, thereby allowing differentiation beyond the critical order and producing an order set of nonzero local scaling exponents. The paper presents this as conceptually analogous to second-order generalized differentiation: once the first nonzero local fractional order is extracted, the next one can be obtained only after subtracting lower-order behavior, just as second derivatives in classical analysis arise after removing first-order terms (Kolwankar, 2013).

6. Computational second-order differentiation

In computational mechanics and scientific computing, second-order generalized differentiation often means differentiating an entire algorithm rather than a closed-form formula. AutoMat does this for generalized standard materials by representing the constitutive model through a Helmholtz free energy $$\partial^2 p(\bar x,\bar v)(u):=D^*(\partial p)(\bar x,\bar v)(u),$$5 and a force potential $$\partial^2 p(\bar x,\bar v)(u):=D^*(\partial p)(\bar x,\bar v)(u),$$6, then applying automatic differentiation to both the potentials and the numerical ODE integration scheme for internal variables. The consistent tangent

$$\partial^2 p(\bar x,\bar v)(u):=D^*(\partial p)(\bar x,\bar v)(u),$$7

is therefore a genuinely second-order differentiated quantity, and the paper shows that expression-level reverse mode and its extension to second-order derivatives can be implemented inside CUDA kernels (Blühdorn et al., 2020).

A more explicit Hessian-construction strategy appears in elastic locomotion. There the inverse problem is solved by Newton’s method, and the core tool is a mixed second-order differentiation algorithm that combines reverse AD with complex-step finite difference. The framework treats AD as a generic function $$\partial^2 p(\bar x,\bar v)(u):=D^*(\partial p)(\bar x,\bar v)(u),$$8 and promotes CSFD through the AD computation, so that the $$\partial^2 p(\bar x,\bar v)(u):=D^*(\partial p)(\bar x,\bar v)(u),$$9-th Hessian column is approximated by

$$\breve\partial^2 p(\bar x,\bar v)(u):=\hat D^*(\partial p)(\bar x,\bar v)(u)$$0

This avoids the memory cost of double reverse mode and makes barrier-based inverse simulation with wide-area contacts compatible with line-search Newton (Shen et al., 2024).

In quantitative finance, Vibrato plus automatic differentiation provides a related remedy for nonsmooth payoffs. Vibrato conditions on the last Gaussian time step of an Euler scheme, analytically differentiates the conditional density, and then applies AD to the resulting expression. The paper shows that direct analytical differentiation of the Vibrato formula and applying Vibrato twice are equivalent for second derivatives, while naive second-order AD on the full payoff code is unstable because option payoffs are typically not twice differentiable. In this setting, second-order generalized differentiation is inseparable from regularization: the Gaussian last-step conditioning supplies the smoothing needed for Gamma, Vanna, Vomma, and higher-order Greeks (Pagès et al., 2016).

Across these settings, second-order generalized differentiation functions less as a single operator than as a design principle: replace missing classical curvature by a structure-compatible second-order object, preserve first-order geometry, and recover the Hessian only where smoothness genuinely justifies it.