Perturbed Newton Methods

• Perturbed Newton Methods are Newton-type algorithms that incorporate controlled perturbations (e.g., inexact solves, regularization, damping) to handle singularity and noise.
• They unify approaches like classical Newton, inexact Newton, and Levenberg–Marquardt by varying matrix or right-hand-side perturbations to improve local convergence.
• These methods are applied in equilibrium problems, optimization, inverse scattering, and graphics simulation, often enhanced by stabilization techniques like ISS and projection.

Perturbed Newton methods (pNMs) are Newton-type schemes in which the ideal Newton step is modified by inexactness, regularization, damping, projection, approximate derivatives, or explicit disturbance terms. In the recent literature, pNMs are formulated both as perturbed linear solves,

$[f'(x_k)+\Omega_k]\,d=-f(x_k)+\omega_k,$

followed by $x_{k+1}=x_k+w_{k+1}$, and as disturbed equilibrium iterations $z^{k+1}=f(z^k,v^k)$ analyzed by input-to-state stability (ISS) techniques (Dallas, 17 Aug 2025, Liu et al., 2024). Across these formulations, this suggests a shared objective: to retain Newton-type fast local behavior while making the iteration usable under singularity, indefiniteness, noise, inexact subproblem solves, moving problem data, and large-scale computational constraints.

1. Conceptual scope and perturbation mechanisms

In the root-finding literature, pNMs are treated as a unifying framework that includes Newton, inexact Newton, Levenberg–Marquardt (LM), and quasi-Newton-type methods. Within that framework, the perturbation can enter through a matrix term $\Omega_k$, a right-hand-side perturbation $\omega_k$, or both. Newton corresponds to $\Omega_k=0,\ \omega_k=0$; inexact Newton uses $\Omega_k=0,\ \omega_k\neq 0$; and LM can be written with $\Omega_k=\mu_k f'(x_k)^{-T}$ and $\omega_k=0$ when $f'(x_k)^T$ is invertible (Dallas, 17 Aug 2025).

In equilibrium problems, the perturbation is interpreted more broadly. The disturbances $x_{k+1}=x_k+w_{k+1}$0 or residuals $x_{k+1}=x_k+w_{k+1}$1 represent inexact evaluation of gradients, Jacobians, or Hessians, numerical residuals from approximately solving the Newton subproblem, or external disturbances and parameter variations in online tracking settings. The paper on Nash equilibrium (NE) and generalized Nash equilibrium (GNE) problems adopts the dynamic-system viewpoint $x_{k+1}=x_k+w_{k+1}$2 and makes local ISS around the equilibrium solution the central robustness notion (Liu et al., 2024).

Other problem classes realize perturbation differently. In large-scale separable convex optimization, the perturbation arises because the primal subproblem is solved only up to a prescribed accuracy, so the dual Newton direction is built from approximate gradient and Hessian information. In nonlinear simulation, the perturbation is a deliberate projection of element Hessians to restore descent robustness when the assembled Hessian is indefinite. In nonlinear inverse problems, regularization and inexact inner Krylov solves perturb the Newton correction equations by design (Dinh et al., 2011, Fernández-Fernández et al., 27 May 2025, Hohage et al., 2010).

This breadth suggests that “perturbed Newton method” is best understood as a family of Newton-type constructions rather than a single canonical iteration.

2. Problem formulations and representative update equations

For NE problems, the equilibrium conditions can be written as a variational inequality

$x_{k+1}=x_k+w_{k+1}$3

with pseudogradient

$x_{k+1}=x_k+w_{k+1}$4

or equivalently as the generalized equation

$x_{k+1}=x_k+w_{k+1}$5

The nominal Josephy–Newton step linearizes $x_{k+1}=x_k+w_{k+1}$6 and solves

$x_{k+1}=x_k+w_{k+1}$7

where $x_{k+1}=x_k+w_{k+1}$8 is the game Hessian assembled from the second partial derivatives of the players’ objectives. For GNE problems with coupled constraints, the same reduction is generally unavailable because the problem yields a QVI rather than a standard VI. The coupled-constraint case is therefore reformulated through the KKT system and a semismooth complementarity equation

$x_{k+1}=x_k+w_{k+1}$9

with perturbed semismooth Newton update

$z^{k+1}=f(z^k,v^k)$0

(Liu et al., 2024).

In inexact path-following for separable convex optimization, the primal-dual decomposition is also cast in Newton-type form, but at the dual level. After barrier smoothing, the dual master problem uses the approximate primal point $z^{k+1}=f(z^k,v^k)$1 to define

$z^{k+1}=f(z^k,v^k)$2

These are explicitly not the true derivatives of an exact smooth function; they are used as perturbed Newton ingredients inside two phases: inexact perturbed damped Newton-type iterations for initialization and inexact perturbed full-step Newton-type iterations for the path-following phase (Dinh et al., 2011).

Regularized Newton methods for nonlinear inverse scattering employ an outer Newton linearization together with Tikhonov-type damping. With $z^{k+1}=f(z^k,v^k)$3, the correction $z^{k+1}=f(z^k,v^k)$4 is defined by

$z^{k+1}=f(z^k,v^k)$5

where

$z^{k+1}=f(z^k,v^k)$6

The choices $z^{k+1}=f(z^k,v^k)$7 and $z^{k+1}=f(z^k,v^k)$8 recover, respectively, Levenberg–Marquardt and IRGNM (Hohage et al., 2010).

A distinct formulation appears in graphics simulation. There the Newton equation $z^{k+1}=f(z^k,v^k)$9 is modified by projecting selected element Hessians before assembly. Standard Projected Newton replaces every element $\Omega_k$0 by

$\Omega_k$1

whereas Progressively Projected Newton projects only elements satisfying

$\Omega_k$2

The perturbation is therefore not an approximation error but a curvature modification chosen to secure a descent direction while keeping the global Hessian closer to its original form (Fernández-Fernández et al., 27 May 2025).

3. Regularity, stability, and local fast convergence

The NE and GNE literature develops local robustness in terms of ISS. For NE problems, if $\Omega_k$3 is a stable NE, then the perturbed Josephy–Newton iteration is locally ISS around $\Omega_k$4: there exist $\Omega_k$5 and constants $\Omega_k$6 such that

$\Omega_k$7

The same analysis yields a quadratic-type bound,

$\Omega_k$8

which exhibits nominal Q-quadratic convergence together with an additive disturbance term. For GNE problems, if $\Omega_k$9 is a quasi-regular solution of the semismooth KKT system, then the perturbed semismooth Newton method is locally ISS: $\omega_k$0 On the NE side, the paper emphasizes that VI stability rather than strong regularity drives the result: strict semicopositivity of the game Hessian on the critical cone yields isolatedness and stability of the NE, and this condition is weaker than strict monotonicity of the pseudogradient; positive definiteness implies the semicopositivity condition, but not conversely (Liu et al., 2024).

A related but more abstract local theory is built around perturbation mappings for generalized nonlinear programs. The mappings $\omega_k$1, $\omega_k$2, and $\omega_k$3 encode multiplier-free and multiplier-dependent stationarity perturbations, and isolated calmness of these mappings is the decisive local regularity property for semismooth* Newton-type methods. The derivative criteria

$\omega_k$4

characterize isolated calmness at and around a point under local closedness assumptions. Under the composite qualification condition

$\omega_k$5

isolated calmness of $\omega_k$6 is tied to nonexistence of critical multipliers together with a fuzzy inner calm* condition; in standard nonlinear programs, the fuzzy inner calm* requirement holds automatically, so isolated calmness becomes equivalent to absence of critical multipliers under the usual CQ. These results provide sufficient conditions for local superlinear convergence of the semismooth* Newton method (Benko et al., 2024).

In singular root-finding, the corresponding local regularity is 2-regularity. If $\omega_k$7 is singular with one-dimensional null space $\omega_k$8, then 2-regularity along $\omega_k$9 means that

$\Omega_k=0,\ \omega_k=0$0

is invertible as a map from $\Omega_k=0,\ \omega_k=0$1 to $\Omega_k=0,\ \omega_k=0$2. Under this hypothesis, Anderson-accelerated pNMs with $\Omega_k=0,\ \omega_k=0$3-safeguarding converge locally linearly in a starlike domain of convergence, while the range-space error component is quadratically small: $\Omega_k=0,\ \omega_k=0$4 For locally superlinear fixed-point iterations, the same safeguarding preserves the order of convergence rather than degrading it (Dallas, 17 Aug 2025).

4. Globalization, damping, projection, and safeguarding

Several pNM lines of work are explicitly designed to enlarge Newton’s convergence domain. For polynomial root finding, the Robust Newton Method (RNM) replaces the classical step $\Omega_k=0,\ \omega_k=0$5 by a perturbation chosen from descent sectors of the squared modulus $\Omega_k=0,\ \omega_k=0$6. In the generic case $\Omega_k=0,\ \omega_k=0$7, the iterate lies on the line segment joining $\Omega_k=0,\ \omega_k=0$8 to the classical Newton iterate: $\Omega_k=0,\ \omega_k=0$9 so the method follows the Newton direction with a shrinking factor at most $\Omega_k=0,\ \omega_k\neq 0$0. The Modulus Reduction Theorem yields an a priori decrease in $\Omega_k=0,\ \omega_k\neq 0$1, and in $\Omega_k=0,\ \omega_k\neq 0$2 iterations, independently of the degree of $\Omega_k=0,\ \omega_k\neq 0$3, either $\Omega_k=0,\ \omega_k\neq 0$4 or $\Omega_k=0,\ \omega_k\neq 0$5. RNM converges globally to a root or a critical point; a modified RNM that switches to a higher-order descent rule near critical points necessarily converges to a root (Kalantari, 2020).

A different globalization strategy appears in majorant-based damped Newton methods for nonlinear equations $\Omega_k=0,\ \omega_k\neq 0$6. The step is

$\Omega_k=0,\ \omega_k\neq 0$7

with explicit step-size choices

$\Omega_k=0,\ \omega_k\neq 0$8

or

$\Omega_k=0,\ \omega_k\neq 0$9

These formulas are obtained by minimizing an upper bound on the next residual. The resulting algorithms interpolate between damped Newton when the residual is large and pure Newton when it is small. Under global Lipschitz and metric-regularity assumptions, the first algorithm converges for any initial point; the analysis also gives an explicit bound on the number of damped steps before the pure Newton phase begins (Polyak et al., 2017).

Projection-based globalization addresses indefiniteness rather than poor initialization. In graphics simulation, a Newton step is a descent direction if the Hessian is SPD. Standard Projected Newton achieves this by projecting every element Hessian, but unconditional projection can be harmful because it distorts the true Hessian too much, removing useful negative curvature information and slowing convergence. Progressively Projected Newton starts from the unmodified Hessian, detects indefiniteness through an SPD-capable linear solver, and progressively projects only a residual-selected subset of element Hessians. This preserves more curvature than both standard Projected Newton and Project-on-Demand Newton (Fernández-Fernández et al., 27 May 2025).

Safeguarding plays the analogous role for acceleration. In Anderson acceleration of pNMs, the $\Omega_k=\mu_k f'(x_k)^{-T}$0-safeguarded update uses a damping factor $\Omega_k=\mu_k f'(x_k)^{-T}$1 on the Anderson correction. The ratios

$\Omega_k=\mu_k f'(x_k)^{-T}$2

with $\Omega_k=\mu_k f'(x_k)^{-T}$3, shrink automatically when the underlying pNM is already converging rapidly, so the method can reduce or turn off the Anderson step and preserve the original superlinear order (Dallas, 17 Aug 2025).

5. Distributed, decomposition, and inexact large-scale computation

A major theme in pNM research is that the Newton step need not be computed exactly, centrally, or monolithically. For NE problems, agent-distributed Josephy–Newton updates are constructed so that agent $\Omega_k=\mu_k f'(x_k)^{-T}$4 solves

$\Omega_k=\mu_k f'(x_k)^{-T}$5

This uses only $\Omega_k=\mu_k f'(x_k)^{-T}$6, $\Omega_k=\mu_k f'(x_k)^{-T}$7, $\Omega_k=\mu_k f'(x_k)^{-T}$8, and the current iterates of the other players. Two constructions are given: a local distributed Josephy–Newton scheme with Q-quadratic local convergence, and a best-response plus Josephy–Newton strategy. For GNE problems, each agent analogously updates only its own primal-dual block through a local semismooth Newton step derived from the agent’s KKT residual (Liu et al., 2024).

In large-scale separable convex optimization, inexactness is built into the path-following architecture. The dual algorithm has two phases. Phase 1 computes an initial point by inexact perturbed damped Newton-type iterations and terminates once the inexact Newton decrement is below a threshold $\Omega_k=\mu_k f'(x_k)^{-T}$9. Phase 2 performs inexact perturbed full-step Newton-type path-following while reducing the barrier parameter by

$\omega_k=0$0

The method allows both approximate Hessian matrices and approximate gradient vectors, both derived from inexact primal solves. The Phase 2 iteration complexity to obtain a $\omega_k=0$1-solution is bounded, and the worst-case complexity is

$\omega_k=0$2

where $\omega_k=0$3 is the barrier parameter (Dinh et al., 2011).

Regularized Newton methods for inverse medium scattering illustrate a different large-scale pattern: an outer Newton iteration with an inner Krylov solver and a dynamically updated spectral preconditioner. The preconditioner

$\omega_k=0$4

is assembled from approximate eigenpairs obtained from Lanczos Ritz information generated by CG. Because $\omega_k=0$5, a preconditioner built early may lose effectiveness later; the update mechanism restores spectral clustering near $\omega_k=0$6 for the preconditioned operator. The same spectral information is also used to implement a computationally cheap Lepskiĭ-type stopping rule (Hohage et al., 2010).

6. Applications, empirical behavior, and limitations

The equilibrium-tracking application developed most explicitly is constrained game-theoretic MPC. Each sample-time optimization is an NE or GNE problem, but the solver performs only $\omega_k=0$7 Newton-type iterations per time step: $\omega_k=0$8 If $\omega_k=0$9, the tracking error satisfies

$f'(x_k)^T$0

with $f'(x_k)^T$1. The resulting real-time solver is both time-distributed and agent-distributed, and the paper proves bounded solution tracking errors (Liu et al., 2024).

In graphics simulation, Progressively Projected Newton is evaluated against Projected Newton and Project-on-Demand Newton on contact-free and contact-rich deformables, co-dimensional, and rigid-body simulations. The reported headline result is that PPN consistently performs fewer than $f'(x_k)^T$2 of the projections required by PN or PDN and, in the vast majority of cases, converges in fewer Newton iterations. The reported speedups are up to $f'(x_k)^T$3 over PN and $f'(x_k)^T$4 over the best alternative. The notable exceptions are simulations with very large time steps and quasistatics, where PN remains a better choice (Fernández-Fernández et al., 27 May 2025).

In nonlinear inverse scattering, updated spectral preconditioning reduces inner Krylov work substantially. For the acoustic example with $f'(x_k)^T$5, total inner CG steps were reported as $f'(x_k)^T$6 for standard IRGNM, $f'(x_k)^T$7 for preconditioned frozen IRGNM without updating, $f'(x_k)^T$8 for updated preconditioned IRGNM method I, and $f'(x_k)^T$9 for method II. In one example, the updated method reached an $x_{k+1}=x_k+w_{k+1}$00-error of about $x_{k+1}=x_k+w_{k+1}$01 about $x_{k+1}=x_k+w_{k+1}$02 times earlier than Newton-CG. For random Gaussian noise, the discrepancy principle often stops too early, whereas Lepskiĭ stopping is much closer to the optimal stopping index and yields more stable reconstructions (Hohage et al., 2010).

Anderson-accelerated pNMs display a similarly mixed empirical picture. On singular Chandrasekhar and near-bifurcation flow problems, AA can produce strong speedups, but large-depth unsafeguarded AA can fail or become unstable. Near nonsingular regimes, unsafeguarded AA can slow convergence when the base method is already fast. The $x_{k+1}=x_k+w_{k+1}$03-safeguarded variants stabilize convergence, preserve fast local behavior, and can effectively turn off AA when the underlying pNM is already superlinear (Dallas, 17 Aug 2025).

Taken together, these results make two limitations explicit. First, perturbation is not uniformly beneficial: excessive Hessian projection, overly aggressive acceleration, or poorly timed freezing of Jacobians can degrade performance. Second, the relevant regularity notion is problem-dependent: stable VI solutions, quasi-regular KKT points, isolated calmness of perturbation mappings, 2-regularity, and self-concordant barrier structure play analogous roles in different subfields, but they are not interchangeable.