Newton Method with Successive Linearization

• Newton Method with Successive Linearization is a family of iterative techniques that replaces complex nonlinear problems with local linear or piecewise linear models to facilitate solution finding.
• It employs both classical smooth Newton steps and advanced variants like reduced subspace projections and piecewise linearizations to handle nonsmooth and large-scale dynamics.
• The approach has broad applications in ODE solvers, optimal control, and micromagnetics while addressing challenges such as NP-hard subproblems and approximation errors.

Newton method with successive linearization denotes a family of iterative procedures that solve nonlinear equations, inclusions, or constrained optimization subproblems by repeatedly replacing the current nonlinear model with a local linear or piecewise linear surrogate and then solving the resulting simplified problem. In the classical smooth case this is the standard Newton step; in broader settings it includes quasi-linearization for boundary value problems, projection-based reduced Newton steps for implicit time integration, piecewise linear Newton methods for piecewise differentiable mappings, coderivative-based Newton methods for generalized equations, and Newton-like sequential linearization schemes in optimization and control (Ritschel, 2023).

1. Core formulation

In its standard residual form, successive linearization starts from a nonlinear system $R(x)=0$ and, at iteration $\ell$, replaces it by the linear model

$$R(x^{(\ell)}) + R_x(x^{(\ell)})\,\Delta x^{(\ell)} = 0,$$

followed by

$x^{(\ell+1)} = x^{(\ell)} + \Delta x^{(\ell)}.$

For implicit Euler applied to

$\dot x(t)=f(x(t),u(t),d(t),p),$

this takes the concrete form

$$R_k(x_{k+1}) = x_{k+1}-x_k-f(x_{k+1},u_k,d_k,p)\,\Delta t_k,$$

with Newton iterations generated by linearizing $R_k$ around the current time-step iterate $x_{k+1}^{(\ell)}$ (Ritschel, 2023). The same perspective appears in nonlinear two-point boundary value problems, where quasi-linearization replaces the nonlinear right-hand side by a local linear function in $u$ and $u'$; in that setting, quasi-linearization is explicitly identified with Newton’s method for the nonlinear operator $\ell$0 (Faragó et al., 2020).

A broader formulation arises for piecewise differentiable mappings. There, one does not linearize by a single Jacobian alone, but by a continuous piecewise linear model built from the evaluation program itself. Tangent models use one base point, secant models use two base points, and the secant model satisfies the bilinear error estimate

$\ell$1

while the tangent model satisfies the quadratic estimate

$\ell$2

This turns Newton’s method with successive linearization into a method with successive piecewise linearization, where each iterate is defined as the closest root of the local piecewise linear model (Griewank et al., 2017).

2. Smooth large-scale systems and reduced Newton steps

A representative large-scale smooth realization is the Newton-like method for implicit numerical ODE solvers in which the nonlinear residual and the full Jacobian are kept unchanged, but each Newton step is approximated in a reduced subspace (Ritschel, 2023). Instead of solving

$\ell$3

the method seeks

$\ell$4

projects by $\ell$5, and solves

$\ell$6

With $\ell$7, this is a Galerkin or Petrov–Galerkin projection of the Newton step. The basis is recomputed at every time step after an initialization phase, using proper orthogonal decomposition of recent state snapshots

$\ell$8

thin SVD $\ell$9, and the truncation rule $$R(x^{(\ell)}) + R_x(x^{(\ell)})\,\Delta x^{(\ell)} = 0,$$0 with $$R(x^{(\ell)}) + R_x(x^{(\ell)})\,\Delta x^{(\ell)} = 0,$$1, $$R(x^{(\ell)}) + R_x(x^{(\ell)})\,\Delta x^{(\ell)} = 0,$$2. The paper uses $$R(x^{(\ell)}) + R_x(x^{(\ell)})\,\Delta x^{(\ell)} = 0,$$3 initial full-Newton steps and $$R(x^{(\ell)}) + R_x(x^{(\ell)})\,\Delta x^{(\ell)} = 0,$$4 snapshots per POD window (Ritschel, 2023).

This method preserves the structure of Newton’s method but restricts the update to $$R(x^{(\ell)}) + R_x(x^{(\ell)})\,\Delta x^{(\ell)} = 0,$$5. Accordingly, it is described as a safeguarded, inexact Newton method whose inexactness arises from subspace restriction rather than from an approximate Jacobian. It is also stated to be neither standard quasi-Newton nor Picard: the exact residual and exact Jacobian are retained, but the dimension of the update is reduced (Ritschel, 2023).

A related smooth-dynamics use of successive linearization appears in parameter identification for ODEs via multiple shooting and generalized Gauss–Newton. There, Local Linearization replaces the nonlinear ODE on each short time slab by a linear time-varying problem,

$$R(x^{(\ell)}) + R_x(x^{(\ell)})\,\Delta x^{(\ell)} = 0,$$6

solved exactly through matrix exponentials. The same local linearization is used to propagate variational equations for sensitivities, yielding accurate derivatives “with no more computational cost than that involved in the integration of the ODE” (Carbonell et al., 2015). In this setting, successive linearization acts both in time and inside the outer Gauss–Newton loop.

3. Piecewise linear, semismooth, and generalized-equation variants

For piecewise differentiable functions defined by evaluation programs, Algorithmic Piecewise Differentiation constructs tangent and secant piecewise linear models by propagating local rules through arithmetic operations, smooth elementals, and $$R(x^{(\ell)}) + R_x(x^{(\ell)})\,\Delta x^{(\ell)} = 0,$$7 (Griewank et al., 2017). Newton steps are then defined as roots of the local piecewise linear equation rather than solutions of a single linear system: $$R(x^{(\ell)}) + R_x(x^{(\ell)})\,\Delta x^{(\ell)} = 0,$$8 Under the assumption that the tangent piecewise linearization at the root is bijective on a ball, tangent mode converges quadratically,

$$R(x^{(\ell)}) + R_x(x^{(\ell)})\,\Delta x^{(\ell)} = 0,$$9

while secant mode converges with order

$x^{(\ell+1)} = x^{(\ell)} + \Delta x^{(\ell)}.$0

matching the classical secant rate in a nonsmooth setting (Griewank et al., 2017).

The open Newton method weakens this requirement from local bijectivity of the piecewise linearization to local openness (Radons et al., 2018). The paper proves a weak implicit function theorem by local mapping degree theory and shows that this weaker criterion is genuinely broader: there exist piecewise smooth functions $x^{(\ell+1)} = x^{(\ell)} + \Delta x^{(\ell)}.$1 satisfying the openness condition such that every neighborhood of the root contains a point where all elements of the Clarke Jacobian are singular. In such neighborhoods, classical semismooth Newton steps are not defined, whereas the piecewise-linear Newton method remains applicable (Radons et al., 2018). This establishes the open Newton method as an independent algorithm rather than a reformulation of Clarke-Jacobian semismooth Newton.

A further generalization appears for generalized equations $x^{(\ell+1)} = x^{(\ell)} + \Delta x^{(\ell)}.$2, where the Newton-type step linearizes both the single-valued and the multi-valued part of $x^{(\ell+1)} = x^{(\ell)} + \Delta x^{(\ell)}.$3 by coderivative information. The resulting semismooth$x^{(\ell+1)} = x^{(\ell)} + \Delta x^{(\ell)}.$4 Newton method uses matrices $x^{(\ell+1)} = x^{(\ell)} + \Delta x^{(\ell)}.$5 built from coderivative graph elements and updates via

$x^{(\ell+1)} = x^{(\ell)} + \Delta x^{(\ell)}.$6

after first approximating $x^{(\ell+1)} = x^{(\ell)} + \Delta x^{(\ell)}.$7 by a nearby graph point $x^{(\ell+1)} = x^{(\ell)} + \Delta x^{(\ell)}.$8. Under semismoothness$x^{(\ell+1)} = x^{(\ell)} + \Delta x^{(\ell)}.$9 and a suitable regularity condition, local superlinear convergence follows (Gfrerer et al., 2019). For structured generalized equations of the form

$\dot x(t)=f(x(t),u(t),d(t),p),$0

the paper derives an implementable variant based on a strictly convex quadratic program in the approximation step and a reduced linear system in the Newton step (Gfrerer et al., 2019).

4. Inexact, constrained, and structure-preserving formulations

In regularized optimization, successive linearization often appears as successive quadratic approximation. For composite problems

$\dot x(t)=f(x(t),u(t),d(t),p),$1

the outer iteration minimizes the local model

$\dot x(t)=f(x(t),u(t),d(t),p),$2

with $\dot x(t)=f(x(t),u(t),d(t),p),$3 allowed to be Newton, quasi-Newton, or scaled identity. The paper analyzes inexact subproblem solutions satisfying a fixed multiplicative precision criterion and proves that this suffices to retain the same order of convergence rate as the exact version. Under optimal-set strong convexity, the algorithms converge at global linear rates; for general convex problems the overall rate is $\dot x(t)=f(x(t),u(t),d(t),p),$4; for nonconvex problems, a first-order optimality criterion converges to zero at rate $\dot x(t)=f(x(t),u(t),d(t),p),$5 (Lee et al., 2018). In this sense, successive linearization and successive quadratic approximation lie on a continuous spectrum from proximal gradient to proximal Newton.

In phase-field fracture, Newton’s method with successive linearization is combined with a primal–dual active set strategy for crack irreversibility. The paper formulates a complementarity system for the phase-field constraint, derives the primal–dual active set method from it, and combines this with Newton linearization of the coupled quasi-monolithic system. A linearization in the degradation function is used “to enhance the numerical stability,” and a modified combined active-set Newton approach is proposed to reduce the computational cost. Afterwards, an iteration on the linearization is designed “to iterate the problem to the monolithic limit” (Kolditz et al., 2023).

In computational micromagnetics, the mass-lumped midpoint scheme for the Landau–Lifshitz–Gilbert equation yields nonlinear algebraic systems at each time step. The paper compares a constraint-preserving fixed-point linearization with a Newton linearization and shows that the Newton solver leads to a considerably lower number of nonlinear iterations (Fratta et al., 2022). The midpoint discretization retains a discrete energy identity, and for skyrmion dynamics the midpoint scheme is reported to be superior to dissipative tangent plane schemes because of its conservation properties (Fratta et al., 2022). This is a structure-preserving example in which successive linearization is subordinate to a geometric time integrator.

5. Sequential optimization, data-driven linearization, and transformed Newton steps

In nonlinear optimization and control, successive linearization frequently appears as a Newton-like sequential convexification. For voltage control under nonlinear DistFlow constraints, a data-driven successive linearization method estimates the local sensitivity matrix $\dot x(t)=f(x(t),u(t),d(t),p),$6 by weighted least squares,

$\dot x(t)=f(x(t),u(t),d(t),p),$7

then solves a convex trust-region subproblem with linearized power-flow constraint

$\dot x(t)=f(x(t),u(t),d(t),p),$8

The paper proves convergence to a neighborhood of KKT points; if the Jacobian estimation error vanishes, any limit point is a stationary point of the exact-penalty problem, and if that point is feasible it is a KKT point of the original nonlinear program (Dong et al., 10 Mar 2026). The method is explicitly positioned as a data-driven, perturbed sequential linear programming or Newton-like scheme rather than a gradient controller.

A related optimal-control formulation is successive convex programming for solar-sail trajectory optimization. There, nonlinear dynamics, nonlinear control feasibility conditions, and terminal constraints depending on free final time are all successively linearized around the previous iterate; trust regions and virtual control are added to maintain validity and feasibility (Song et al., 2019). The paper interprets this as a Newton-type sequential convex/linear scheme specialized to trajectory optimization.

A different scalar-root variant modifies Newton’s method by transforming the residual through generalized moment generating functions. For

$\dot x(t)=f(x(t),u(t),d(t),p),$9

the degree-0 transform is

$$R_k(x_{k+1}) = x_{k+1}-x_k-f(x_{k+1},u_k,d_k,p)\,\Delta t_k,$$0

and higher degrees are defined recursively by

$$R_k(x_{k+1}) = x_{k+1}-x_k-f(x_{k+1},u_k,d_k,p)\,\Delta t_k,$$1

At each iteration one chooses the integer degree

$$R_k(x_{k+1}) = x_{k+1}-x_k-f(x_{k+1},u_k,d_k,p)\,\Delta t_k,$$2

to minimize local curvature, and then performs the Newton-like update

$$R_k(x_{k+1}) = x_{k+1}-x_k-f(x_{k+1},u_k,d_k,p)\,\Delta t_k,$$3

For simple roots the method remains quadratically convergent, but the paper emphasizes its reduced sensitivity to poor initial guesses (Herzog, 2023). This is still successive linearization, but of an adaptively chosen transformed residual rather than of $$R_k(x_{k+1}) = x_{k+1}-x_k-f(x_{k+1},u_k,d_k,p)\,\Delta t_k,$$4 itself.

6. Applications, performance, and limitations

The projection-based reduced Newton method for implicit Euler was demonstrated on CO$$R_k(x_{k+1}) = x_{k+1}-x_k-f(x_{k+1},u_k,d_k,p)\,\Delta t_k,$$5 injection into a heterogeneous oil reservoir with $$R_k(x_{k+1}) = x_{k+1}-x_k-f(x_{k+1},u_k,d_k,p)\,\Delta t_k,$$6 to $$R_k(x_{k+1}) = x_{k+1}-x_k-f(x_{k+1},u_k,d_k,p)\,\Delta t_k,$$7 state variables. Relative to full Newton, the Newton-like reduced method achieved speedup between $$R_k(x_{k+1}) = x_{k+1}-x_k-f(x_{k+1},u_k,d_k,p)\,\Delta t_k,$$8 and $$R_k(x_{k+1}) = x_{k+1}-x_k-f(x_{k+1},u_k,d_k,p)\,\Delta t_k,$$9, with mean speedup $R_k$0 and standard deviation about $R_k$1 (Ritschel, 2023). The benefit increased with problem size, while the same residual tolerance and the same time-step sequence were used.

For successive piecewise linearization, the secant-based piecewise linear Newton method achieved the predicted convergence orders in a two-dimensional nonsmooth example: in the noise-free case, estimated convergence orders were $R_k$2 for tangent mode and $R_k$3 for secant mode; with oscillatory noise, $R_k$4 and $R_k$5 were observed (Griewank et al., 2017). The paper also reports that the secant version appears more robust to high-frequency noise.

In voltage control, the data-driven successive linearization method converged in about $R_k$6 iterations on the IEEE 33-bus test system and produced voltages and actions almost identical to those of the model-based convex relaxation in the time-invariant case. Over 100 random load cases, the mean cost achieved by successive linearization was reported as $R_k$7 lower than convex relaxation, $R_k$8 lower than feedback optimization, and $R_k$9 lower than linear control; in time-varying scenarios the average cost was $x_{k+1}^{(\ell)}$0 lower than feedback optimization and $x_{k+1}^{(\ell)}$1 lower than linear control (Dong et al., 10 Mar 2026).

These successes coexist with clear limitations. Projection-based reduced Newton may lose exact quadratic convergence in the full space because the step is restricted to $x_{k+1}^{(\ell)}$2; if the reduced space is not rich enough, the algorithm must fall back to a full Newton step (Ritschel, 2023). Piecewise linear Newton methods require solving piecewise linear systems, and solving such systems is stated to be NP-hard in general (Griewank et al., 2017). The secant piecewise linear model also depends on the chosen evaluation program, so mathematically equivalent expressions such as $x_{k+1}^{(\ell)}$3 and $x_{k+1}^{(\ell)}$4 can yield different secant models (Griewank et al., 2017). Data-driven successive linearization converges only to a neighborhood of the stationary set if Jacobian estimation error remains nonzero (Dong et al., 10 Mar 2026). In micromagnetics, Newton linearization does not preserve the pointwise unit-length constraint exactly, although the deviation is controlled and vanishes in the limit as discretization parameters and solver tolerances go to zero (Fratta et al., 2022).

Several recurring misconceptions are addressed explicitly in the literature. A reduced Newton step based on POD is not a quasi-Newton method, because it uses the exact residual and exact Jacobian and only reduces the update dimension (Ritschel, 2023). An open Newton method for piecewise smooth functions is not merely a semismooth Newton variant, because it can remain applicable when all nearby Clarke Jacobian elements are singular (Radons et al., 2018). Data-driven successive linearization for nonlinear control is structurally closer to sequential linear programming or a perturbed Newton-like penalty method than to projected-gradient feedback optimization (Dong et al., 10 Mar 2026). Across these variants, the unifying feature is not the particular algebraic form of the step, but the repeated replacement of the current nonlinear model by a local linear, piecewise linear, or convexified surrogate whose solution defines the next iterate.