---
title: Newton Method with Successive Linearization
url: https://www.emergentmind.com/topics/newton-method-with-successive-linearization
type: topic
---

# Newton Method with Successive Linearization

Newton method with successive linearization denotes a family of iterative procedures that solve nonlinear equations, inclusions, or constrained optimization subproblems by repeatedly replacing the current nonlinear model with a local linear or piecewise linear surrogate and then solving the resulting simplified problem. In the classical smooth case this is the standard Newton step; in broader settings it includes quasi-linearization for boundary value problems, projection-based reduced Newton steps for implicit time integration, piecewise linear Newton methods for piecewise differentiable mappings, coderivative-based Newton methods for generalized equations, and Newton-like sequential linearization schemes in optimization and control [2303.06978].

## 1. Core formulation

In its standard residual form, successive linearization starts from a nonlinear system \(R(x)=0\) and, at iteration \(\ell\), replaces it by the linear model
\[
R(x^{(\ell)}) + R_x(x^{(\ell)})\,\Delta x^{(\ell)} = 0,
\]
followed by
\[
x^{(\ell+1)} = x^{(\ell)} + \Delta x^{(\ell)}.
\]
For implicit Euler applied to
\[
\dot x(t)=f(x(t),u(t),d(t),p),
\]
this takes the concrete form
\[
R_k(x_{k+1}) = x_{k+1}-x_k-f(x_{k+1},u_k,d_k,p)\,\Delta t_k,
\]
with Newton iterations generated by linearizing \(R_k\) around the current time-step iterate \(x_{k+1}^{(\ell)}\) [2303.06978]. The same perspective appears in nonlinear two-point boundary value problems, where quasi-linearization replaces the nonlinear right-hand side by a local linear function in \(u\) and \(u'\); in that setting, quasi-linearization is explicitly identified with Newton’s method for the nonlinear operator \(\mathcal F(u)=0\) [2003.03104].

A broader formulation arises for piecewise differentiable mappings. There, one does not linearize by a single Jacobian alone, but by a continuous piecewise linear model built from the evaluation program itself. Tangent models use one base point, secant models use two base points, and the secant model satisfies the bilinear error estimate
\[
\|F(x)-\lozenge_{\check x}^{\hat x}F(x)\|
\le \tfrac12 \gamma_F \|x-\check x\|\,\|x-\hat x\|,
\]
while the tangent model satisfies the quadratic estimate
\[
\|F(x)-\lozenge_{\mathring x}F(x)\|
\le \tfrac12 \gamma_F \|x-\mathring x\|^2.
\]
This turns Newton’s method with successive linearization into a method with successive piecewise linearization, where each iterate is defined as the closest root of the local piecewise linear model [1701.04368].

## 2. Smooth large-scale systems and reduced Newton steps

A representative large-scale smooth realization is the Newton-like method for implicit numerical ODE solvers in which the nonlinear residual and the full Jacobian are kept unchanged, but each Newton step is approximated in a reduced subspace [2303.06978]. Instead of solving
\[
A_k^{(\ell)} \Delta x_{k+1}^{(\ell)} = b_k^{(\ell)},
\]
the method seeks
\[
\Delta x_{k+1}^{(\ell)} \approx V_k \Delta \hat x_{k+1}^{(\ell)},
\]
projects by \(W_k^T\), and solves
\[
\hat A_k^{(\ell)} \Delta \hat x_{k+1}^{(\ell)} = \hat b_k^{(\ell)},\qquad
\hat A_k^{(\ell)}=W_k^T A_k^{(\ell)}V_k,\quad
\hat b_k^{(\ell)}=W_k^T b_k^{(\ell)}.
\]
With \(W_k=V_k\), this is a Galerkin or Petrov–Galerkin projection of the Newton step. The basis is recomputed at every time step after an initialization phase, using proper orthogonal decomposition of recent state snapshots
\[
X_k=[x_{k_h},\dots,x_k],
\]
thin SVD \(X_k=U\Sigma R^T\), and the truncation rule \(\sigma_i\ge \epsilon \Sigma_{11}\) with \(\epsilon=50\,\epsilon_m\), \(\epsilon_m=2^{-52}\). The paper uses \(N_b=\ln(n_x)\) initial full-Newton steps and \(N_h=\sqrt[3]{n_x}\) snapshots per POD window [2303.06978].

This method preserves the structure of Newton’s method but restricts the update to \(\mathrm{range}(V_k)\). Accordingly, it is described as a safeguarded, inexact Newton method whose inexactness arises from subspace restriction rather than from an approximate Jacobian. It is also stated to be neither standard quasi-Newton nor Picard: the exact residual and exact Jacobian are retained, but the dimension of the update is reduced [2303.06978].

A related smooth-dynamics use of successive linearization appears in parameter identification for ODEs via multiple shooting and generalized Gauss–Newton. There, Local Linearization replaces the nonlinear ODE on each short time slab by a linear time-varying problem,
\[
\dot y(t)=f(t_n,y_{t_n},p)+f_x(t_n,y_{t_n},p)(y(t)-y_{t_n})+f_t(t_n,y_{t_n},p)(t-t_n),
\]
solved exactly through matrix exponentials. The same local linearization is used to propagate variational equations for sensitivities, yielding accurate derivatives “with no more computational cost than that involved in the integration of the ODE” [1507.00975]. In this setting, successive linearization acts both in time and inside the outer Gauss–Newton loop.

## 3. Piecewise linear, semismooth, and generalized-equation variants

For piecewise differentiable functions defined by evaluation programs, Algorithmic Piecewise Differentiation constructs tangent and secant piecewise linear models by propagating local rules through arithmetic operations, smooth elementals, and \(|\cdot|\) [1701.04368]. Newton steps are then defined as roots of the local piecewise linear equation rather than solutions of a single linear system:
\[
x_{k+1}=N(x_k)\quad\text{or}\quad x_{k+1}=N(x_k,x_{k-1}).
\]
Under the assumption that the tangent piecewise linearization at the root is bijective on a ball, tangent mode converges quadratically,
\[
\|x_{k+1}-x^*\|\le \frac{1}{4\rho}\|x_k-x^*\|^2,
\]
while secant mode converges with order
\[
\phi=\frac{1+\sqrt{5}}{2},
\]
matching the classical secant rate in a nonsmooth setting [1701.04368].

The open Newton method weakens this requirement from local bijectivity of the piecewise linearization to local openness [1808.00213]. The paper proves a weak implicit function theorem by local mapping degree theory and shows that this weaker criterion is genuinely broader: there exist piecewise smooth functions \(f:\mathbb R^2\to\mathbb R^2\) satisfying the openness condition such that every neighborhood of the root contains a point where all elements of the Clarke Jacobian are singular. In such neighborhoods, classical semismooth Newton steps are not defined, whereas the piecewise-linear Newton method remains applicable [1808.00213]. This establishes the open Newton method as an independent algorithm rather than a reformulation of Clarke-Jacobian semismooth Newton.

A further generalization appears for generalized equations \(0\in F(x)\), where the Newton-type step linearizes both the single-valued and the multi-valued part of \(F\) by coderivative information. The resulting semismooth\(^*\) Newton method uses matrices \((A,B)\) built from coderivative graph elements and updates via
\[
x^{(k+1)}=\hat x^{(k)}-A^{-1}B\hat y^{(k)},
\]
after first approximating \((x^{(k)},0)\) by a nearby graph point \((\hat x^{(k)},\hat y^{(k)})\in\operatorname{gph}F\). Under semismoothness\(^*\) and a suitable regularity condition, local superlinear convergence follows [1904.09167]. For structured generalized equations of the form
\[
0\in f(x)+Vg(x)^\top N_D(g(x)),
\]
the paper derives an implementable variant based on a strictly convex quadratic program in the approximation step and a reduced linear system in the Newton step [1904.09167].

## 4. Inexact, constrained, and structure-preserving formulations

In regularized optimization, successive linearization often appears as successive quadratic approximation. For composite problems
\[
\min_x F(x):=f(x)+\psi(x),
\]
the outer iteration minimizes the local model
\[
Q_{H_k}^{x^k}(d)
=
\nabla f(x^k)^T d + \tfrac12 d^T H_k d + \psi(x^k+d)-\psi(x^k),
\]
with \(H_k\) allowed to be Newton, quasi-Newton, or scaled identity. The paper analyzes inexact subproblem solutions satisfying a fixed multiplicative precision criterion and proves that this suffices to retain the same order of convergence rate as the exact version. Under optimal-set strong convexity, the algorithms converge at global linear rates; for general convex problems the overall rate is \(O(1/k)\); for nonconvex problems, a first-order optimality criterion converges to zero at rate \(O(1/\sqrt{k})\) [1803.01298]. In this sense, successive linearization and successive quadratic approximation lie on a continuous spectrum from proximal gradient to proximal Newton.

In phase-field fracture, Newton’s method with successive linearization is combined with a primal–dual active set strategy for crack irreversibility. The paper formulates a complementarity system for the phase-field constraint, derives the primal–dual active set method from it, and combines this with Newton linearization of the coupled quasi-monolithic system. A linearization in the degradation function is used “to enhance the numerical stability,” and a modified combined active-set Newton approach is proposed to reduce the computational cost. Afterwards, an iteration on the linearization is designed “to iterate the problem to the monolithic limit” [2306.03501].

In computational micromagnetics, the mass-lumped midpoint scheme for the Landau–Lifshitz–Gilbert equation yields nonlinear algebraic systems at each time step. The paper compares a constraint-preserving fixed-point linearization with a Newton linearization and shows that the Newton solver leads to a considerably lower number of nonlinear iterations [2203.06445]. The midpoint discretization retains a discrete energy identity, and for skyrmion dynamics the midpoint scheme is reported to be superior to dissipative tangent plane schemes because of its conservation properties [2203.06445]. This is a structure-preserving example in which successive linearization is subordinate to a geometric time integrator.

## 5. Sequential optimization, data-driven linearization, and transformed Newton steps

In nonlinear optimization and control, successive linearization frequently appears as a Newton-like sequential convexification. For voltage control under nonlinear DistFlow constraints, a data-driven successive linearization method estimates the local sensitivity matrix \(\widetilde{\mathbf S}_k\) by weighted least squares,
\[
\widetilde{\mathbf S}_k
=
\arg\min_{\mathbf S}
\sum_{l=k-\tau+1}^k
\lambda^{k-l}\|\Delta \mathbf y_l-\mathbf S\,\Delta \mathbf u_l\|^2,
\]
then solves a convex trust-region subproblem with linearized power-flow constraint
\[
\mathbf y=\mathbf y_k+\widetilde{\mathbf S}_k(\mathbf u-\mathbf u_k),\qquad
\|\mathbf u-\mathbf u_k\|\le r_k.
\]
The paper proves convergence to a neighborhood of KKT points; if the Jacobian estimation error vanishes, any limit point is a stationary point of the exact-penalty problem, and if that point is feasible it is a KKT point of the original nonlinear program [2603.10138]. The method is explicitly positioned as a data-driven, perturbed sequential linear programming or Newton-like scheme rather than a gradient controller.

A related optimal-control formulation is successive convex programming for solar-sail trajectory optimization. There, nonlinear dynamics, nonlinear control feasibility conditions, and terminal constraints depending on free final time are all successively linearized around the previous iterate; trust regions and virtual control are added to maintain validity and feasibility [1901.02145]. The paper interprets this as a Newton-type sequential convex/linear scheme specialized to trajectory optimization.

A different scalar-root variant modifies Newton’s method by transforming the residual through generalized moment generating functions. For
\[
f(x)=0,\qquad \xi_n=x_{n+1}-x_n,
\]
the degree-0 transform is
\[
\mathcal H_0(x_n,\xi_n)
=
-\sigma(f(x_n))(f(x_n+\xi_n)-f(x_n)),
\]
and higher degrees are defined recursively by
\[
\mathcal H_k(x_n,\xi_n)=\ln\bigl[1+\mathcal H_{k+1}(x_n,\xi_n)\bigr].
\]
At each iteration one chooses the integer degree
\[
\varkappa
=
-r\!\left(\frac{h_{0,2}(x_n)}{h_{0,1}^2(x_n)}\right)
\]
to minimize local curvature, and then performs the Newton-like update
\[
x_{n+1}
=
x_n+\frac{\mathfrak H_\varkappa(x_n)}{h_{0,1}(x_n)}.
\]
For simple roots the method remains quadratically convergent, but the paper emphasizes its reduced sensitivity to poor initial guesses [2311.17269]. This is still successive linearization, but of an adaptively chosen transformed residual rather than of \(f\) itself.

## 6. Applications, performance, and limitations

The projection-based reduced Newton method for implicit Euler was demonstrated on CO\(_2\) injection into a heterogeneous oil reservoir with \(4{,}800\) to \(52{,}800\) state variables. Relative to full Newton, the Newton-like reduced method achieved speedup between \(39\%\) and \(84\%\), with mean speedup \(66\%\) and standard deviation about \(12\%\) [2303.06978]. The benefit increased with problem size, while the same residual tolerance and the same time-step sequence were used.

For successive piecewise linearization, the secant-based piecewise linear Newton method achieved the predicted convergence orders in a two-dimensional nonsmooth example: in the noise-free case, estimated convergence orders were \(\gamma\approx 2.08\) for tangent mode and \(\gamma\approx 1.65\) for secant mode; with oscillatory noise, \(\gamma\approx 2.01\) and \(\gamma\approx 1.66\) were observed [1701.04368]. The paper also reports that the secant version appears more robust to high-frequency noise.

In voltage control, the data-driven successive linearization method converged in about \(3\) iterations on the IEEE 33-bus test system and produced voltages and actions almost identical to those of the model-based convex relaxation in the time-invariant case. Over 100 random load cases, the mean cost achieved by successive linearization was reported as \(81.82\%\) lower than convex relaxation, \(98.92\%\) lower than feedback optimization, and \(99.56\%\) lower than linear control; in time-varying scenarios the average cost was \(44.33\%\) lower than feedback optimization and \(84.89\%\) lower than linear control [2603.10138].

These successes coexist with clear limitations. Projection-based reduced Newton may lose exact quadratic convergence in the full space because the step is restricted to \(\mathrm{range}(V_k)\); if the reduced space is not rich enough, the algorithm must fall back to a full Newton step [2303.06978]. Piecewise linear Newton methods require solving piecewise linear systems, and solving such systems is stated to be NP-hard in general [1701.04368]. The secant piecewise linear model also depends on the chosen evaluation program, so mathematically equivalent expressions such as \(f(x)=\log(\exp(x))\) and \(f(x)=x\) can yield different secant models [1701.04368]. Data-driven successive linearization converges only to a neighborhood of the stationary set if Jacobian estimation error remains nonzero [2603.10138]. In micromagnetics, Newton linearization does not preserve the pointwise unit-length constraint exactly, although the deviation is controlled and vanishes in the limit as discretization parameters and solver tolerances go to zero [2203.06445].

Several recurring misconceptions are addressed explicitly in the literature. A reduced Newton step based on POD is not a quasi-Newton method, because it uses the exact residual and exact Jacobian and only reduces the update dimension [2303.06978]. An open Newton method for piecewise smooth functions is not merely a semismooth Newton variant, because it can remain applicable when all nearby Clarke Jacobian elements are singular [1808.00213]. Data-driven successive linearization for nonlinear control is structurally closer to sequential linear programming or a perturbed Newton-like penalty method than to projected-gradient feedback optimization [2603.10138]. Across these variants, the unifying feature is not the particular algebraic form of the step, but the repeated replacement of the current nonlinear model by a local linear, piecewise linear, or convexified surrogate whose solution defines the next iterate.

Source: https://www.emergentmind.com/topics/newton-method-with-successive-linearization