Affine-Covariant Damped Newton Iteration

• The method unifies differential geometry, optimization, and numerical analysis to solve variational equations on manifolds with an affine-invariant damping strategy.
• It leverages intrinsic retraction and parallel transport frameworks to calculate Newton steps in a coordinate-free fashion, providing robustness on Banach manifolds.
• Empirical insights show that the adaptive, affine-covariant damping outperforms fixed-step methods by ensuring stability and accelerating convergence.

Affine-Covariant Damped Newton Iteration is a geometric and algorithmic framework for solving nonlinear variational equations and root-finding problems on manifolds—particularly those mapping into (dual) vector bundles—via Newton's method equipped with a step-size (damping) strategy that is invariant under affine coordinate changes. The method unifies ideas from differential geometry, optimization, and numerical analysis to ensure both global convergence and local superlinear (often quadratic) rates, with applications to variational problems, critical point computation, and related tasks on manifolds and infinite-dimensional settings (Weigl et al., 18 Jul 2025, Weigl et al., 2024, Hanzely et al., 2022).

1. Geometric and Analytic Framework

The method is formulated for a $C^1$ Banach manifold $X$ (potentially infinite-dimensional) and a $C^1$ vector bundle $E \to Y$ over a manifold $Y$, with dual bundle $E^* \to Y$. The root-finding problem takes the form:

$$F\colon X \to E^*, \quad F(x) = 0^*_{y(x)}, \text{ where } y(x) = p^*(F(x)) \in Y$$

Here, $F(x)$ is a covector in the fiber $E^*_{y(x)}$. The problem covers stationary equations for functionals ($F(x) = df(x)$) and general variational equations on manifolds.

Affine structure is incorporated through:

• Affine Connection ($X$0 or $X$1): Endows $X$2 with a notion of parallel transport and "straight lines" via a connection on the tangent or general vector bundle. The dual connection $X$3 acts on $X$4.
• Retraction ($X$5): A $X$6 map $X$7 generalizes the exponential map, providing an intrinsic way to update points via tangent directions.
• Transport Operator ($X$8): Parallel transport and its adjoint are used to move vectors and covectors between fibers coherently.

This geometric setup allows Newton's method to be defined in a coordinate-free, affine-invariant manner (Weigl et al., 18 Jul 2025, Weigl et al., 2024).

2. Algorithmic Formulation: Newton Step and Affine-Covariant Damping

Newton Step

At a current iterate $X$9:

1. Compute $C^1$0.
2. Use the dual connection $C^1$1 to map the derivative to the appropriate fiber:

$C^1$2

1. The Newton direction $C^1$3 solves the fiberwise Newton equation:

$C^1$4

Assuming invertibility of $C^1$5, set $C^1$6 as the undamped update (Weigl et al., 18 Jul 2025, Weigl et al., 2024).

Affine-Covariant Damping

To ensure global convergence, $C^1$7 is replaced by a fraction $C^1$8. The selection of $C^1$9 is performed in an affine-covariant manner via a "Newton path" procedure:

• Newton Path in Fiber: For fixed $E \to Y$0, find $E \to Y$1 such that

$E \to Y$2

• Residual Back-Transport: Residuals $E \to Y$3 are transported back to the fixed fiber $E \to Y$4 using the adjoint of the transport operator.
• Step Acceptance: For each candidate $E \to Y$5, solve the simplified Newton equation, compute a quality factor $E \to Y$6, and accept $E \to Y$7 if $E \to Y$8.

This mechanism achieves invariance with respect to affine coordinate changes and ensures that the update direction is consistent with the geometry of the problem (Weigl et al., 18 Jul 2025, Weigl et al., 2024, Hanzely et al., 2022).

3. Pseudocode Details and Local Convergence Analysis

The iteration alternates between solving for a Newton direction $E \to Y$9 and adjusting the damping parameter $Y$0:

1. Solve $Y$1 for $Y$2.
2. Initialize $Y$3.
3. Repeat:
   • Set $Y$4.
   • Solve for the simplified Newton-path direction $Y$5 from the affine-covariant damped condition.
   • Compute $Y$6.
   • If $Y$7, accept $Y$8; otherwise update $Y$9.
   • Fail and exit if $E^* \to Y$0.
4. Update $E^* \to Y$1.

Termination is triggered on a pure Newton step with $E^* \to Y$2 and sufficient step smallness (Weigl et al., 18 Jul 2025, Weigl et al., 2024).

4. Convergence Theory

Local Convergence

Under standard assumptions—$E^* \to Y$3 of class $E^* \to Y$4, invertibility of $E^* \to Y$5 at the solution, Lipschitz continuity of $E^* \to Y$6 and the connection, and $E^* \to Y$7 regularity of the retraction:

• Superlinear (Quadratic) Convergence: For initial points $E^* \to Y$8 sufficiently close to a nondegenerate zero $E^* \to Y$9, the iteration eventually admits undamped ($$F\colon X \to E^*, \quad F(x) = 0^*_{y(x)}, \text{ where } y(x) = p^*(F(x)) \in Y$$0) steps, and the error satisfies

$$F\colon X \to E^*, \quad F(x) = 0^*_{y(x)}, \text{ where } y(x) = p^*(F(x)) \in Y$$1

• A Posteriori Contractivity: Using the local estimator $$F\colon X \to E^*, \quad F(x) = 0^*_{y(x)}, \text{ where } y(x) = p^*(F(x)) \in Y$$2, if $$F\colon X \to E^*, \quad F(x) = 0^*_{y(x)}, \text{ where } y(x) = p^*(F(x)) \in Y$$3, then the iteration converges superlinearly.

Global Convergence

If $$F\colon X \to E^*, \quad F(x) = 0^*_{y(x)}, \text{ where } y(x) = p^*(F(x)) \in Y$$4 and $$F\colon X \to E^*, \quad F(x) = 0^*_{y(x)}, \text{ where } y(x) = p^*(F(x)) \in Y$$5 are Lipschitz on relevant level sets, every accumulation point either solves $$F\colon X \to E^*, \quad F(x) = 0^*_{y(x)}, \text{ where } y(x) = p^*(F(x)) \in Y$$6 or achieves small residual norm; the damping strategy prevents stalling at points far from the solution (Weigl et al., 18 Jul 2025, Weigl et al., 2024).

In the finite-dimensional convex case with self-concordance (as in the Affine-Invariant Cubic Newton scheme), global $$F\colon X \to E^*, \quad F(x) = 0^*_{y(x)}, \text{ where } y(x) = p^*(F(x)) \in Y$$7 rate and local quadratic rate can be shown, with explicit step-size $$F\colon X \to E^*, \quad F(x) = 0^*_{y(x)}, \text{ where } y(x) = p^*(F(x)) \in Y$$8 computable from local curvature:

$$F\colon X \to E^*, \quad F(x) = 0^*_{y(x)}, \text{ where } y(x) = p^*(F(x)) \in Y$$9

where $F(x)$0 and $F(x)$1 is the self-concordance constant (Hanzely et al., 2022).

5. Applications

Variational Problems and Functionals

When $F(x)$2 for $F(x)$3, the affine-covariant damped Newton method specializes to an optimization algorithm on manifolds, recovering classical Riemannian Newton variants and Newton-SQP steps (Weigl et al., 18 Jul 2025, Weigl et al., 2024). The Hessian is replaced by the covariant Hessian and transport by the Levi-Civita connection if $F(x)$4 is Riemannian.

Vector Fields and Fixed Point Computation

For vector fields $F(x)$5, solving $F(x)$6 follows by the same scheme, with the connection and parallel transport induced by the retraction differential (Weigl et al., 2024).

Broader Algorithmic Context

The methodology generalizes to root-finding in dual vector bundles and enables intrinsic, coordinate-free numerical algorithms for problems ranging from geometric PDEs to critical point computation and model reduction.

6. Comparison, Invariance, and Practical Considerations

Affine and Coordinate Invariance

A fundamental property of affine-covariant damped Newton iteration is invariance under affine coordinate changes: the outcomes and steps are independent of local trivialization or choice of coordinates. This is achieved by explicit use of connection maps, bundle morphisms, and local Hessian-induced metrics (Weigl et al., 18 Jul 2025, Weigl et al., 2024, Hanzely et al., 2022).

Algorithmic Comparison

The affine-covariant damped Newton method matches or improves upon global and local convergence rates achieved by cubic-regularized Newton methods, trust-region schemes, and regularized second-order methods. It dispenses with auxiliary subproblems or line searches by relying solely on geometric quantities intrinsic to the problem (Hanzely et al., 2022).

Implementation and Empirical Insights

Empirically, affine-covariant damped Newton iterations demonstrate competitive wall-clock and iteration count performance versus cubic and regularized methods in convex optimization scenarios, especially due to their closed-form damping factor and invariance properties. The step-size adapts automatically, in contrast to fixed-step schemes which may exhibit instability or slow progression (Hanzely et al., 2022).

7. Relation to Classical Results and Extensions

This framework generalizes classical damped Newton methods on $F(x)$7 to general Banach manifolds and bundles, unifying geometric and analytic approaches. For $F(x)$8 with a Riemannian metric, the theory recovers Newton methods of Deuflhard, Gabay, and Smith in manifold optimization. The approach is extensible to infinite-dimensional and PDE contexts, as demonstrated in variational equation applications (Weigl et al., 18 Jul 2025, Weigl et al., 2024).