Wolfe Conditions in Optimization

• Wolfe Conditions are line-search criteria that select a step length by combining the Armijo sufficient-decrease rule with a curvature inequality, ensuring effective descent in optimization.
• They exist in weak and strong variants; the weak form relaxes the curvature requirement, aiding methods like the Dai–Yuan update in preserving global descent.
• Extensions to Riemannian manifolds and adaptations in spectral conjugate-gradient and Newton methods reveal practical implementations that enhance convergence and stability.

Searching arXiv for the supplied paper and closely related Wolfe-condition papers to ground the article in cited sources. Wolfe conditions are line-search acceptance criteria for selecting a step length along a descent direction in iterative optimization. In the standard Euclidean setting, they combine an Armijo sufficient-decrease inequality with a curvature inequality involving directional derivatives, with constants $0<c_1<c_2<1$. The literature represented here distinguishes the weak Wolfe and strong Wolfe variants, analyzes their role in conjugate-gradient global convergence, extends them to optimization on Riemannian manifolds via retractions and vector transports, and adapts them to modified spectral conjugate-gradient, adaptive SOR, and coderivative-based Newton methods (Sato, 2014).

1. Euclidean formulation

For minimizing $f:\mathbb R^n\to\mathbb R$ along a descent direction $d_k$, the standard line search seeks $\alpha_k>0$ satisfying the Armijo condition

$$f(x_k+\alpha_k d_k)\;\le\;f(x_k) + c_1\,\alpha_k\,\nabla f(x_k)^T d_k, \qquad 0<c_1<1,$$

together with a curvature condition (Sato, 2014). In the standard Wolfe form, the curvature inequality is

$$\nabla f(x_k+\alpha_k d_k)^T d_k \;\ge\;c_2\,\nabla f(x_k)^T d_k, \qquad c_1<c_2<1,$$

which is also the formulation used in a modified spectral conjugate-gradient context (Wu et al., 2023).

The first inequality enforces sufficient decrease in the objective along the trial step. The second controls how far the step progresses along the descent direction by constraining the post-step directional derivative. This pairing is central to line-search globalization arguments because it links function decrease to directional derivative control, rather than relying on a fixed stepsize.

A recurring structural feature in the cited work is that Wolfe conditions are imposed only after establishing that the search direction is a descent direction. In smooth unconstrained minimization this is the usual prerequisite; in the nonsmooth $C^{1,1}$ setting of coderivative-based Newton methods, it is recovered from the inclusion

$$-\nabla\phi(x_k) \in \partial^2\phi(x_k)(d_k) + p_k\,B_k\,d_k,$$

after which the line search still uses the ordinary gradient $\nabla\phi$, which is well defined and Lipschitz (Chao et al., 2024).

2. Weak and strong variants

The literature distinguishes two curvature variants. The strong Wolfe condition requires

$$\bigl|\nabla f(x_k+\alpha_k d_k)^T d_k\bigr| \;\le\;c_2\,\bigl|\nabla f(x_k)^T d_k\bigr|, \qquad c_1<c_2<1,$$

whereas the weak Wolfe condition requires only

$f:\mathbb R^n\to\mathbb R$0

(Sato, 2014).

This distinction is not merely notational. In Euclidean conjugate-gradient methods, the Fletcher–Reeves parameter

$f:\mathbb R^n\to\mathbb R$1

can fail to maintain global descent unless the strong Wolfe bound is enforced. By contrast, Dai–Yuan’s parameter

$f:\mathbb R^n\to\mathbb R$2

admits a natural descent-property proof under only the weak curvature condition (Sato, 2014).

The cited Riemannian analysis makes the same point in manifold form: a Dai–Yuan-type update preserves

$f:\mathbb R^n\to\mathbb R$3

under the weak Wolfe pair, allowing a Zoutendijk-type argument, whereas the Fletcher–Reeves parameter requires the stronger absolute-value bound. A practical implication is that the weak Wolfe pair can be less restrictive without sacrificing the convergence mechanism available to Dai–Yuan-type recurrences.

3. Riemannian generalization

On a Riemannian manifold $f:\mathbb R^n\to\mathbb R$4, line search is formulated through a retraction $f:\mathbb R^n\to\mathbb R$5 satisfying

$f:\mathbb R^n\to\mathbb R$6

The Riemannian gradient is defined by

$f:\mathbb R^n\to\mathbb R$7

and a Riemannian conjugate-gradient step has the form

$f:\mathbb R^n\to\mathbb R$8

(Sato, 2014).

In this setting, the weak Wolfe conditions become

$f:\mathbb R^n\to\mathbb R$9

and

$d_k$0

with $d_k$1 (Sato, 2014).

The differentiated retraction

$d_k$2

plays the role of vector transport. Since this transport may increase norm, the algorithm in (Sato, 2014) introduces a scaled transport $d_k$3 preserving norm,

$d_k$4

and uses the hybrid rule

$d_k$5

so that

$d_k$6

This construction is specific to the manifold setting: the line search must simultaneously account for the geometry of the feasible space and the transport of search directions between tangent spaces. The resulting Wolfe conditions are therefore not a direct substitution of Euclidean derivatives by manifold gradients; they also incorporate the differential of the retraction.

4. Convergence mechanisms

In the Riemannian Dai–Yuan analysis, global convergence is established under two assumptions: $d_k$7 is $d_k$8 and bounded below, and the pullbacks $d_k$9 have Lipschitz-continuous directional derivatives. Under these assumptions, if $\alpha_k>0$0 is generated with the weak Wolfe conditions, then

$\alpha_k>0$1

(Sato, 2014).

The proof structure is explicit. First, each $\alpha_k>0$2 is a descent direction and the denominators in the Dai–Yuan parameter remain nonzero. Second, a Riemannian Zoutendijk theorem yields

$\alpha_k>0$3

where

$\alpha_k>0$4

Third, the scaled-transport bound combines with the Dai–Yuan recurrence to control $\alpha_k>0$5. The contradiction argument then excludes the possibility that $\alpha_k>0$6 stays uniformly bounded away from zero.

A closely related pattern appears in the improved spectral conjugate-gradient method with modified Wolfe line search. Under the assumptions that $\alpha_k>0$7 is bounded below on $\alpha_k>0$8, the level set $\alpha_k>0$9 is bounded, and $$f(x_k+\alpha_k d_k)\;\le\;f(x_k) + c_1\,\alpha_k\,\nabla f(x_k)^T d_k, \qquad 0<c_1<1,$$0 is Lipschitz on a neighborhood of the level set, the analysis proves existence of $$f(x_k+\alpha_k d_k)\;\le\;f(x_k) + c_1\,\alpha_k\,\nabla f(x_k)^T d_k, \qquad 0<c_1<1,$$1 satisfying the modified Wolfe conditions, a Zoutendijk-type summability condition

$$f(x_k+\alpha_k d_k)\;\le\;f(x_k) + c_1\,\alpha_k\,\nabla f(x_k)^T d_k, \qquad 0<c_1<1,$$2

and the central conclusion

$$f(x_k+\alpha_k d_k)\;\le\;f(x_k) + c_1\,\alpha_k\,\nabla f(x_k)^T d_k, \qquad 0<c_1<1,$$3

(Wu et al., 2023).

In coderivative-based Newton methods, the curvature condition is used differently but with a related purpose. The Wolfe inequality

$$f(x_k+\alpha_k d_k)\;\le\;f(x_k) + c_1\,\alpha_k\,\nabla f(x_k)^T d_k, \qquad 0<c_1<1,$$4

implies a uniform lower bound on the accepted step: $$f(x_k+\alpha_k d_k)\;\le\;f(x_k) + c_1\,\alpha_k\,\nabla f(x_k)^T d_k, \qquad 0<c_1<1,$$5 This excludes the pathology $$f(x_k+\alpha_k d_k)\;\le\;f(x_k) + c_1\,\alpha_k\,\nabla f(x_k)^T d_k, \qquad 0<c_1<1,$$6, and under semismooth* or an alternative $$f(x_k+\alpha_k d_k)\;\le\;f(x_k) + c_1\,\alpha_k\,\nabla f(x_k)^T d_k, \qquad 0<c_1<1,$$7 choice, full steps $$f(x_k+\alpha_k d_k)\;\le\;f(x_k) + c_1\,\alpha_k\,\nabla f(x_k)^T d_k, \qquad 0<c_1<1,$$8 are eventually accepted, recovering superlinear Newton behavior (Chao et al., 2024).

5. Modified Wolfe constructions and algorithmic variants

The modified spectral conjugate-gradient method in (Wu et al., 2023) alters both the secant relation and the line-search curvature test. The secant vector is

$$f(x_k+\alpha_k d_k)\;\le\;f(x_k) + c_1\,\alpha_k\,\nabla f(x_k)^T d_k, \qquad 0<c_1<1,$$9

with

$$\nabla f(x_k+\alpha_k d_k)^T d_k \;\ge\;c_2\,\nabla f(x_k)^T d_k, \qquad c_1<c_2<1,$$0

Because $$\nabla f(x_k+\alpha_k d_k)^T d_k \;\ge\;c_2\,\nabla f(x_k)^T d_k, \qquad c_1<c_2<1,$$1 can fail to be positive, the method introduces a scalar $$\nabla f(x_k+\alpha_k d_k)^T d_k \;\ge\;c_2\,\nabla f(x_k)^T d_k, \qquad c_1<c_2<1,$$2 and replaces the curvature test by

$$\nabla f(x_k+\alpha_k d_k)^T d_k \;\ge\;c_2\,\nabla f(x_k)^T d_k, \qquad c_1<c_2<1,$$3

This retains negative values of $$\nabla f(x_k+\alpha_k d_k)^T d_k \;\ge\;c_2\,\nabla f(x_k)^T d_k, \qquad c_1<c_2<1,$$4 while still guaranteeing a curvature-type condition.

The same paper integrates the modified Wolfe search into a spectral conjugate-gradient recurrence

$$\nabla f(x_k+\alpha_k d_k)^T d_k \;\ge\;c_2\,\nabla f(x_k)^T d_k, \qquad c_1<c_2<1,$$5

with a truncated Hager–Zhang-type $$\nabla f(x_k+\alpha_k d_k)^T d_k \;\ge\;c_2\,\nabla f(x_k)^T d_k, \qquad c_1<c_2<1,$$6 and a truncated spectral parameter $$\nabla f(x_k+\alpha_k d_k)^T d_k \;\ge\;c_2\,\nabla f(x_k)^T d_k, \qquad c_1<c_2<1,$$7. The resulting algorithm, SCGMMWLS, satisfies the uniform descent property

$$\nabla f(x_k+\alpha_k d_k)^T d_k \;\ge\;c_2\,\nabla f(x_k)^T d_k, \qquad c_1<c_2<1,$$8

(Wu et al., 2023).

The nonsmooth framework of (Chao et al., 2024) preserves the formal Wolfe inequalities themselves. In GRNM-W, one computes $$\nabla f(x_k+\alpha_k d_k)^T d_k \;\ge\;c_2\,\nabla f(x_k)^T d_k, \qquad c_1<c_2<1,$$9 from a generalized Hessian inclusion, initializes $C^{1,1}$0, and repeatedly reduces $C^{1,1}$1 until both the Armijo and curvature conditions are satisfied. The same Wolfe checks are used in CNFB and CNAL, which combine the generalized regularized Newton step with the forward-backward envelope and the augmented Lagrangian method, respectively.

These constructions suggest that Wolfe conditions are best understood as a globalization template rather than a single immutable formula. The sufficient-decrease component is stable across settings, while the curvature component is the locus of adaptation when the search model departs from classical smooth Euclidean descent.

6. Practical behavior, parameter choices, and terminological distinctions

The numerical evidence in (Sato, 2014) comes from Rayleigh quotient minimization on $C^{1,1}$2. With default line-search parameters $C^{1,1}$3 and $C^{1,1}$4, it is often easier to find a step satisfying the weak Wolfe pair $C^{1,1}$5 than the strong Wolfe conditions. In those experiments, “DY+weak Wolfe” typically outperforms both “DY+strong Wolfe” and “FR+strong Wolfe” in CPU time. The same study reports that $C^{1,1}$6 has minor effect, whereas a larger $C^{1,1}$7, such as $C^{1,1}$8, can further reduce total time for many problems; the recommended defaults are $C^{1,1}$9 for robustness or $$-\nabla\phi(x_k) \in \partial^2\phi(x_k)(d_k) + p_k\,B_k\,d_k,$$0 when aggressive steps are desired.

In (Wu et al., 2023), performance profiles on Andrei’s unconstrained-optimization collection show that among $$-\nabla\phi(x_k) \in \partial^2\phi(x_k)(d_k) + p_k\,B_k\,d_k,$$1, the choice $$-\nabla\phi(x_k) \in \partial^2\phi(x_k)(d_k) + p_k\,B_k\,d_k,$$2 wins over 85% of problems, and that M1 with $$-\nabla\phi(x_k) \in \partial^2\phi(x_k)(d_k) + p_k\,B_k\,d_k,$$3 dominates the standard-Wolfe comparison M2, winning about 80% in NI, about 70% in NF, and about 63% in NG. This suggests that line-search design and secant design are coupled: a modified Wolfe test can be materially beneficial when the secant model permits informative negative curvature surrogates.

The adaptive SOR method of (Miyatake et al., 2018) uses Wolfe conditions in a different role. For SPD linear systems, the quadratic objective

$$-\nabla\phi(x_k) \in \partial^2\phi(x_k)(d_k) + p_k\,B_k\,d_k,$$4

allows the SOR relaxation parameter $$-\nabla\phi(x_k) \in \partial^2\phi(x_k)(d_k) + p_k\,B_k\,d_k,$$5 to be reinterpreted as a step size $$-\nabla\phi(x_k) \in \partial^2\phi(x_k)(d_k) + p_k\,B_k\,d_k,$$6 through

$$-\nabla\phi(x_k) \in \partial^2\phi(x_k)(d_k) + p_k\,B_k\,d_k,$$7

Because $$-\nabla\phi(x_k) \in \partial^2\phi(x_k)(d_k) + p_k\,B_k\,d_k,$$8 is already computed, the adaptive Wolfe-based update introduces no extra matrix-vector products. On the Poisson problem on $$-\nabla\phi(x_k) \in \partial^2\phi(x_k)(d_k) + p_k\,B_k\,d_k,$$9, the Wolfe-based adaptive scheme remains robust, keeps $\nabla\phi$0 in a safe interior band of $\nabla\phi$1, and achieves convergence rates comparable to a locally optimal steepest-descent adaptive scheme but without extra $\nabla\phi$2 products (Miyatake et al., 2018).

A frequent terminological confusion is with Wolf’s condition in Navier–Stokes regularity theory. That condition is a scale-invariant smallness criterion for suitable weak solutions in half-cylinders near a boundary and is unrelated to line-search step acceptance in optimization (Seregin, 2015). The near-homonymy is purely lexical; the two concepts belong to different analytical traditions.