---
title: Wolfe Conditions in Optimization
url: https://www.emergentmind.com/topics/wolfe-conditions
type: topic
---

# Wolfe Conditions in Optimization

Searching arXiv for the supplied paper and closely related Wolfe-condition papers to ground the article in cited sources.
Wolfe conditions are line-search acceptance criteria for selecting a step length along a descent direction in iterative optimization. In the standard Euclidean setting, they combine an Armijo sufficient-decrease inequality with a curvature inequality involving directional derivatives, with constants \(0<c_1<c_2<1\). The literature represented here distinguishes the weak Wolfe and strong Wolfe variants, analyzes their role in conjugate-gradient global convergence, extends them to optimization on Riemannian manifolds via retractions and vector transports, and adapts them to modified spectral conjugate-gradient, adaptive SOR, and coderivative-based Newton methods [1405.4371].

## 1. Euclidean formulation

For minimizing \(f:\mathbb R^n\to\mathbb R\) along a descent direction \(d_k\), the standard line search seeks \(\alpha_k>0\) satisfying the Armijo condition
\[
f(x_k+\alpha_k d_k)\;\le\;f(x_k) + c_1\,\alpha_k\,\nabla f(x_k)^T d_k,
\qquad 0<c_1<1,
\]
together with a curvature condition [1405.4371]. In the standard Wolfe form, the curvature inequality is
\[
\nabla f(x_k+\alpha_k d_k)^T d_k
\;\ge\;c_2\,\nabla f(x_k)^T d_k,
\qquad c_1<c_2<1,
\]
which is also the formulation used in a modified spectral conjugate-gradient context [2302.09816].

The first inequality enforces sufficient decrease in the objective along the trial step. The second controls how far the step progresses along the descent direction by constraining the post-step directional derivative. This pairing is central to line-search globalization arguments because it links function decrease to directional derivative control, rather than relying on a fixed stepsize.

A recurring structural feature in the cited work is that Wolfe conditions are imposed only after establishing that the search direction is a descent direction. In smooth unconstrained minimization this is the usual prerequisite; in the nonsmooth \(C^{1,1}\) setting of coderivative-based Newton methods, it is recovered from the inclusion
\[
-\nabla\phi(x_k) \in \partial^2\phi(x_k)(d_k) + p_k\,B_k\,d_k,
\]
after which the line search still uses the ordinary gradient \(\nabla\phi\), which is well defined and Lipschitz [2407.02146].

## 2. Weak and strong variants

The literature distinguishes two curvature variants. The strong Wolfe condition requires
\[
\bigl|\nabla f(x_k+\alpha_k d_k)^T d_k\bigr|
\;\le\;c_2\,\bigl|\nabla f(x_k)^T d_k\bigr|,
\qquad c_1<c_2<1,
\]
whereas the weak Wolfe condition requires only
\[
\nabla f(x_k+\alpha_k d_k)^T d_k
\;\ge\;c_2\,\nabla f(x_k)^T d_k,
\qquad c_1<c_2<1
\]
[1405.4371].

This distinction is not merely notational. In Euclidean conjugate-gradient methods, the Fletcher–Reeves parameter
\[
\beta_k^{\rm FR}
= \frac{\|\nabla f(x_k)\|^2}{\|\nabla f(x_{k-1})\|^2}
\]
can fail to maintain global descent unless the strong Wolfe bound is enforced. By contrast, Dai–Yuan’s parameter
\[
\beta_k^{\rm DY}
= \frac{\nabla f(x_k)^T\nabla f(x_k)}{(\eta_{k-1})^T(\nabla f(x_k)-\nabla f(x_{k-1}))}
\]
admits a natural descent-property proof under only the weak curvature condition [1405.4371].

The cited Riemannian analysis makes the same point in manifold form: a Dai–Yuan-type update preserves
\[
g_{x_k}\bigl(\mathrm{grad}\,f(x_k),\eta_k\bigr)<0
\]
under the weak Wolfe pair, allowing a Zoutendijk-type argument, whereas the Fletcher–Reeves parameter requires the stronger absolute-value bound. A practical implication is that the weak Wolfe pair can be less restrictive without sacrificing the convergence mechanism available to Dai–Yuan-type recurrences.

## 3. Riemannian generalization

On a Riemannian manifold \((M,g)\), line search is formulated through a retraction \(R:TM\to M\) satisfying
\[
R_x(0_x)=x,\qquad DR_x(0_x)=\mathrm{id}_{T_xM}.
\]
The Riemannian gradient is defined by
\[
g_x\bigl(\mathrm{grad}\,f(x),\xi\bigr)=Df(x)[\xi]
\quad\forall\,\xi\in T_xM,
\]
and a Riemannian conjugate-gradient step has the form
\[
x_{k+1}=R_{x_k}\bigl(\alpha_k\,\eta_k\bigr)
\]
[1405.4371].

In this setting, the weak Wolfe conditions become
\[
f\bigl(R_{x_k}(\alpha_k\eta_k)\bigr)
\;\le\;f(x_k) + c_1\,\alpha_k\,g_{x_k}\bigl(\mathrm{grad}\,f(x_k),\eta_k\bigr),
\tag{W1}
\]
and
\[
g_{x_{k+1}}\!\Bigl(\mathrm{grad}\,f(x_{k+1}),
DR_{x_k}(\alpha_k\eta_k)[\eta_k]\Bigr)
\;\ge\;c_2\,g_{x_k}\bigl(\mathrm{grad}\,f(x_k),\eta_k\bigr),
\tag{W2}
\]
with \(0<c_1<c_2<1\) [1405.4371].

The differentiated retraction
\[
\mathcal T^R_{\eta}(\xi)=DR_x(\eta)[\xi]
\]
plays the role of vector transport. Since this transport may increase norm, the algorithm in [1405.4371] introduces a scaled transport \(\mathcal T^0_{\eta}(\xi)\) preserving norm,
\[
\|\mathcal T^0_{\eta}(\xi)\|_{R_x(\eta)}=\|\xi\|_x,
\]
and uses the hybrid rule
\[
\mathcal T^{(k)}_{\alpha_k\eta_k}(\eta_k)
=\begin{cases}
\mathcal T^R_{\alpha_k\eta_k}(\eta_k),
&\|\mathcal T^R_{\alpha_k\eta_k}(\eta_k)\|\le\|\eta_k\|,\\
\mathcal T^0_{\alpha_k\eta_k}(\eta_k),&\text{otherwise},
\end{cases}
\]
so that
\[
\|\mathcal T^{(k)}_{\alpha_k\eta_k}(\eta_k)\|\le\|\eta_k\|.
\]

This construction is specific to the manifold setting: the line search must simultaneously account for the geometry of the feasible space and the transport of search directions between tangent spaces. The resulting Wolfe conditions are therefore not a direct substitution of Euclidean derivatives by manifold gradients; they also incorporate the differential of the retraction.

## 4. Convergence mechanisms

In the Riemannian Dai–Yuan analysis, global convergence is established under two assumptions: \(f\) is \(C^1\) and bounded below, and the pullbacks \(t\mapsto f(R_x(t\eta))\) have Lipschitz-continuous directional derivatives. Under these assumptions, if \(\{x_k\}\) is generated with the weak Wolfe conditions, then
\[
\liminf_{k\to\infty}\|\mathrm{grad}\,f(x_k)\|_{x_k}=0
\]
[1405.4371].

The proof structure is explicit. First, each \(\eta_k\) is a descent direction and the denominators in the Dai–Yuan parameter remain nonzero. Second, a Riemannian Zoutendijk theorem yields
\[
\sum\cos^2\theta_k\,\|\mathrm{grad}f(x_k)\|^2<\infty,
\]
where
\[
\cos\theta_k=
-\frac{g_{x_k}(\mathrm{grad}f(x_k),\eta_k)}
{\|\mathrm{grad}f(x_k)\|\,\|\eta_k\|}.
\]
Third, the scaled-transport bound combines with the Dai–Yuan recurrence to control \(\|\eta_k\|^2/\langle g_k,\eta_k\rangle^2\). The contradiction argument then excludes the possibility that \(\|\mathrm{grad}f(x_k)\|\) stays uniformly bounded away from zero.

A closely related pattern appears in the improved spectral conjugate-gradient method with modified Wolfe line search. Under the assumptions that \(f\) is bounded below on \(\mathbb R^n\), the level set \(\{x\mid f(x)\le f(x_0)\}\) is bounded, and \(\nabla f\) is Lipschitz on a neighborhood of the level set, the analysis proves existence of \(\alpha_k\) satisfying the modified Wolfe conditions, a Zoutendijk-type summability condition
\[
\sum_k \frac{(g_k^T d_k)^2}{\|d_k\|^2}<\infty,
\]
and the central conclusion
\[
\liminf_{k\to\infty}\|g_k\|=0
\]
[2302.09816].

In coderivative-based Newton methods, the curvature condition is used differently but with a related purpose. The Wolfe inequality
\[
\bigl\langle\nabla\phi(x_k + T_k d_k),\,d_k\bigr\rangle
\;\ge\;c_2\;\langle\nabla\phi(x_k),\,d_k\rangle
\]
implies a uniform lower bound on the accepted step:
\[
T_k \ge \frac{K(1-c_2)}{\ell}>0.
\]
This excludes the pathology \(T_k\to0\), and under semismooth* or an alternative \(c_1,c_2\) choice, full steps \(T_k=1\) are eventually accepted, recovering superlinear Newton behavior [2407.02146].

## 5. Modified Wolfe constructions and algorithmic variants

The modified spectral conjugate-gradient method in [2302.09816] alters both the secant relation and the line-search curvature test. The secant vector is
\[
z_k^{(m)} = y_k + \frac{m}{m-2}\,\mu_k\,\|s_k\|^2\,s_k,
\qquad
y_k:=g_{k+1}-g_k,
\]
with
\[
\mu_k = 2\,(f_k-f_{k+1}) + (g_k+g_{k+1})^\top s_k.
\]
Because \(s_k^T z_k^{(m)}\) can fail to be positive, the method introduces a scalar \(t_k\) and replaces the curvature test by
\[
(g(x_k+\alpha_k d_k) + \min\{t_k,0\}\,s_k)^\top d_k
\;\ge\;\sigma\,g_k^T d_k.
\]
This retains negative values of \(\mu_k\) while still guaranteeing a curvature-type condition.

The same paper integrates the modified Wolfe search into a spectral conjugate-gradient recurrence
\[
d_{k+1}=-\theta_{k+1}\,g_{k+1}+\beta_{k+1}\,d_k,
\qquad d_0=-g_0,
\]
with a truncated Hager–Zhang-type \(\beta_{k+1}\) and a truncated spectral parameter \(\theta_{k+1}\). The resulting algorithm, SCGMMWLS, satisfies the uniform descent property
\[
g_{k+1}^T d_{k+1}\le -\eta\|g_{k+1}\|^2
\]
[2302.09816].

The nonsmooth framework of [2407.02146] preserves the formal Wolfe inequalities themselves. In GRNM-W, one computes \(d_k\) from a generalized Hessian inclusion, initializes \(T_k\leftarrow1\), and repeatedly reduces \(T_k\) until both the Armijo and curvature conditions are satisfied. The same Wolfe checks are used in CNFB and CNAL, which combine the generalized regularized Newton step with the forward-backward envelope and the augmented Lagrangian method, respectively.

These constructions suggest that Wolfe conditions are best understood as a globalization template rather than a single immutable formula. The sufficient-decrease component is stable across settings, while the curvature component is the locus of adaptation when the search model departs from classical smooth Euclidean descent.

## 6. Practical behavior, parameter choices, and terminological distinctions

The numerical evidence in [1405.4371] comes from Rayleigh quotient minimization on \(S^{n-1}\). With default line-search parameters \(c_1=10^{-4}\) and \(c_2=0.1\), it is often easier to find a step satisfying the weak Wolfe pair \((\mathrm{W1}),(\mathrm{W2})\) than the strong Wolfe conditions. In those experiments, “DY+weak Wolfe” typically outperforms both “DY+strong Wolfe” and “FR+strong Wolfe” in CPU time. The same study reports that \(c_1\) has minor effect, whereas a larger \(c_2\), such as \(c_2=0.4\), can further reduce total time for many problems; the recommended defaults are \((c_1,c_2)=(10^{-4},0.1)\) for robustness or \((0.1,0.4)\) when aggressive steps are desired.

In [2302.09816], performance profiles on Andrei’s unconstrained-optimization collection show that among \(m=3,4,5,\infty\), the choice \(m=3\) wins over 85% of problems, and that M1 with \(m=3\) dominates the standard-Wolfe comparison M2, winning about 80% in NI, about 70% in NF, and about 63% in NG. This suggests that line-search design and secant design are coupled: a modified Wolfe test can be materially beneficial when the secant model permits informative negative curvature surrogates.

The adaptive SOR method of [1806.09922] uses Wolfe conditions in a different role. For SPD linear systems, the quadratic objective
\[
f(x)=\tfrac12\,x^\top A x - x^\top b
\]
allows the SOR relaxation parameter \(\omega\) to be reinterpreted as a step size \(h\) through
\[
h = \frac{2\omega}{2-\omega},
\qquad
\omega = \frac{2h}{2+h}.
\]
Because \(r^{(k)}=b-Ax^{(k)}\) is already computed, the adaptive Wolfe-based update introduces no extra matrix-vector products. On the Poisson problem on \([0,1]^2\), the Wolfe-based adaptive scheme remains robust, keeps \(\omega^{(k)}\) in a safe interior band of \((0,2)\), and achieves convergence rates comparable to a locally optimal steepest-descent adaptive scheme but without extra \(Ar\) products [1806.09922].

A frequent terminological confusion is with Wolf’s condition in Navier–Stokes regularity theory. That condition is a scale-invariant smallness criterion for suitable weak solutions in half-cylinders near a boundary and is unrelated to line-search step acceptance in optimization [1511.03179]. The near-homonymy is purely lexical; the two concepts belong to different analytical traditions.

Source: https://www.emergentmind.com/topics/wolfe-conditions