On the Complexity of BFGS Method for Smooth Convex Optimization

Abstract: We study the BFGS method with an Armijo-Wolfe line search for minimizing convex functions with Lipschitz-continuous gradients, without assuming strong convexity. We establish a global iteration complexity bound of $\mathcal{O}(k<sup>{-1/2})$ for the smallest gradient norm among the first $k$ iterates. Moreover, when the initial sublevel set is bounded, we show that the function value gap converges at a rate of $\mathcal{O}(k<sup>{-1})$. Our analysis leverages the classical trace-log-determinant potential function and reveals that a key inequality underlying this potential function remains valid without strong convexity.

• The paper establishes the first explicit global complexity bounds for classical full-memory BFGS on smooth convex functions without strong convexity, proving an unconditional best-iterate gradient rate of O(k^-1/2) and an O(k^-1) function-gap rate when the initial sublevel set is bounded.
• The analysis uses a trace-log-determinant potential and shows that smoothness alone preserves a product lower bound on line-search quality ratios, enabling descent estimates through an AM-GM argument.
• The results match gradient descent’s O(1/k) function-gap rate under bounded sublevel sets but leave open whether distance-based bounds, convergence under minimal assumptions, sharper lower bounds, or analogous results for limited-memory BFGS are possible.

Overview

This paper establishes the first explicit global iteration complexity bounds for the classical BFGS method with an Armijo-Wolfe line search on smooth convex functions that are not strongly convex. The setting is deliberately minimal: $f:\mathbb{R}^d\to\mathbb{R}$ is convex, differentiable, has $L$-Lipschitz gradient, and is bounded below, with no strong convexity or other curvature assumptions. Two guarantees are obtained: a best-iterate gradient bound of $\mathcal{O}(k^{-1/2})$ holding unconditionally, and an $\mathcal{O}(k^{-1})$ function-value gap rate when the initial sublevel set is bounded (2608.16009).

The technical contribution is notable for its economy. Classical global convergence results for BFGS on convex problems are qualitative only [byrd1987global, powell1976algorithm], and the recent wave of non-asymptotic analyses—covering superlinear local rates and global rates under strong convexity or self-concordance [jin2023non, jin2024non, jin2025non, rodomanov2022rates]—does not cover the plain convex case. This paper shows that the existing machinery already suffices once one key observation is made.

Algorithmic setup

The algorithm is standard full-memory BFGS. At each iteration, a descent direction $p_k = -B_k^{-1}g_k$ is computed from the positive-definite Hessian approximation $B_k$, and a step size $\alpha_k$ satisfies both the Armijo (sufficient decrease) condition and the Wolfe (curvature) condition:

$$f(x_{k+1}) \le f(x_k) + c_1 g_k^\top s_k, \qquad g_{k+1}^\top s_k \ge c_2\, g_k^\top s_k,$$

with $0 < c_1 < c_2 < 1$, followed by the usual inverse-Hessian-free update of $B_{k+1}$ using the pair $L$0.

Two elementary estimates drive everything. First, the Wolfe condition implies the curvature lower bound $L$1, which preserves positive definiteness of $L$2. Second, $L$3-smoothness gives $L$4. Combining these with the Armijo condition yields the per-step descent inequality

$L$5

where $L$6 is the line-search quality ratio (appearing as $L$7 in prior work [jin2025non]).

The key observation

The central lemma concerns the classical trace-log-determinant potential

$L$8

A short computation shows that each BFGS step changes this potential by at most $L$9, where $\mathcal{O}(k^{-1/2})$0. Since $\mathcal{O}(k^{-1/2})$1 by smoothness alone—not by strong convexity—telescoping gives

$\mathcal{O}(k^{-1/2})$2

The paper's stated contribution is precisely the recognition that this product lower bound on the line-search ratios, classically derived in the strongly convex analysis of Byrd and Nocedal [byrd1989tool], requires no strong convexity whatsoever. This is what allows the potential-function argument to transfer to the plain convex setting.

Complexity bounds

Combining the descent inequality with the geometric-mean bound on $\mathcal{O}(k^{-1/2})$3 via AM-GM yields the unconditional best-iterate guarantee

$\mathcal{O}(k^{-1/2})$4

where $\mathcal{O}(k^{-1/2})$5. Note the dependence on the initial Hessian approximation through $\mathcal{O}(k^{-1/2})$6: taking $\mathcal{O}(k^{-1/2})$7 makes $\mathcal{O}(k^{-1/2})$8 and removes this factor entirely.

For function values, define $\mathcal{O}(k^{-1/2})$9. A telescoping argument on $\mathcal{O}(k^{-1})$0, again via AM-GM over the products of $\mathcal{O}(k^{-1})$1 and $\mathcal{O}(k^{-1})$2, gives

$\mathcal{O}(k^{-1})$3

hence $\mathcal{O}(k^{-1})$4. When $\mathcal{O}(k^{-1})$5, this combines with the smoothness bound $\mathcal{O}(k^{-1})$6 and convexity ($\mathcal{O}(k^{-1})$7, the radius of the initial sublevel set around $\mathcal{O}(k^{-1})$8) into the clean chain

$\mathcal{O}(k^{-1})$9

The implication is that BFGS with a Wolfe line search achieves the same $p_k = -B_k^{-1}g_k$0 function-gap rate as gradient descent on smooth convex functions, without any strong convexity assumption—a rate previously unavailable for this method in this generality.

Limitations and open questions

The paper is candid about its scope. First, the function-gap rate depends on $p_k = -B_k^{-1}g_k$1 (or $p_k = -B_k^{-1}g_k$2), i.e., boundedness of the entire initial sublevel set, rather than merely on the distance $p_k = -B_k^{-1}g_k$3; whether a rate depending only on the latter is achievable is left open. Second, under only convexity, $p_k = -B_k^{-1}g_k$4-smoothness, and nonemptiness of $p_k = -B_k^{-1}g_k$5, it remains unknown whether every BFGS Armijo-Wolfe sequence converges in function value at all—the authors explicitly ask whether a single smooth convex counterexample exists. Third, the bounds are worst-case complexity statements; no lower bounds are provided, so the tightness of the $p_k = -B_k^{-1}g_k$6 and $p_k = -B_k^{-1}g_k$7 rates is unresolved. Finally, the analysis covers full-memory BFGS only; limited-memory variants fall outside its scope.

It should also be noted that the authors acknowledge the main ingredients were present in the literature, with the proof initially produced by Gemini 3.1 Pro and simplified by GPT 5.6 Sol; the contribution lies in the observation and assembly rather than new machinery.

Conclusion

By showing that the Byrd–Nocedal trace-log-determinant potential inequality survives without strong convexity, this paper delivers the first explicit global complexity rates for standard BFGS with an Armijo-Wolfe line search on general smooth convex objectives: $p_k = -B_k^{-1}g_k$8 best-iterate gradient norm unconditionally, and $p_k = -B_k^{-1}g_k$9 function gap under bounded sublevel sets. The result closes a gap between qualitative classical convergence theory and modern non-asymptotic quasi-Newton analysis, while leaving open the question of convergence under minimal assumptions and of sharper distance-based constants.