Fully-Corrective Frank-Wolfe (fc-FW)

• fc-FW is a Frank-Wolfe variant that re-optimizes over the convex hull of all selected atoms rather than using a simple line search.
• It decouples atom generation from active-set correction, leading to enhanced convergence rates under smoothness and strong convexity conditions.
• The method extends to quadratic objectives and infinite-dimensional measure spaces, balancing computational cost with improved optimization accuracy.

Fully-corrective Frank-Wolfe (fc-FW), also written FCFW, is a Frank-Wolfe / conditional-gradient variant in which each iteration first obtains a new atom from the linear minimization oracle and then re-optimizes over the convex hull of the accumulated active atoms, instead of only taking a line-search step toward the newly selected atom (Lacoste-Julien et al., 2015). In one standard formulation, the problem is $\min_{x\in M} f(x)$ with $M=\operatorname{conv}(A)$, and the fully-corrective mechanism is implemented by maintaining a correction set of atoms and solving a restricted optimization problem over its convex hull; in more recent terminology, the exact fully-corrective step is $$x_{t+1}=\arg\min_{x\in \operatorname{conv}(S_t)} f(x)$$, where $S_t$ is the active set (Halbey et al., 3 Jun 2025). The same principle has also been extended beyond finite-dimensional polytopes to optimization over measure spaces, where the atoms become Dirac measures and the corrective step rebalances their weights (Yu et al., 13 Jul 2025).

1. Canonical formulation

The finite-dimensional fc-FW formulation considered in the linear-convergence analysis of Frank-Wolfe variants starts from

$$\min_{x \in M} f(x), \qquad M=\operatorname{conv}(A),$$

where $A$ is a finite set of atoms and the only primitive needed for atom generation is a linear minimization oracle

$$s_t \in \operatorname{argmin}_{x\in A} \langle \nabla f(x^{(t)}), x\rangle.$$

Standard Frank-Wolfe then performs a line search along $s_t-x^{(t)}$,

$$x^{(t+1)} = x^{(t)} + \gamma_t (s_t - x^{(t)}), \qquad \gamma_t \in \arg\min_{\gamma\in[0,1]} f(x^{(t)}+\gamma(s_t-x^{(t)})).$$

The defining modification in FCFW is that, after the new atom $s_t$ is identified, the algorithm performs a correction over a correction polytope built from previously selected atoms rather than committing to a single one-dimensional line-search update (Lacoste-Julien et al., 2015).

One algorithmic presentation makes this explicit by maintaining a set of correction atoms $M=\operatorname{conv}(A)$0, computing the Frank-Wolfe gap

$M=\operatorname{conv}(A)$1

stopping when $M=\operatorname{conv}(A)$2, and otherwise invoking a correction subroutine: $M=\operatorname{conv}(A)$3 A more recent abstraction places the exact fully-corrective move in the active-set notation

$M=\operatorname{conv}(A)$4

thereby isolating fc-FW as the case where correction means exact minimization over the current active hull (Halbey et al., 3 Jun 2025).

This formulation makes the distinctive computational tradeoff transparent. The Frank-Wolfe linear oracle remains the mechanism for discovering new atoms, but the main work shifts to restricted re-optimization over the active set. The data explicitly notes that this subproblem can be expensive because it can be as hard as the original problem (Halbey et al., 3 Jun 2025).

2. Correction subproblems and active-set mechanics

In the FCFW formulation analyzed together with away-step, pairwise, and minimum-norm-point variants, the correction routine is intentionally abstract. It is only required to return a new iterate whose active set $M=\operatorname{conv}(A)$5 is contained in a correction set $M=\operatorname{conv}(A)$6, to make at least as much progress as a Frank-Wolfe line search,

$M=\operatorname{conv}(A)$7

and to ensure that the away gap at the new point is small enough,

$M=\operatorname{conv}(A)$8

This is the paper’s notion of approximate correction, and it is broader than exact minimization over the active hull (Lacoste-Julien et al., 2015).

A more general modern framework, termed Corrective Frank-Wolfe (CFW), formalizes this relaxation further. At iteration $M=\operatorname{conv}(A)$9, with active set $$x_{t+1}=\arg\min_{x\in \operatorname{conv}(S_t)} f(x)$$0 and current point $$x_{t+1}=\arg\min_{x\in \operatorname{conv}(S_t)} f(x)$$1, the method computes an away vertex

$$x_{t+1}=\arg\min_{x\in \operatorname{conv}(S_t)} f(x)$$2

a local FW vertex

$$x_{t+1}=\arg\min_{x\in \operatorname{conv}(S_t)} f(x)$$3

and the global FW vertex

$$x_{t+1}=\arg\min_{x\in \operatorname{conv}(S_t)} f(x)$$4

It then compares the local pairwise gap $$x_{t+1}=\arg\min_{x\in \operatorname{conv}(S_t)} f(x)$$5 with the global FW gap $$x_{t+1}=\arg\min_{x\in \operatorname{conv}(S_t)} f(x)$$6. If the local pairwise gap is larger, it triggers a corrective step; otherwise it performs a global FW step (Halbey et al., 3 Jun 2025).

Within this framework, a corrective step need not be fully corrective. It suffices to output $$x_{t+1}=\arg\min_{x\in \operatorname{conv}(S_t)} f(x)$$7 and $$x_{t+1}=\arg\min_{x\in \operatorname{conv}(S_t)} f(x)$$8 such that either it is a drop step with $$x_{t+1}=\arg\min_{x\in \operatorname{conv}(S_t)} f(x)$$9 and $S_t$0, or it is a descent step satisfying

$S_t$1

The fully-corrective step is then one specific admissible correction among others, alongside local pairwise and global pairwise steps (Halbey et al., 3 Jun 2025).

This separation of atom generation from active-set correction is central to contemporary treatments. It suggests that fc-FW is best viewed not merely as a single algorithmic template, but as the exact endpoint of a wider family of active-set correction rules.

3. Convergence rates and geometric constants

The classical linear-convergence analysis covering FCFW proves geometric decay under smoothness and strong convexity. If $S_t$2 has $S_t$3-Lipschitz gradient and is $S_t$4-strongly convex over $S_t$5, and if $S_t$6, then for away-step FW, pairwise FW, FCFW, and Wolfe’s minimum norm point algorithm, one has on every good step

$S_t$7

where $S_t$8 is the pyramidal width and $S_t$9. If $$\min_{x \in M} f(x), \qquad M=\operatorname{conv}(A),$$0 denotes the number of good steps up to iteration $$\min_{x \in M} f(x), \qquad M=\operatorname{conv}(A),$$1, then

$$\min_{x \in M} f(x), \qquad M=\operatorname{conv}(A),$$2

For FCFW, the theorem states $$\min_{x \in M} f(x), \qquad M=\operatorname{conv}(A),$$3, so the geometric decrease applies iteration by iteration; in the general convex case $$\min_{x \in M} f(x), \qquad M=\operatorname{conv}(A),$$4, the result reverts to

$$\min_{x \in M} f(x), \qquad M=\operatorname{conv}(A),$$5

The same paper interprets the inverse rate constant as the product of the standard condition number $$\min_{x \in M} f(x), \qquad M=\operatorname{conv}(A),$$6 of the objective and a geometric condition number $$\min_{x \in M} f(x), \qquad M=\operatorname{conv}(A),$$7 of the feasible polytope (Lacoste-Julien et al., 2015).

The geometric term is not merely formal. The analysis introduces pyramidal width as a domain quantity that plays the role of a condition number of the constraint set. Explicit values are given for several polytopes, including $$\min_{x \in M} f(x), \qquad M=\operatorname{conv}(A),$$8 for the unit cube in $$\min_{x \in M} f(x), \qquad M=\operatorname{conv}(A),$$9, illustrating that the geometry of the atom set enters the global linear rate directly (Lacoste-Julien et al., 2015).

The same line of work also extends beyond strict strong convexity of $A$0 itself. For objectives of the form

$A$1

with $A$2 strongly convex, a generalized geometric strong-convexity constant yields the same linear-convergence structure. This is the source of the paper’s statement that the variants enjoy global linear convergence under a weaker condition than strong convexity of the objective (Lacoste-Julien et al., 2015).

A later corrective-step analysis obtains a different, framework-level set of guarantees. Under smoothness and compactness, CFW satisfies

$A$3

and under $A$4-sharpness it satisfies

$A$5

Its lazified variant LCFW achieves $A$6, and under sharpness $A$7 for some constant $A$8. These results are not fc-FW-specific, but they place exact full correction inside a broader class of active-set methods with certified descent (Halbey et al., 3 Jun 2025).

4. Exact full correction, approximate correction, and quadratic specializations

A persistent issue in fc-FW is that the fully-corrective subproblem can be as hard as the original optimization problem. The corrective-step framework addresses this by decoupling the notion of “correction” from exact minimization. In that framework, AFW, PFW, BPCG, and fc-FW are all interpreted as instances of active-set correction, but only fc-FW insists on exact re-optimization over $A$9 (Halbey et al., 3 Jun 2025).

This perspective becomes especially concrete for convex quadratic objectives

$$s_t \in \operatorname{argmin}_{x\in A} \langle \nabla f(x^{(t)}), x\rangle.$$0

Two specialized corrective algorithms are proposed there. QC-LP solves a linear program encoding affine-hull optimality together with simplex feasibility; if the LP is feasible, the solution is essentially a fully-corrective step, and if it is infeasible the method falls back to a local pairwise step. QC-MNP solves the affine-hull system $$s_t \in \operatorname{argmin}_{x\in A} \langle \nabla f(x^{(t)}), x\rangle.$$1; if the resulting barycentric weights are all nonnegative, the step is accepted as fully corrective, while negative weights trigger a Wolfe-style ratio test and atom removal (Halbey et al., 3 Jun 2025).

The paper presents QC-MNP as a specialized variant of Wolfe’s minimum-norm-point algorithm with one explicit simplification: it solves only one quadratic subproblem per iteration, unlike the classical MNP-correction in Lacoste-Julien & Jaggi, which may solve several linear systems in a single correction phase. The paper emphasizes that this makes QC-MNP more efficient and less aggressive in dropping potentially useful atoms (Halbey et al., 3 Jun 2025).

These constructions are not exact fc-FW in full generality. The paper states explicitly that QC-LP and QC-MNP are approximate fc-FW-style corrections specialized to quadratic objectives and reduced to an LP or linear system over the active set. Their significance is therefore structural rather than definitional: they show how the fully-corrective idea can be approximated by cheaper inner routines while remaining inside a certified corrective framework (Halbey et al., 3 Jun 2025).

5. Infinite-dimensional fc-FW on measure spaces

A direct generalization of fc-FW to an infinite-dimensional setting appears in a measure-space optimization problem for emergency response to out-of-hospital cardiac arrest. The decision variable is a nonnegative finite Borel measure $$s_t \in \operatorname{argmin}_{x\in A} \langle \nabla f(x^{(t)}), x\rangle.$$2 on $$s_t \in \operatorname{argmin}_{x\in A} \langle \nabla f(x^{(t)}), x\rangle.$$3 with fixed total mass $$s_t \in \operatorname{argmin}_{x\in A} \langle \nabla f(x^{(t)}), x\rangle.$$4, and the feasible set is

$$s_t \in \operatorname{argmin}_{x\in A} \langle \nabla f(x^{(t)}), x\rangle.$$5

With $$s_t \in \operatorname{argmin}_{x\in A} \langle \nabla f(x^{(t)}), x\rangle.$$6, the problem is

$$s_t \in \operatorname{argmin}_{x\in A} \langle \nabla f(x^{(t)}), x\rangle.$$7

The paper proves that this feasible set is convex, weak$$s_t \in \operatorname{argmin}_{x\in A} \langle \nabla f(x^{(t)}), x\rangle.$$8-compact when $$s_t \in \operatorname{argmin}_{x\in A} \langle \nabla f(x^{(t)}), x\rangle.$$9 is compact, and therefore admits an optimizer (Yu et al., 13 Jul 2025).

The first-order object is the influence function

$s_t-x^{(t)}$0

Because the von Mises derivative satisfies

$s_t-x^{(t)}$1

the linearized Frank-Wolfe oracle over measures reduces to minimizing the scalar function $s_t-x^{(t)}$2 over $s_t-x^{(t)}$3. The corresponding measure-space FW recursion is

$s_t-x^{(t)}$4

The fully-corrective version keeps all previously generated atoms and reoptimizes their weights: $s_t-x^{(t)}$5 If the active atoms are $s_t-x^{(t)}$6, this becomes the finite-dimensional convex program

$s_t-x^{(t)}$7

The paper therefore describes the method as the measure-space analogue of classical fc-FW (Yu et al., 13 Jul 2025).

The convergence theory is notable because exact global solution of the oracle is not required. A sufficient condition is only

$s_t-x^{(t)}$8

Under $s_t-x^{(t)}$9-smoothness with $$x^{(t+1)} = x^{(t)} + \gamma_t (s_t - x^{(t)}), \qquad \gamma_t \in \arg\min_{\gamma\in[0,1]} f(x^{(t)}+\gamma(s_t-x^{(t)})).$$0, this yields the sufficient decrease bound

$$x^{(t+1)} = x^{(t)} + \gamma_t (s_t - x^{(t)}), \qquad \gamma_t \in \arg\min_{\gamma\in[0,1]} f(x^{(t)}+\gamma(s_t-x^{(t)})).$$1

from which the paper derives monotone decrease of $$x^{(t+1)} = x^{(t)} + \gamma_t (s_t - x^{(t)}), \qquad \gamma_t \in \arg\min_{\gamma\in[0,1]} f(x^{(t)}+\gamma(s_t-x^{(t)})).$$2, convergence of $$x^{(t+1)} = x^{(t)} + \gamma_t (s_t - x^{(t)}), \qquad \gamma_t \in \arg\min_{\gamma\in[0,1]} f(x^{(t)}+\gamma(s_t-x^{(t)})).$$3 when $$x^{(t+1)} = x^{(t)} + \gamma_t (s_t - x^{(t)}), \qquad \gamma_t \in \arg\min_{\gamma\in[0,1]} f(x^{(t)}+\gamma(s_t-x^{(t)})).$$4, the limit $$x^{(t+1)} = x^{(t)} + \gamma_t (s_t - x^{(t)}), \qquad \gamma_t \in \arg\min_{\gamma\in[0,1]} f(x^{(t)}+\gamma(s_t-x^{(t)})).$$5, summability of $$x^{(t+1)} = x^{(t)} + \gamma_t (s_t - x^{(t)}), \qquad \gamma_t \in \arg\min_{\gamma\in[0,1]} f(x^{(t)}+\gamma(s_t-x^{(t)})).$$6, and the stationarity rate

$$x^{(t+1)} = x^{(t)} + \gamma_t (s_t - x^{(t)}), \qquad \gamma_t \in \arg\min_{\gamma\in[0,1]} f(x^{(t)}+\gamma(s_t-x^{(t)})).$$7

If the oracle is solved globally, the paper states that fc-FW inherits the standard Frank-Wolfe objective rate

$$x^{(t+1)} = x^{(t)} + \gamma_t (s_t - x^{(t)}), \qquad \gamma_t \in \arg\min_{\gamma\in[0,1]} f(x^{(t)}+\gamma(s_t-x^{(t)})).$$8

For discrete incident distributions under $$x^{(t+1)} = x^{(t)} + \gamma_t (s_t - x^{(t)}), \qquad \gamma_t \in \arg\min_{\gamma\in[0,1]} f(x^{(t)}+\gamma(s_t-x^{(t)})).$$9 travel, the influence function is strictly concave on each rectangle of an induced grid, so minimization reduces to a finite support-reduction search over $s_t$0 grid vertices (Yu et al., 13 Jul 2025).

This measure-space extension shows that full correction is not tied to finite simplices or finite-dimensional coordinate systems. A plausible implication is that the essential object in fc-FW is not the ambient space, but the repeated construction of an atomic support together with exact or approximate reweighting over its convex hull.

6. Relation to neighboring Frank-Wolfe variants and recurrent misidentifications

A recurrent misconception is that any active-set or correction-like Frank-Wolfe variant is fully corrective. The data does not support that equivalence. In the corrective-step framework, fc-FW is only one specific correction rule among local pairwise, global pairwise, away-step, blended pairwise, and lazified variants; what distinguishes fc-FW is exact minimization over the current active hull, not merely the use of active-set information (Halbey et al., 3 Jun 2025).

A second recurrent confusion concerns Wolfe’s minimum-norm-point algorithm. The linear-convergence analysis explicitly states that MNP is often confused with FCFW, but they are not the same. FCFW minimizes $s_t$1 over the correction polytope $s_t$2, whereas MNP performs correction as a sequence of affine projections on the active set. After correction, MNP guarantees minimization over both the affine hull and the convex hull of the final correction set, while FCFW guarantees minimization over the whole correction polytope used in the correction (Lacoste-Julien et al., 2015).

Several recent Frank-Wolfe papers are related only tangentially to fc-FW. “Nonsmooth Frank-Wolfe using Uniform Affine Approximations” proposes FWUA, which replaces local linearization by a uniform affine approximation over an $s_t$3-neighborhood, but it still uses one atom $s_t$4 per iteration and does not perform optimization over the whole active set; the paper states explicitly that it is not about fully-corrective Frank-Wolfe in the standard sense (Cheung et al., 2017). The self-concordant and generalized self-concordant analyses develop adaptive step sizes, backtracking, local linear minimization oracles, and away-step mechanisms, but they do not analyze the defining fc-FW update of re-optimizing over the entire active set (Dvurechensky et al., 2020, Dvurechensky et al., 2020). The discretization-focused work on multistep Frank-Wolfe and LMO averaging similarly does not solve a convex-hull re-optimization subproblem; its AvgFW method maintains a recursive average of past LMO outputs rather than computing the best point in their convex hull (Chen et al., 2023).

The simplex-based SWAP algorithm is closer to pairwise correction than to full correction. It updates by

$s_t$5

changing only two coordinates, and the paper contrasts this with fully-corrective procedures that solve the reduced optimization problem on the entire active face

$s_t$6

The paper presents SWAP precisely as a cheaper local alternative to that full restricted reoptimization (Allende et al., 2013).

Finally, foundational analyses of standard Frank-Wolfe remain relevant to fc-FW even when they are not about full correction. The paper “New Analysis and Results for the Frank-Wolfe Method” develops the curvature-based gap framework, arbitrary step-size guarantees, warm-start step sizes, and robustness to approximate linear minimization, but it does not analyze fully-corrective Frank-Wolfe. This suggests that such results serve primarily as baseline tools—especially through the Frank-Wolfe gap and approximate-oracle analysis—rather than as fc-FW-specific theory (Freund et al., 2013).