Accelerated Backward Forward Method for Convex Optimization

Abstract: We analyze the convergence rate of an accelerated backward forward method for solving convex composite optimization problems. The method was developed by Taylor, Hendrickx and Glineur, and is different from the FISTA algorithm in its placement of the proximal operator. When the smooth part of the objective function is convex, we establish a fast convergence rate of $\mathcal{O}\left( \frac{1}{k^2} \right)$ for the function values, and prove the weak convergence of the iterates. When the smooth part is strongly convex, we propose a variant of the method, and establish an accelerated linear convergence rate for the function values.

• The paper establishes theoretical convergence rates for the ABF algorithm, achieving O(1/k^2) for convex problems and accelerated linear convergence for strongly convex cases.
• It leverages energy sequences and fixed-point analysis to demonstrate the weak convergence of iterates to minimizers while ensuring boundedness.
• The study reveals that the ABF method offers improved worst-case performance compared to FISTA, enhancing efficiency in convex composite optimization.

Accelerated Backward Forward Method for Convex Optimization: Analysis and Convergence

Overview

The paper "Accelerated Backward Forward Method for Convex Optimization" (2604.26420) rigorously investigates the convergence properties of an accelerated backward forward algorithm (ABF) for convex composite minimization—i.e., minimizing $F(x) = f(x) + g(x)$ over a Hilbert space $H$, where $f$ is convex and $L$-smooth and $g$ is convex, proper, and lower-semicontinuous. Distinct from the widely-adopted FISTA, the ABF algorithm features a different order of applying the proximal operator, which results in strictly improved worst-case performance as previously established via Performance Estimation Problems (PEP), but lacking convergence rate proofs. This paper fills this gap by establishing theoretical convergence rates for both convex and strongly convex cases and demonstrating weak convergence of iterates.

Algorithmic Structure and Fixed-Point Analysis

The ABF algorithm operates by alternating a gradient step on the smooth component $f$ and an extrapolated proximal step on $g$, using distinct step sizes. This setting allows for iteration-dependent parameters and more flexible extrapolation schemes. The fixed-point analysis confirms that the asymptotic fixed points of the method coincide with minimizers of $F$, ensuring that the algorithm targets the correct solution set. Furthermore, the authors prove that the gradient of $f$ is constant across all minimizers in $\mathcal{S}$, leveraging convexity and $H$0-smoothness.

Function Value Gap Estimations

Through detailed technical analysis, the paper derives key inequalities bounding the decrease in function value per iteration. The results hold for both the convex and strongly convex regimes, elucidating how the algorithm's parameter selection affects convergence. In particular, the authors construct energy sequences to facilitate telescoping arguments, which play a central role in concluding convergence rates.

Convergence Rate for Convex Problems

The principal result in the convex setting is that the ABF algorithm achieves the $H$1 rate for function value suboptimality, matching FISTA's theoretical guarantee. Notably, the PEP-based analysis suggests the ABF algorithm has a slightly, but strictly, better worst-case convergence rate than FISTA in practice. The convergence proof employs a generalized Nesterov sequence for extrapolation parameterization and constructs an energy function whose monotonicity ensures convergence. Further, it is shown that every weak subsequential limit point of the iterates is a minimizer, and that the sequences $H$2, $H$3, and $H$4 are bounded.

Weak Convergence of Iterates

Utilizing energy sequences and auxiliary lemmas, the paper establishes weak convergence of the ABF iterates ({x_k}) to a minimizer of $H$5. The convergence mechanism critically relies on the regularity of extrapolation sequences and the strong convergence of the gradient, which is constant over minimizers by virtue of smoothness and convexity. Additionally, weak convergence is extended to the auxiliary sequences $H$6 and $H$7, each converging to affine transformations of the minimizer.

Accelerated Linear Convergence for Strongly Convex Functions

For the regime where $H$8 is $H$9-strongly convex, the authors propose a variant of ABF with tailored parameter selection ($f$0, $f$1) and prove an accelerated linear convergence rate—specifically, $f$2 for function value suboptimality, which aligns with the optimal rate established for FISTA-SC. The proof employs an advanced energy argument and leverages strong convexity to ensure dominance of contraction terms.

Implications and Future Directions

The results rigorously validate the accelerated worst-case performance of the ABF algorithm, confirming and sharpening the bounds previously observed empirically and via PEP. This reinforces the theoretical foundation for backward forward splitting methods as viable alternatives to forward backward algorithms in convex composite optimization. The extension of accelerated linear convergence to the strongly convex case provides practitioners with new algorithmic options, particularly relevant for machine learning and signal processing applications where composite objectives appear frequently.

Practical implications include potentially improved convergence in large-scale or ill-conditioned problems leveraging the flexibility in the step size and proximal placement. Theoretically, this paves the way for further investigation of backward forward schemes under additional structure, such as non-Euclidean geometries or stochastic settings. Future research might focus on adaptive parameterization, composite functions with additional smooth terms, or integrating momentum for variance reduction in stochastic optimization.

Conclusion

This paper delivers a comprehensive convergence analysis for the accelerated backward forward algorithm in both convex and strongly convex settings. It establishes $f$3 and accelerated linear convergence rates, respectively, and demonstrates weak convergence of iterates. These findings confirm the algorithm’s competitiveness with state-of-the-art methods and elucidate its theoretical guarantees, expanding the toolkit available for convex composite optimization.