A New Combination of Preconditioned Gradient Descent Methods and Vector Extrapolation Techniques for Nonlinear Least-Squares Problems

Abstract: Vector extrapolation methods are widely used in large-scale simulation studies, and numerous extrapolation-based acceleration techniques have been developed to enhance the convergence of linear and nonlinear fixed-point iterative methods. While classical extrapolation strategies often reduce the number of iterations or the computational cost, they do not necessarily lead to a significant improvement in the accuracy of the computed approximations. In this paper, we study the combination of preconditioned gradient-based methods with extrapolation strategies and propose an extrapolation-accelerated framework that simultaneously improves convergence and approximation accuracy. The focus is on the solution of nonlinear least-squares problems through the integration of vector extrapolation techniques with preconditioned gradient descent methods. A comprehensive set of numerical experiments is carried out to study the behavior of polynomial-type extrapolation methods and the vector $\varepsilon$-algorithm when coupled with gradient descent schemes, with and without preconditioning. The results demonstrate the impact of extrapolation techniques on both convergence rate and solution accuracy, and report iteration counts, computational times, and relative reconstruction errors. The performance of the proposed hybrid approaches is further assessed through a benchmarking study against Gauss--Newton methods based on generalized Krylov subspaces.