Convergence analysis of Levenberg-Marquardt method with Singular Scaling for nonzero residue nonlinear least-squares problems

Abstract: Recently, a Levenberg-Marquardt method with Singular Scaling matrix, called LMMSS, was proposed and successfully applied in parameter estimation in heat conduction problems, where the choice of suitable singular scaling matrix resulted in better quality approximate solutions than those of the classical Levenberg-Marquardt. In this paper, we study convergence properties of LMMSS when applied to nonzero residual nonlinear least-squares problems. We show that the local convergence of the iterates depends both on the control of the gradient linearization error and on a suitable choice of the regularization parameter. Incidentally, we show that the rate of convergence is dictated by a measure of nonlinearity and residual size, so that if such a measure goes to zero quickly enough, the convergence can be superlinear, otherwise, in general, we show that not even linear convergence can be expected if such a measure is not small enough. Additionally, we propose a globalized version of the method and prove that any limit point of the generated sequence is stationary for the least-squares function. Some examples are provided to illustrate our theoretical results.