Variable Projected Augmented Lagrangian Methods for Generalized Lasso Problems

Abstract: We introduce variable projected augmented Lagrangian (VPAL) methods for solving generalized nonlinear Lasso problems with improved speed and accuracy. By eliminating the nonsmooth variable via soft-thresholding, VPAL transforms the problem into a smooth reduced formulation. For linear models, we develop a preconditioned variant that mimics Newton-type updates and yields significant acceleration. We prove convergence guarantees for both standard and preconditioned VPAL under mild assumptions and show that variable projection leads to sharper convergence and higher solution quality. The method seamlessly extends to nonlinear inverse problems, where it outperforms traditional approaches in applications such as phase retrieval and contrast enhanced MRI (LIP-CAR). Across tasks including deblurring, inpainting, and sparse-view tomography, VPAL consistently delivers state-of-the-art reconstructions, positioning variable projection as a powerful tool for modern large-scale inverse problems.