Solve TRTLS with a singular Tikhonov penalty

Solve the Tikhonov-regularized total least-squares problem (TRTLS) when the regularization matrix L has fewer rows than columns, so that the penalty matrix L^T L is singular, including determining whether an optimum is attained and computing an optimal solution when it exists.

Background

The paper studies the nonconvex reformulation of TRTLS as minimizing a quadratic fractional function plus the quadratic penalty $\rho\|Lx\|^2$. When $L$ is square and nonsingular, the penalty is positive definite and existing methods efficiently solve the resulting problem. When $L$ has fewer rows than columns, the penalty is singular, and the objective may fail to attain its infimum. The authors identify this singular-penalty case as substantially more difficult and formulate a general problem involving positive semidefinite matrices $W$ and singular $B$, which they analyze using simultaneous diagonalization via congruence.

The paper subsequently resolves several transformed cases and completely characterizes the TRTLS problem when $\operatorname{rank}(A)<n$, but it notes that the case $\operatorname{rank}(A)=n$ is more complicated. The explicit open-problem statement concerns the singular-penalty regime at the point where the general problem is introduced.