Refined Least Squares for Support Recovery

Abstract: We study the problem of exact support recovery based on noisy observations and present Refined Least Squares (RLS). Given a set of noisy measurement $$ \myvec{y} = \myvec{X}\myvec{\theta}^* + \myvec{\omega},$$ and $\myvec{X} \in \mathbb{R}^{N \times D}$ which is a (known) Gaussian matrix and $\myvec{\omega} \in \mathbb{R}^N$ is an (unknown) Gaussian noise vector, our goal is to recover the support of the (unknown) sparse vector $\myvec{\theta}^* \in \left{-1,0,1\right}^D$. To recover the support of the $\myvec{\theta}^*$ we use an average of multiple least squares solutions, each computed based on a subset of the full set of equations. The support is estimated by identifying the most significant coefficients of the average least squares solution. We demonstrate that in a wide variety of settings our method outperforms state-of-the-art support recovery algorithms.