Approximate Solutions of Linear Systems at a Universal Rate (2207.03388v1)

Abstract: Let $A \in \mathbb{R}^{n \times n}$ be invertible, $x \in \mathbb{R}^n$ unknown and $b =Ax $ given. We are interested in approximate solutions: vectors $y \in \mathbb{R}^n$ such that $|Ay - b|$ is small. We prove that for all $0< \varepsilon <1 $ there is a composition of $k$ orthogonal projections onto the $n$ hyperplanes generated by the rows of $A$, where $$k \leq 2 \log\left(\frac{1}{\varepsilon} \right) \frac{ n}{ \varepsilon^{2}}$$ which maps the origin to a vector $y\in \mathbb{R}^n$ satisfying $| A y - Ax| \leq \varepsilon \cdot |A| \cdot | x|$. We note that this upper bound on $k$ is independent of the matrix $A$. This procedure is stable in the sense that $|y| \leq 2|x|$. The existence proof is based on a probabilistically refined analysis of the Random Kaczmarz method which seems to achieve this rate when solving for $A x = b$ with high likelihood.