Surrounding the solution of a Linear System of Equations from all sides

Abstract: Suppose $A \in \mathbb{R}^{n \times n}$ is invertible and we are looking for the solution of $Ax = b$. Given an initial guess $x_1 \in \mathbb{R}$, we show that by reflecting through hyperplanes generated by the rows of $A$, we can generate an infinite sequence $(x_k){k=1}^{\infty}$ such that all elements have the same distance to the solution, i.e. $|x_k - x| = |x_1 - x|$. If the hyperplanes are chosen at random, averages over the sequence converge and $$ \mathbb{E} \left| x - \frac{1}{m} \sum{k=1}^{m}{ x_k} \right| \leq \frac{1 + |A|_F |A^{-1}|}{\sqrt{m}} \cdot|x-x_1|.$$ The bound does not depend on the dimension of the matrix. This introduces a purely geometric way of attacking the problem: are there fast ways of estimating the location of the center of a sphere from knowing many points on the sphere? Our convergence rate (coinciding with that of the Random Kaczmarz method) comes from averaging, can one do better?