Randomly Permuted Orthogonal Products and Fast Dimension Reduction

Abstract: We study the effect of random signed permutations on products of orthogonal matrices and their applications to fast dimension reduction. Let $A,B \in \mathbb{R}^{d\times d}$ be orthogonal matrices and let $Σ\in \mathbb{R}^{d\times d}$ be a uniformly random signed permutation matrix. We analyze the random orthogonal matrix [ U=A ΣB, ] and show that, under mild assumptions on the size of the entries of $A$ and $B$, [ \max_{i,j=1,\ldots,d} |U_{ij}| =O\left ( \sqrt{\frac{\log d}{d}}\right ) ] with high probability. As an application, we show that ORA, an analogue of the Kac walk in which every update is a $π/4$ rotation, reaches the same maximal entry scale after $O(d\log d)$ updates. This resolves a question of Jain et al. and improves the running time of their construction. We also show that parallel ORA reaches this scale after $O(\log d)$ rounds. We then study the random embedding [ Φ = \sqrt{\frac{d}{m}}\, P_I U D_{ξ'}, ] where $P_I$ restricts to $m$ coordinates and $ξ&#39;$ is an independent Rademacher vector. We identify two parameters controlling norm preservation and show that, throughout the corresponding admissible range, $Φ$ achieves optimal embedding dimension $m\asymp\varepsilon<sup>{-2}\log(N)$. Finally, we extend the result to structured infinite models, including sparse vectors, low-rank matrices, and finite unions of subspaces.