Geometric sharp large deviations for random projections of $\ell_p^n$ spheres and balls (2001.04053v4)

Abstract: Accurate estimation of tail probabilities of projections of high-dimensional probability measures is of relevance in high-dimensional statistics and asymptotic geometric analysis. Whereas large deviation principles identify the asymptotic exponential decay rate of probabilities, sharp large deviation estimates also provide the "prefactor" in front of the exponentially decaying term. For fixed $p \in (1,\infty)$, consider independent sequences $(X^{(n,p)})_{n \in \mathbb{N}}$ and $(\Theta^n)_{n \in \mathbb{N}}$ of random vectors with $\Theta^n$ distributed according to the normalized cone measure on the unit $\ell_2^n$ sphere, and $X^{(n,p)}$ distributed according to the normalized cone measure on the unit $\ell_p^n$ sphere. For almost every realization $(\theta^n)_{n\in\mathbb{N}}$ of $(\Theta^n)_{n\in\mathbb{N}}$, (quenched) sharp large deviation estimates are established for suitably normalized (scalar) projections of $X^{(n,p)}$ onto $\theta^n$, that are asymptotically exact (as the dimension $n$ tends to infinity). Furthermore, the case when $(X^{(n,p)})_{n \in \mathbb{N}}$ is replaced with $(\mathscr{X}^{(n,p)})_{n \in \mathbb{N}}$, where $\mathscr{X}^{(n,p)}$ is distributed according to the uniform (or normalized volume) measure on the unit $\ell_p^n$ ball, is also considered. In both cases, in contrast to the (quenched) large deviation rate function, the prefactor exhibits a dependence on the projection directions $(\theta^n)_{n \in\mathbb{N}}$ that encodes additional geometric information that enables one to distinguish between projections of balls and spheres. Moreover, comparison with numerical estimates obtained by direct computation and importance sampling shows that the obtained analytical expressions for tail probabilities provide good approximations even for moderate values of $n$.