---
title: Multiple Random Projections Overview
url: https://www.emergentmind.com/topics/multiple-random-projections
type: topic
---

# Multiple Random Projections Overview

to=arxiv_search 总代理联系  大发游戏官网Params  大发时时彩开奖ia  天天中彩票有_coupon code
Multiple random projections denote methods that apply several random linear maps to the same data and then combine the resulting low-dimensional views, rather than relying on a single sketch. Across the literature, the combination step appears as estimator averaging, within-block screening, thresholded voting, singular-value aggregation, multiple-comparison correction, repeated testing with mode aggregation, per-bin mixture selection, and multishot physical measurement. The paradigm is used in randomized regression, classification, adversarial robustness, generative modeling, change-point analysis, outlier detection, beamforming, low-memory embeddings, invertible filter-bank constructions, and optical implementations of random feature maps [1701.05325], [1504.04595], [2410.04922], [2606.10377].

## 1. Core formulations and design patterns

At the level of notation, the literature instantiates multiple random projections through several families of maps: orthonormal projections \(A_m:\mathbb R^p\to\mathbb R^d\), Gaussian sketches \(R_j\in\mathbb R^{k_j\times d}\), one-dimensional random directions \(r^{(m)}\in\mathbb R^p\), complex beamforming compressions \(\Psi_p\in\mathbb C^{N_d\times N_m}\), and structured tensorized maps \(A^{(1)}\odot\cdots\odot A^{(N)}\). The projection law varies by task and includes Haar-distributed orthonormal matrices, i.i.d. Gaussian entries, Rademacher entries, sparse “3-point” projections, subsampled randomized Hadamard transforms, and complex Gaussian transmission matrices [1504.04595], [2602.19988], [1701.05325], [2507.05662], [1510.06664].

A common distinction is between **independent repetition** and **structured composition**. Independent repetition draws many unrelated projections and aggregates the downstream outputs, as in random-projection ensemble classification, RP-Ensemble defenses, repeated change-point detection, or multiple compressed beamformers. Structured composition instead builds one projection from several smaller random components, as in Tensor Random Projection (TRP), bidirectional row-and-column sketching for OLS, paraunitary cascades, or optical multishot recovery of a Gaussian linear map [2105.00105], [2606.10377], [2106.10746], [2305.12988].

| Setting | Projection mechanism | Combination rule |
|---|---|---|
| Classification | \(B_1B_2\) Haar or Gaussian \(A_{b_1,b_2}\in\mathbb R^{d\times p}\) | within-block selection and thresholded voting |
| Regression dimension reduction | grouped \(P_{g,m}\in\mathbb R^{p\times d}\) | empirical-error screening and SVD of \(\bar P\) |
| Change-point analysis | \(M\) one-dimensional random vectors \(r^{(m)}\) | adjusted \(p\)-values, then repeated mode |
| Tensor RP | \(A^{(1)}\odot\cdots\odot A^{(N)}\) | optional averaging over \(T\) independent TRPs |
| Beamforming | \(P\) compressed views \(\Psi_p\in\mathbb C^{N_d\times N_m}\) | time-frequency minimum-power mixture |

This diversity matters because “multiple random projections” does not imply a single statistical effect. In some settings repetition reduces variance or Monte Carlo error; in others an additional projection stage degrades statistical accuracy relative to a one-sided baseline. The combination operator is therefore part of the method, not an implementation detail.

## 2. Regression, sketching, and ensemble dimension reduction

In large-scale least-squares regression, Thanei, Heinze, and Meinshausen describe a \(K\)-fold sketch-and-solve procedure in which \(K\) independent column sketches \(\phi_i\in\mathbb R^{p\times d}\) are drawn, each sketched problem is solved in parallel, and the lifted estimators \(\hat\beta_i\) are averaged as
\[
\bar\beta_K=\frac1K\sum_{i=1}^K \hat\beta_i.
\]
The limiting estimator \(\hat\beta_\infty\) has an AMSE bound
\[
E[\|X\beta-X\hat\beta_\infty\|^2]\le \sigma^2\tau+\sum_{i=1}^p \beta_i^2\lambda_i w_i^2,
\]
with \(d^2/p\le \tau\le d\), and the paper states that averaging strictly reduces the MSE relative to a single sketch. In the isotropic case \(\Sigma=I_p\), the variance reduction factor is \(d/p\) compared to a single sketch, and the resulting bias-variance behavior is compared to ridge regression and principal component regression [1701.05325].

A different use of multiple projections appears in fixed-design OLS under bidirectional sketching. In "Bidirectional Random Projections" [2606.10377], the one-sided estimator uses a Gaussian column sketch \(R\in\mathbb R^{p\times p_1}\), while the bidirectional estimator also uses a Gaussian row sketch \(W\in\mathbb R^{n_1\times n}\), forming \((\tilde X,\tilde Y)=(WXR,WY)\). The one-sided excess-risk bound is
\[
E_{R,E}[L_R(\hat\beta_R)]-L(\beta_*)\le \sigma^2\cdot(p_1/n)+(1/p_1)\|\beta_*\|_M^2,
\]
whereas the bidirectional bound is
\[
E_{R,W,E}[L_R(\hat\beta_{RW})]-L(\beta_*)\le C_1\sigma^2(p_1/n)+C_2(1/p_1)\|\beta_*\|_M^2+b.
\]
The gap satisfies
\[
\Delta(p_1)=B_{RW}-B_R=O(p_1)+O(1/p_1),
\]
and the \(O(1/p_1)\) coefficient changes sign at \(r=\sqrt{n_1/n}=1/2\). The paper further states that the bi-sketched estimator is always slightly worse than the one-sided sketch in the reported experiments, and that the two-sided reduction strictly compromises statistical accuracy relative to a one-sided column sketch [2606.10377].

Multiple random projections are also used for supervised subspace estimation rather than direct prediction. In "Random-projection ensemble dimension reduction" [2410.04922], Zhou and Cannings divide the projections into \(G\) groups of size \(M\), evaluate each projected regressor on a hold-out set, keep the best \(P_{g,*}\) in each group, and aggregate via
\[
\bar P=\frac1G\sum_{g=1}^G P_{g,*}P_{g,*}^T.
\]
An SVD of \(\bar P\) yields directions \(u_j\) and singular values \(\sigma_j\), where \(\sigma_j\) are interpreted as importance scores for selecting the final dimension. The theoretical perturbation bound contains an \(O(G^{-1/2})\) term, so the subspace error stabilizes as the number of groups increases. The paper provides default recommendations \(G=200\), \(M=10p\), \(d=\lceil\sqrt p\rceil\), and \(n_1=2n/3\), and also studies a second-pass “double-RPE” strategy when the initial recommended dimension is still too large [2410.04922].

## 3. Classification, adversarial robustness, and generative modeling

Ensemble classification by multiple random projections was formalized by Cannings and Samworth through \(B_1B_2\) independent projections \(\A_{b_1,b_2}\in\mathbb R^{d\times p}\), typically Haar-distributed on the Stiefel manifold. Within each block \(b_1\), the projection minimizing an estimated test error \(\hat L(\A_{b_1,b_2})\) is selected, producing \(B_1\) retained projections. For a test point \(x\), the vote proportion is
\[
\nu_n(x)=\frac1{B_1}\sum_{b_1=1}^{B_1}\mathbf 1\{\hat y_{b_1}(x)=1\},
\]
and the final classifier is \(f(x)=\mathbf 1\{\nu_n(x)\ge \alpha\}\), with \(\alpha\) chosen data-adaptively. Under a sufficient dimension reduction boundary condition \(Y\perp X\mid \A^*X\), the excess risk can be controlled by terms that do not depend on the original dimension \(p\) and a Monte Carlo term that is \(O(B_1^{-1})\) [1504.04595].

Carbone et al. use multiple Gaussian projections for adversarial robustness in two ways. RP-Ensemble trains \(p\) separate classifiers \(\psi_j\) on projected data \(XR_j^T\), with \(R_j\in\mathbb R^{k_j\times d}\) and \((R_j)_{ab}\sim \mathcal N(0,1/k_j)\), and combines them at inference by summing class probabilities with the original full-dimensional classifier. RP-Regularizer instead augments the training loss by one of two penalties,
\[
\mathcal R_{v1}(\theta)\simeq \frac1{np}\sum_{i=1}^n\sum_{j=1}^p \|\nabla_x \ell(f(X_j^\dagger[i],\theta),y_i)\|_2^2
\]
or
\[
\mathcal R_{v2}(\theta)\simeq \frac1{np}\sum_{i,j}\|R_j[\nabla_x\ell(f(x_i,\theta),y_i)]\|_2^2.
\]
The paper reports, on MNIST, baseline FGSM accuracy \(5.9\%\to\) up to \(\approx64\)–\(69\%\), PGD \(0.7\%\to\approx52\)–\(60\%\), DeepFool \(34.8\%\to\approx94\)–\(96\%\), and C\&W \(29\%\to\approx93\)–\(95\%\). On CIFAR-10, baseline DeepFool \(39.8\%\to\approx58\)–\(61\%\) and C\&W \(0.0\%\to\approx48\)–\(58\%\) [2102.09230].

In generative modeling, "Stabilizing GAN Training with Multiple Random Projections" [1705.07831] replaces a single discriminator with \(m\) discriminators \(D_i\), each receiving only \(R_i x\) or \(R_i G(z)\), where \(R_i\in\mathbb R^{k\times d}\) is a fixed random projection. The joint objective is
\[
\min_G\max_{D_1,\dots,D_m}\sum_{i=1}^m V_i(D_i,G),
\]
and the theoretical motivation combines an information bottleneck argument with projected-marginal matching. The paper states that projected discriminators are unable to reject generated samples perfectly and continue to provide meaningful gradients to the generator throughout training. Empirically, with \(m=48\) projections, the generator loss remains in a regime that supplies useful gradients through \(100\,K\) iterations, whereas a single-discriminator GAN saturates early; increasing \(m\) from \(12\to24\to48\) reduces high-frequency artifacts, but beyond \(m\approx24\) no further gains in qualitative image quality are reported [1705.07831].

## 4. Sequential and detection-oriented uses

Multiple random projections are especially natural when a difficult high-dimensional problem can be reduced to many one-dimensional or low-dimensional tests. In single-change-point analysis, Xu and Rho project a \(p\)-variate time series \(X_t\) onto \(M\) random directions \(r^{(m)}\), forming scalar series
\[
X_t^{(m)}=\langle X_t,r^{(m)}\rangle,\qquad m=1,\dots,M.
\]
Each projected series is analyzed with a standard or weighted CUSUM statistic, and the resulting \(p\)-values are combined using Bonferroni, Holm, Fisher’s method, BH FDR control, or the harmonic mean \(p\). The paper states that \(M\approx200\) gives stable size and power across its simulation settings, that RMSE stabilizes for \(M\ge 200\), and that if variability in estimated locations is high the entire procedure should be repeated \(R\approx500\)–\(1000\) times, with the mode of the estimated locations used as the final guide [2602.19988].

High-dimensional outlier detection offers a sequential variant. The procedure of "High-dimensional outlier detection using random projections" [2005.08923] draws independent \(V\sim N_d(0,I_d)\), normalizes \(V/\|V\|\), and computes the standardized univariate statistic
\[
Y^V=\frac{z-m_V}{M_V},
\]
where \(z=x'V\), \(m_V\) is a robust center, and \(M_V\) is a robust scale such as MADN. Two thresholds \(0<a\le b\) define a continuation region: if \(|Y^V|<a\), declare regular; if \(|Y^V|>b\), declare outlier; otherwise draw another projection. The stopping time is
\[
K_n^{a,b}=\inf\{k\ge1: |Y^k|<a\ \lor\ |Y^k|>b\}.
\]
The paper states that the overall Type I error is exactly \(\alpha\), derives
\[
E[K_n\mid \|X\|=t]=\frac1{1-P(|Y^V|\in(a,b)\mid \|X\|=t)},
\]
and recommends repeating the classification \(T\approx100\) times to reduce randomness [2005.08923].

In beamforming, multiple random projections become a first-stage compression scheme. The framework of "Beamforming with Random Projections: Upper and Lower Bounds" [2507.05662] draws \(P\) independent projection matrices \(\Psi_p\in\mathbb C^{N_d\times N_m}\), forms low-dimensional sensor views \(y_p=\Psi_p y\), computes a compressed MVDR beamformer for each projection, and then chooses the output with minimum instantaneous power at each time-frequency bin:
\[
\gamma[i,k]=\arg\min_{p=1,\dots,P}|z_p[i,k]|^2,\qquad z[i,k]=z_{\gamma[i,k]}[i,k].
\]
The paper states that the mixture output can exceed the full-dimensional MVDR in SINR, while adding a new computational degree of freedom. Complexity is \(P\cdot O(N_d^3)\) rather than \(O(N_m^3)\), and to keep complexity roughly equal the recommended scaling is \(P\approx (N_m/N_d)^3\) [2507.05662].

## 5. Structured, low-memory, invertible, and optical constructions

Tensor Random Projection is a canonical structured multiple-projection map. If \(d=\prod_{n=1}^N d_n\) and \(A^{(n)}\in\mathbb R^{d_n\times k}\) are independent small random projection matrices, the TRP map is
\[
\phi_{\rm TRP}(x)=\bigl(A^{(1)}\odot\cdots\odot A^{(N)}\bigr)^T x.
\]
Storage drops from \(\mathcal O(dk)\) for a conventional dense map to \(\sum_{n=1}^N d_nk\), and when \(d_n\approx d^{1/N}\) this becomes \(Nk\,d^{1/N}\ll kd\). The map is unbiased as an isometry, since \(E\|\phi_{\rm TRP}(x)\|_2^2=\|x\|_2^2\), but its variance depends on the fourth moment \(\Delta\):
\[
\operatorname{Var}(\|\phi_{\rm TRP}(x)\|_2^2)=\frac1k\Bigl[(\Delta^N-3)\|x\|_4^4+2\|x\|_2^4\Bigr].
\]
Averaging \(T\) independent TRPs in TRP(\(T\)) reduces the first term by \(1/T\). For \(N=2\), the embedding is a JL map with \(k=\mathcal O(\epsilon^{-2}\log^4 n)\). Empirically, when \(d=40000\), TRP(5) uses only \(k(d_1+d_2)\approx k\cdot 400\) storage versus \(k\cdot 40000\) for ordinary Gaussian RP, a \(1/100\) reduction in memory, and on MNIST with Gaussian maps and \(k=200\) the reported inner-product RMSE is \(0.1198\pm0.0147\) for RP, \(0.1540\pm0.0290\) for TRP, and \(0.1262\pm0.0166\) for TRP(5) [2105.00105].

A separate line of work constructs invertible random projections from hierarchical Givens rotations. In "Random Paraunitary Projections" [2106.10746], an \(M\times M\) unitary matrix is factored into \(\ell=M(M-1)/2\) plane rotations, with random angles generated on the fly from a hierarchy of seeds. The forward transform requires approximately \(3M(M-1)\) flops and storage approximately \(M+N_s+(\ell/N_s)\), while the inverse is obtained by reversing the rotation order and signs. These unitary blocks are then assembled into random paraunitary filter banks and an adaptive under-decimated system in which the compression ratio \(q_t\) varies blockwise according to a local sparsity metric \(\rho_t=\|x_t\|_1/\|x_t\|_2\) [2106.10746].

Physical implementations make multiple random projections literal rather than algorithmic. In the optical scattering system of "Random Projections through multiple optical scattering: Approximating kernels at the speed of light" [1510.06664], a scattering medium realizes a complex Gaussian transmission matrix \(H\), so that \(y=Hx\) and features are obtained by a nonlinear detection stage \(\phi(x)=\sigma(Wx)\), with \(\sigma(u)=|u|\) in the experiments. The induced kernel converges to an elliptic kernel as the number of optical features \(N\) grows. On MNIST, the reported optical misclassification rates are approximately \(3.7\%\) at \(N=1000\), \(2.7\%\) at \(N=2000\), \(2.2\%\) at \(N=5000\), and \(1.9\%\) at \(N=10000\), approaching an asymptotic kernel performance of \(1.31\%\) [1510.06664].

A related optical method removes holography by combining multiple intensity measurements. With a fixed anchor \(x_a\), Ohana et al. derive the linear map
\[
L(x)=\frac{|Mx_a|^2+|Mx|^2-|M(x_a-x)|^2}{2|Mx_a|},
\]
which is obtained from intensity-only observations yet is linear in \(x\). The paper states that the resulting matrix has real-valued, independent, and identically distributed Gaussian entries; experimentally, a \(4096\times4096\) matrix \(L\) was built, its singular-value density followed the Marchenko–Pastur law, and Johnson–Lindenstrauss tests on \(100\) random vectors in \(\mathbb R^{10^6}\) yielded relative-error \(\sigma\approx0.22\) in both optical and numerical simulations [2305.12988].

## 6. Geometry, asymptotics, and recurring trade-offs

The geometric theory of random projections on low-dimensional structure is sharpened in "Random projections of random manifolds" [1607.04331]. For a \(K\)-dimensional smooth Gaussian random manifold \(M\subset\mathbb R^N\), with volume \(V=\prod_\alpha L_\alpha/\lambda_\alpha\), the paper gives the approximate high-probability bound
\[
M^*(\varepsilon,\delta)\lesssim \frac{16}{\varepsilon^2}\Bigl[\ln V+\ln \frac1\delta+K\ln\Bigl(\frac{9\sqrt3\,e\,N}{\varepsilon\sqrt K}\Bigr)\Bigr],
\]
so the required projection dimension grows only logarithmically in the ambient dimension \(N\). The numerical experiments are summarized by
\[
M^*_{\rm sim}\approx \frac{1.2\ln V+2.5K}{\varepsilon^2},
\]
with substantially smaller prefactors than the theoretical bound, and the abstract states that the new bounds are tighter than previous results by several orders of magnitude [1607.04331].

Across the broader literature, the principal trade-off is not simply “more projections are better,” but rather how many projections are used, how strong each projected model is, and how the outputs are aggregated. Averaging over independent sketches strictly reduces MSE in large-scale regression [1701.05325]; increasing the number of groups in ensemble dimension reduction reduces Monte Carlo error at rate \(O(G^{-1/2})\) [2410.04922]; and TRP(\(T\)) reduces variance relative to plain TRP [2105.00105]. By contrast, bidirectional OLS introduces an excess-risk gap \(\Delta(p_1)=O(p_1)+O(1/p_1)\) and “never outperforms the one-sided sketch in the limit of large \(p_1\)” [2606.10377]. In MRP-GAN, training time grows roughly linearly in \(m\), and beyond \(m\approx24\) no further qualitative image gains are reported [1705.07831]. In change-point analysis, larger \(M\) beyond \(200\) yields diminishing returns [2602.19988]. In beamforming, smaller \(N_d\) increases projection diversity but also per-projection distortion, so \(N_d\) and \(P\) become coupled design variables [2507.05662].

A plausible implication is that multiple random projections are best understood as a family of aggregation strategies rather than a single dimensionality-reduction primitive. The repeated random views can be used to stabilize optimization, approximate low-dimensional structure, reduce memory, exploit parallelism, or implement randomness directly in hardware, but the resulting statistical and computational behavior depends on whether the multiple projections are averaged, selected, voted, adjusted, mixed, or composed.

Source: https://www.emergentmind.com/topics/multiple-random-projections