Multiple Random Projections Overview

• Multiple random projections are methods that use several independent or structured random linear maps to reduce high-dimensional data while preserving key structural properties.
• They leverage aggregation techniques such as estimator averaging, thresholded voting, and SVD to control bias and variance in various applications.
• Applications include regression, classification, generative modeling, beamforming, outlier detection, and change-point analysis, each with tailored trade-offs.

to=arxiv_search 总代理联系 大发游戏官网Params 大发时时彩开奖ia 天天中彩票有_coupon code Multiple random projections denote methods that apply several random linear maps to the same data and then combine the resulting low-dimensional views, rather than relying on a single sketch. Across the literature, the combination step appears as estimator averaging, within-block screening, thresholded voting, singular-value aggregation, multiple-comparison correction, repeated testing with mode aggregation, per-bin mixture selection, and multishot physical measurement. The paradigm is used in randomized regression, classification, adversarial robustness, generative modeling, change-point analysis, outlier detection, beamforming, low-memory embeddings, invertible filter-bank constructions, and optical implementations of random feature maps (Thanei et al., 2017, Cannings et al., 2015, Zhou et al., 2024, Lan et al., 9 Jun 2026).

1. Core formulations and design patterns

At the level of notation, the literature instantiates multiple random projections through several families of maps: orthonormal projections $A_m:\mathbb R^p\to\mathbb R^d$, Gaussian sketches $R_j\in\mathbb R^{k_j\times d}$, one-dimensional random directions $r^{(m)}\in\mathbb R^p$, complex beamforming compressions $\Psi_p\in\mathbb C^{N_d\times N_m}$, and structured tensorized maps $A^{(1)}\odot\cdots\odot A^{(N)}$. The projection law varies by task and includes Haar-distributed orthonormal matrices, i.i.d. Gaussian entries, Rademacher entries, sparse “3-point” projections, subsampled randomized Hadamard transforms, and complex Gaussian transmission matrices (Cannings et al., 2015, Xu et al., 23 Feb 2026, Thanei et al., 2017, Mittal et al., 8 Jul 2025, Saade et al., 2015).

A common distinction is between independent repetition and structured composition. Independent repetition draws many unrelated projections and aggregates the downstream outputs, as in random-projection ensemble classification, RP-Ensemble defenses, repeated change-point detection, or multiple compressed beamformers. Structured composition instead builds one projection from several smaller random components, as in Tensor Random Projection (TRP), bidirectional row-and-column sketching for OLS, paraunitary cascades, or optical multishot recovery of a Gaussian linear map (Sun et al., 2021, Lan et al., 9 Jun 2026, Queiroz, 2021, Ohana et al., 2023).

Setting	Projection mechanism	Combination rule
Classification	$B_1B_2$ Haar or Gaussian $A_{b_1,b_2}\in\mathbb R^{d\times p}$	within-block selection and thresholded voting
Regression dimension reduction	grouped $P_{g,m}\in\mathbb R^{p\times d}$	empirical-error screening and SVD of $\bar P$
Change-point analysis	$M$ one-dimensional random vectors $R_j\in\mathbb R^{k_j\times d}$0	adjusted $R_j\in\mathbb R^{k_j\times d}$1-values, then repeated mode
Tensor RP	$R_j\in\mathbb R^{k_j\times d}$2	optional averaging over $R_j\in\mathbb R^{k_j\times d}$3 independent TRPs
Beamforming	$R_j\in\mathbb R^{k_j\times d}$4 compressed views $R_j\in\mathbb R^{k_j\times d}$5	time-frequency minimum-power mixture

This diversity matters because “multiple random projections” does not imply a single statistical effect. In some settings repetition reduces variance or Monte Carlo error; in others an additional projection stage degrades statistical accuracy relative to a one-sided baseline. The combination operator is therefore part of the method, not an implementation detail.

2. Regression, sketching, and ensemble dimension reduction

In large-scale least-squares regression, Thanei, Heinze, and Meinshausen describe a $R_j\in\mathbb R^{k_j\times d}$6-fold sketch-and-solve procedure in which $R_j\in\mathbb R^{k_j\times d}$7 independent column sketches $R_j\in\mathbb R^{k_j\times d}$8 are drawn, each sketched problem is solved in parallel, and the lifted estimators $R_j\in\mathbb R^{k_j\times d}$9 are averaged as

$r^{(m)}\in\mathbb R^p$0

The limiting estimator $r^{(m)}\in\mathbb R^p$1 has an AMSE bound

$r^{(m)}\in\mathbb R^p$2

with $r^{(m)}\in\mathbb R^p$3, and the paper states that averaging strictly reduces the MSE relative to a single sketch. In the isotropic case $r^{(m)}\in\mathbb R^p$4, the variance reduction factor is $r^{(m)}\in\mathbb R^p$5 compared to a single sketch, and the resulting bias-variance behavior is compared to ridge regression and principal component regression (Thanei et al., 2017).

A different use of multiple projections appears in fixed-design OLS under bidirectional sketching. In "Bidirectional Random Projections" (Lan et al., 9 Jun 2026), the one-sided estimator uses a Gaussian column sketch $r^{(m)}\in\mathbb R^p$6, while the bidirectional estimator also uses a Gaussian row sketch $r^{(m)}\in\mathbb R^p$7, forming $r^{(m)}\in\mathbb R^p$8. The one-sided excess-risk bound is

$r^{(m)}\in\mathbb R^p$9

whereas the bidirectional bound is

$\Psi_p\in\mathbb C^{N_d\times N_m}$0

The gap satisfies

$\Psi_p\in\mathbb C^{N_d\times N_m}$1

and the $\Psi_p\in\mathbb C^{N_d\times N_m}$2 coefficient changes sign at $\Psi_p\in\mathbb C^{N_d\times N_m}$3. The paper further states that the bi-sketched estimator is always slightly worse than the one-sided sketch in the reported experiments, and that the two-sided reduction strictly compromises statistical accuracy relative to a one-sided column sketch (Lan et al., 9 Jun 2026).

Multiple random projections are also used for supervised subspace estimation rather than direct prediction. In "Random-projection ensemble dimension reduction" (Zhou et al., 2024), Zhou and Cannings divide the projections into $\Psi_p\in\mathbb C^{N_d\times N_m}$4 groups of size $\Psi_p\in\mathbb C^{N_d\times N_m}$5, evaluate each projected regressor on a hold-out set, keep the best $\Psi_p\in\mathbb C^{N_d\times N_m}$6 in each group, and aggregate via

$\Psi_p\in\mathbb C^{N_d\times N_m}$7

An SVD of $\Psi_p\in\mathbb C^{N_d\times N_m}$8 yields directions $\Psi_p\in\mathbb C^{N_d\times N_m}$9 and singular values $A^{(1)}\odot\cdots\odot A^{(N)}$0, where $A^{(1)}\odot\cdots\odot A^{(N)}$1 are interpreted as importance scores for selecting the final dimension. The theoretical perturbation bound contains an $A^{(1)}\odot\cdots\odot A^{(N)}$2 term, so the subspace error stabilizes as the number of groups increases. The paper provides default recommendations $A^{(1)}\odot\cdots\odot A^{(N)}$3, $A^{(1)}\odot\cdots\odot A^{(N)}$4, $A^{(1)}\odot\cdots\odot A^{(N)}$5, and $A^{(1)}\odot\cdots\odot A^{(N)}$6, and also studies a second-pass “double-RPE” strategy when the initial recommended dimension is still too large (Zhou et al., 2024).

3. Classification, adversarial robustness, and generative modeling

Ensemble classification by multiple random projections was formalized by Cannings and Samworth through $A^{(1)}\odot\cdots\odot A^{(N)}$7 independent projections $A^{(1)}\odot\cdots\odot A^{(N)}$8, typically Haar-distributed on the Stiefel manifold. Within each block $A^{(1)}\odot\cdots\odot A^{(N)}$9, the projection minimizing an estimated test error $B_1B_2$0 is selected, producing $B_1B_2$1 retained projections. For a test point $B_1B_2$2, the vote proportion is

$B_1B_2$3

and the final classifier is $B_1B_2$4, with $B_1B_2$5 chosen data-adaptively. Under a sufficient dimension reduction boundary condition $B_1B_2$6, the excess risk can be controlled by terms that do not depend on the original dimension $B_1B_2$7 and a Monte Carlo term that is $B_1B_2$8 (Cannings et al., 2015).

Carbone et al. use multiple Gaussian projections for adversarial robustness in two ways. RP-Ensemble trains $B_1B_2$9 separate classifiers $A_{b_1,b_2}\in\mathbb R^{d\times p}$0 on projected data $A_{b_1,b_2}\in\mathbb R^{d\times p}$1, with $A_{b_1,b_2}\in\mathbb R^{d\times p}$2 and $A_{b_1,b_2}\in\mathbb R^{d\times p}$3, and combines them at inference by summing class probabilities with the original full-dimensional classifier. RP-Regularizer instead augments the training loss by one of two penalties,

$A_{b_1,b_2}\in\mathbb R^{d\times p}$4

or

$A_{b_1,b_2}\in\mathbb R^{d\times p}$5

The paper reports, on MNIST, baseline FGSM accuracy $A_{b_1,b_2}\in\mathbb R^{d\times p}$6 up to $A_{b_1,b_2}\in\mathbb R^{d\times p}$7–$A_{b_1,b_2}\in\mathbb R^{d\times p}$8, PGD $A_{b_1,b_2}\in\mathbb R^{d\times p}$9–$P_{g,m}\in\mathbb R^{p\times d}$0, DeepFool $P_{g,m}\in\mathbb R^{p\times d}$1–$P_{g,m}\in\mathbb R^{p\times d}$2, and C&W $P_{g,m}\in\mathbb R^{p\times d}$3–$P_{g,m}\in\mathbb R^{p\times d}$4. On CIFAR-10, baseline DeepFool $P_{g,m}\in\mathbb R^{p\times d}$5–$P_{g,m}\in\mathbb R^{p\times d}$6 and C&W $P_{g,m}\in\mathbb R^{p\times d}$7–$P_{g,m}\in\mathbb R^{p\times d}$8 (Carbone et al., 2021).

In generative modeling, "Stabilizing GAN Training with Multiple Random Projections" (Neyshabur et al., 2017) replaces a single discriminator with $P_{g,m}\in\mathbb R^{p\times d}$9 discriminators $\bar P$0, each receiving only $\bar P$1 or $\bar P$2, where $\bar P$3 is a fixed random projection. The joint objective is

$\bar P$4

and the theoretical motivation combines an information bottleneck argument with projected-marginal matching. The paper states that projected discriminators are unable to reject generated samples perfectly and continue to provide meaningful gradients to the generator throughout training. Empirically, with $\bar P$5 projections, the generator loss remains in a regime that supplies useful gradients through $\bar P$6 iterations, whereas a single-discriminator GAN saturates early; increasing $\bar P$7 from $\bar P$8 reduces high-frequency artifacts, but beyond $\bar P$9 no further gains in qualitative image quality are reported (Neyshabur et al., 2017).

4. Sequential and detection-oriented uses

Multiple random projections are especially natural when a difficult high-dimensional problem can be reduced to many one-dimensional or low-dimensional tests. In single-change-point analysis, Xu and Rho project a $M$0-variate time series $M$1 onto $M$2 random directions $M$3, forming scalar series

$M$4

Each projected series is analyzed with a standard or weighted CUSUM statistic, and the resulting $M$5-values are combined using Bonferroni, Holm, Fisher’s method, BH FDR control, or the harmonic mean $M$6. The paper states that $M$7 gives stable size and power across its simulation settings, that RMSE stabilizes for $M$8, and that if variability in estimated locations is high the entire procedure should be repeated $M$9–$R_j\in\mathbb R^{k_j\times d}$00 times, with the mode of the estimated locations used as the final guide (Xu et al., 23 Feb 2026).

High-dimensional outlier detection offers a sequential variant. The procedure of "High-dimensional outlier detection using random projections" (Navarro-Esteban et al., 2020) draws independent $R_j\in\mathbb R^{k_j\times d}$01, normalizes $R_j\in\mathbb R^{k_j\times d}$02, and computes the standardized univariate statistic

$R_j\in\mathbb R^{k_j\times d}$03

where $R_j\in\mathbb R^{k_j\times d}$04, $R_j\in\mathbb R^{k_j\times d}$05 is a robust center, and $R_j\in\mathbb R^{k_j\times d}$06 is a robust scale such as MADN. Two thresholds $R_j\in\mathbb R^{k_j\times d}$07 define a continuation region: if $R_j\in\mathbb R^{k_j\times d}$08, declare regular; if $R_j\in\mathbb R^{k_j\times d}$09, declare outlier; otherwise draw another projection. The stopping time is

$R_j\in\mathbb R^{k_j\times d}$10

The paper states that the overall Type I error is exactly $R_j\in\mathbb R^{k_j\times d}$11, derives

$R_j\in\mathbb R^{k_j\times d}$12

and recommends repeating the classification $R_j\in\mathbb R^{k_j\times d}$13 times to reduce randomness (Navarro-Esteban et al., 2020).

In beamforming, multiple random projections become a first-stage compression scheme. The framework of "Beamforming with Random Projections: Upper and Lower Bounds" (Mittal et al., 8 Jul 2025) draws $R_j\in\mathbb R^{k_j\times d}$14 independent projection matrices $R_j\in\mathbb R^{k_j\times d}$15, forms low-dimensional sensor views $R_j\in\mathbb R^{k_j\times d}$16, computes a compressed MVDR beamformer for each projection, and then chooses the output with minimum instantaneous power at each time-frequency bin: $R_j\in\mathbb R^{k_j\times d}$17 The paper states that the mixture output can exceed the full-dimensional MVDR in SINR, while adding a new computational degree of freedom. Complexity is $R_j\in\mathbb R^{k_j\times d}$18 rather than $R_j\in\mathbb R^{k_j\times d}$19, and to keep complexity roughly equal the recommended scaling is $R_j\in\mathbb R^{k_j\times d}$20 (Mittal et al., 8 Jul 2025).

5. Structured, low-memory, invertible, and optical constructions

Tensor Random Projection is a canonical structured multiple-projection map. If $R_j\in\mathbb R^{k_j\times d}$21 and $R_j\in\mathbb R^{k_j\times d}$22 are independent small random projection matrices, the TRP map is

$R_j\in\mathbb R^{k_j\times d}$23

Storage drops from $R_j\in\mathbb R^{k_j\times d}$24 for a conventional dense map to $R_j\in\mathbb R^{k_j\times d}$25, and when $R_j\in\mathbb R^{k_j\times d}$26 this becomes $R_j\in\mathbb R^{k_j\times d}$27. The map is unbiased as an isometry, since $R_j\in\mathbb R^{k_j\times d}$28, but its variance depends on the fourth moment $R_j\in\mathbb R^{k_j\times d}$29: $R_j\in\mathbb R^{k_j\times d}$30 Averaging $R_j\in\mathbb R^{k_j\times d}$31 independent TRPs in TRP($R_j\in\mathbb R^{k_j\times d}$32) reduces the first term by $R_j\in\mathbb R^{k_j\times d}$33. For $R_j\in\mathbb R^{k_j\times d}$34, the embedding is a JL map with $R_j\in\mathbb R^{k_j\times d}$35. Empirically, when $R_j\in\mathbb R^{k_j\times d}$36, TRP(5) uses only $R_j\in\mathbb R^{k_j\times d}$37 storage versus $R_j\in\mathbb R^{k_j\times d}$38 for ordinary Gaussian RP, a $R_j\in\mathbb R^{k_j\times d}$39 reduction in memory, and on MNIST with Gaussian maps and $R_j\in\mathbb R^{k_j\times d}$40 the reported inner-product RMSE is $R_j\in\mathbb R^{k_j\times d}$41 for RP, $R_j\in\mathbb R^{k_j\times d}$42 for TRP, and $R_j\in\mathbb R^{k_j\times d}$43 for TRP(5) (Sun et al., 2021).

A separate line of work constructs invertible random projections from hierarchical Givens rotations. In "Random Paraunitary Projections" (Queiroz, 2021), an $R_j\in\mathbb R^{k_j\times d}$44 unitary matrix is factored into $R_j\in\mathbb R^{k_j\times d}$45 plane rotations, with random angles generated on the fly from a hierarchy of seeds. The forward transform requires approximately $R_j\in\mathbb R^{k_j\times d}$46 flops and storage approximately $R_j\in\mathbb R^{k_j\times d}$47, while the inverse is obtained by reversing the rotation order and signs. These unitary blocks are then assembled into random paraunitary filter banks and an adaptive under-decimated system in which the compression ratio $R_j\in\mathbb R^{k_j\times d}$48 varies blockwise according to a local sparsity metric $R_j\in\mathbb R^{k_j\times d}$49 (Queiroz, 2021).

Physical implementations make multiple random projections literal rather than algorithmic. In the optical scattering system of "Random Projections through multiple optical scattering: Approximating kernels at the speed of light" (Saade et al., 2015), a scattering medium realizes a complex Gaussian transmission matrix $R_j\in\mathbb R^{k_j\times d}$50, so that $R_j\in\mathbb R^{k_j\times d}$51 and features are obtained by a nonlinear detection stage $R_j\in\mathbb R^{k_j\times d}$52, with $R_j\in\mathbb R^{k_j\times d}$53 in the experiments. The induced kernel converges to an elliptic kernel as the number of optical features $R_j\in\mathbb R^{k_j\times d}$54 grows. On MNIST, the reported optical misclassification rates are approximately $R_j\in\mathbb R^{k_j\times d}$55 at $R_j\in\mathbb R^{k_j\times d}$56, $R_j\in\mathbb R^{k_j\times d}$57 at $R_j\in\mathbb R^{k_j\times d}$58, $R_j\in\mathbb R^{k_j\times d}$59 at $R_j\in\mathbb R^{k_j\times d}$60, and $R_j\in\mathbb R^{k_j\times d}$61 at $R_j\in\mathbb R^{k_j\times d}$62, approaching an asymptotic kernel performance of $R_j\in\mathbb R^{k_j\times d}$63 (Saade et al., 2015).

A related optical method removes holography by combining multiple intensity measurements. With a fixed anchor $R_j\in\mathbb R^{k_j\times d}$64, Ohana et al. derive the linear map

$R_j\in\mathbb R^{k_j\times d}$65

which is obtained from intensity-only observations yet is linear in $R_j\in\mathbb R^{k_j\times d}$66. The paper states that the resulting matrix has real-valued, independent, and identically distributed Gaussian entries; experimentally, a $R_j\in\mathbb R^{k_j\times d}$67 matrix $R_j\in\mathbb R^{k_j\times d}$68 was built, its singular-value density followed the Marchenko–Pastur law, and Johnson–Lindenstrauss tests on $R_j\in\mathbb R^{k_j\times d}$69 random vectors in $R_j\in\mathbb R^{k_j\times d}$70 yielded relative-error $R_j\in\mathbb R^{k_j\times d}$71 in both optical and numerical simulations (Ohana et al., 2023).

6. Geometry, asymptotics, and recurring trade-offs

The geometric theory of random projections on low-dimensional structure is sharpened in "Random projections of random manifolds" (Lahiri et al., 2016). For a $R_j\in\mathbb R^{k_j\times d}$72-dimensional smooth Gaussian random manifold $R_j\in\mathbb R^{k_j\times d}$73, with volume $R_j\in\mathbb R^{k_j\times d}$74, the paper gives the approximate high-probability bound

$R_j\in\mathbb R^{k_j\times d}$75

so the required projection dimension grows only logarithmically in the ambient dimension $R_j\in\mathbb R^{k_j\times d}$76. The numerical experiments are summarized by

$R_j\in\mathbb R^{k_j\times d}$77

with substantially smaller prefactors than the theoretical bound, and the abstract states that the new bounds are tighter than previous results by several orders of magnitude (Lahiri et al., 2016).

Across the broader literature, the principal trade-off is not simply “more projections are better,” but rather how many projections are used, how strong each projected model is, and how the outputs are aggregated. Averaging over independent sketches strictly reduces MSE in large-scale regression (Thanei et al., 2017); increasing the number of groups in ensemble dimension reduction reduces Monte Carlo error at rate $R_j\in\mathbb R^{k_j\times d}$78 (Zhou et al., 2024); and TRP($R_j\in\mathbb R^{k_j\times d}$79) reduces variance relative to plain TRP (Sun et al., 2021). By contrast, bidirectional OLS introduces an excess-risk gap $R_j\in\mathbb R^{k_j\times d}$80 and “never outperforms the one-sided sketch in the limit of large $R_j\in\mathbb R^{k_j\times d}$81” (Lan et al., 9 Jun 2026). In MRP-GAN, training time grows roughly linearly in $R_j\in\mathbb R^{k_j\times d}$82, and beyond $R_j\in\mathbb R^{k_j\times d}$83 no further qualitative image gains are reported (Neyshabur et al., 2017). In change-point analysis, larger $R_j\in\mathbb R^{k_j\times d}$84 beyond $R_j\in\mathbb R^{k_j\times d}$85 yields diminishing returns (Xu et al., 23 Feb 2026). In beamforming, smaller $R_j\in\mathbb R^{k_j\times d}$86 increases projection diversity but also per-projection distortion, so $R_j\in\mathbb R^{k_j\times d}$87 and $R_j\in\mathbb R^{k_j\times d}$88 become coupled design variables (Mittal et al., 8 Jul 2025).

A plausible implication is that multiple random projections are best understood as a family of aggregation strategies rather than a single dimensionality-reduction primitive. The repeated random views can be used to stabilize optimization, approximate low-dimensional structure, reduce memory, exploit parallelism, or implement randomness directly in hardware, but the resulting statistical and computational behavior depends on whether the multiple projections are averaged, selected, voted, adjusted, mixed, or composed.