Projected Support Points for Data Reduction

• Projected support points are representative designs that minimize a projection-aware kernel discrepancy, preserving key low-dimensional features in high-dimensional data.
• They use sparsity-inducing and weighted Gaussian kernels to emphasize low-dimensional coordinate similarities, achieving improved integration rates over Monte Carlo methods.
• PSPs offer robust theoretical guarantees and computational methods, making them effective in applications such as Gaussian process emulation and kernel-based learning.

Projected support points (PSPs) are representative point sets for high-dimensional data reduction and experimental design that extend ordinary support points by making the discrepancy criterion sensitive to low-dimensional coordinate projections rather than only to full-space geometry. In the original data-reduction formulation, PSPs are defined by minimizing a kernel discrepancy built from a sparsity-inducing kernel, with the goal that a reduced set of $n \ll N$ points preserves the low-dimensional features of a large dataset that matter for downstream computation (Mak et al., 2017). In later computer-experiment work, $\pi$-projected support points are defined through a generalized Gaussian kernel averaged over coordinate-subspace weights, so that designs are representative both on the full space and on projected subspaces; this is particularly useful for Gaussian process emulation when only some factors or low-order interactions are active (Mak et al., 12 Jul 2025).

1. Support-point foundations

Ordinary support points arise from minimizing the energy distance between a target distribution $F$ and an empirical distribution $F_n$ supported on finitely many points. For $Y \sim F$, and $F_n = n^{-1}\sum_{i=1}^n \delta_{x_i}$, the support-point objective is (Mak et al., 2016)

$$\{\xi_i\}_{i=1}^n \in \arg\min_{x_1,\dots,x_n} E(F,F_n),$$

with

$$E(F,F_n) = \frac{2}{n}\sum_{i=1}^n \mathbb E\|x_i-Y\|_2 - \frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n \|x_i-x_j\|_2 - \mathbb E\|Y-Y'\|_2.$$

Up to the constant term $\mathbb E\|Y-Y'\|_2$, this has the familiar pull–push structure: one term attracts points toward $F$, while the pairwise term repels them from one another. The resulting point sets are deterministic after optimization and are designed to be representative of $\pi$0 in a global distributional sense (Mak et al., 2016).

This construction has nontrivial theory. Support points converge in distribution to $\pi$1, and the paper proves a Koksma–Hlawka-like bound

$\pi$2

for $\pi$3 in a function class $\pi$4 derived from the native space of the kernel $\pi$5 (Mak et al., 2016). It also establishes rate improvements over Monte Carlo for a large class of integrands, with bounds of the form

$\pi$6

under eigenvalue-summability conditions (Mak et al., 2016).

The limitation that motivates PSPs is explicit in later work: ordinary support points are representative primarily of the full distribution $\pi$7 and need not represent the marginal distributions of $\pi$8 well in high dimensions (Mak et al., 12 Jul 2025). The 2016 support-point paper also identifies the missing “projective property” directly, noting that in high dimensions representative points should ideally represent not only the full distribution $\pi$9 but also its marginals (Mak et al., 2016). Projected support points are the response to that limitation.

2. Original projected support points for high-dimensional data reduction

The original PSP formulation considers big data $F$0, with empirical distribution $F$1, and seeks a reduced dataset

$F$2

such that

$F$3

for a large class of downstream functions $F$4 (Mak et al., 2017). The central quantity is the kernel discrepancy

$F$5

which equals the worst-case integration error over the unit ball of the RKHS induced by $F$6 (Mak et al., 2017).

Projected support points differ from earlier discrepancy minimizers through the kernel choice. Instead of the ordinary support-point kernel $F$7, the PSP paper introduces the sparsity-inducing kernel. It begins from the weighted Gaussian family

$F$8

with product-and-order dependent weights

$F$9

A prior $F_n$0 is then placed on the product weights $F_n$1, and the sparsity-inducing kernel is defined by prior averaging: $F_n$2 Projected support points are then discrepancy minimizers under this kernel, either as $F_n$3-weighted PSPs or $F_n$4-expected PSPs (Mak et al., 2017).

The practical finite-data objective is

$F_n$5

The conceptual point is that low-dimensional coordinate matches are rewarded much more heavily than they would be under a full-space radial kernel. The paper states this in design-of-experiments language: effect sparsity, effect hierarchy, and effect heredity are encoded through the product weights $F_n$6, the order weights $F_n$7, and the multiplicative POD structure, respectively (Mak et al., 2017).

For the simplified anisotropic kernel

$F_n$8

with i.i.d. Gamma prior

$F_n$9

the SpIn kernel has the closed form

$Y \sim F$0

This makes the projection sensitivity explicit: similarity can remain large when points match closely on a small number of coordinates even if they are not close in the ambient Euclidean metric (Mak et al., 2017).

3. Theoretical guarantees and optimization in the original PSP framework

The original PSP theory is built around tractability under sparsity assumptions. For functions $Y \sim F$1 in the RKHS of the anisotropic Gaussian kernel, with POD coefficient structure

$Y \sim F$2

the main dimension-robust theorem states that if

$Y \sim F$3

then

$Y \sim F$4

for a constant $Y \sim F$5 independent of $Y \sim F$6 (Mak et al., 2017). This is the paper’s explicit statement that the curse of dimensionality in data reduction can be lifted under sparsity and hierarchy conditions.

For fixed dimension, the paper proves a logarithmically improved rate over Monte Carlo: $Y \sim F$7 under a moment condition on $Y \sim F$8, where $Y \sim F$9 is the active set and $F_n = n^{-1}\sum_{i=1}^n \delta_{x_i}$0 (Mak et al., 2017). The paper also remarks that empirical behavior is often closer to $F_n = n^{-1}\sum_{i=1}^n \delta_{x_i}$1, as is frequently observed in kernel quadrature and herding settings (Mak et al., 2017).

Computation is handled by two algorithms, both based on big-data subsampling and majorization–minimization. The one-shot algorithm, psp.mm, jointly optimizes all $F_n = n^{-1}\sum_{i=1}^n \delta_{x_i}$2 points. The sequential algorithm, psp.mm.seq, adds points greedily using a herding-like criterion

$F_n = n^{-1}\sum_{i=1}^n \delta_{x_i}$3

with the exact argmax approximated by the same stochastic MM machinery (Mak et al., 2017).

The computational bottlenecks are evaluating kernel quantities and updating one point at a time. The paper gives per-iteration costs of

$F_n = n^{-1}\sum_{i=1}^n \delta_{x_i}$4

for psp.mm, and

$F_n = n^{-1}\sum_{i=1}^n \delta_{x_i}$5

for adding the $F_n = n^{-1}\sum_{i=1}^n \delta_{x_i}$6-th point in psp.mm.seq; it recommends the sequential method when $F_n = n^{-1}\sum_{i=1}^n \delta_{x_i}$7 or $F_n = n^{-1}\sum_{i=1}^n \delta_{x_i}$8 (Mak et al., 2017). It also proves that psp.mm converges almost surely to a stationary limiting point set for the practical objective when $F_n = n^{-1}\sum_{i=1}^n \delta_{x_i}$9 is convex and compact (Mak et al., 2017).

4. Projection-aware support points for computer experiments and Gaussian processes

A later line of work repositions projected support points as a design class for computer experiments and GP emulation. In that setting, ordinary support points are recalled as

$$\{\xi_i\}_{i=1}^n \in \arg\min_{x_1,\dots,x_n} E(F,F_n),$$0

where $$\{\xi_i\}_{i=1}^n \in \arg\min_{x_1,\dots,x_n} E(F,F_n),$$1 is typically $$\{\xi_i\}_{i=1}^n \in \arg\min_{x_1,\dots,x_n} E(F,F_n),$$2 (Mak et al., 12 Jul 2025). The later paper stresses that these SPs need not represent marginal distributions well. Its PSPs therefore replace the energy-distance kernel with a generalized Gaussian kernel that explicitly accumulates distances over all coordinate subspaces: $$\{\xi_i\}_{i=1}^n \in \arg\min_{x_1,\dots,x_n} E(F,F_n),$$3 The $$\{\xi_i\}_{i=1}^n \in \arg\min_{x_1,\dots,x_n} E(F,F_n),$$4-projected support points are then defined as

$$\{\xi_i\}_{i=1}^n \in \arg\min_{x_1,\dots,x_n} E(F,F_n),$$5

This criterion explicitly quantifies similarities on the full design space and on its projected subspaces (Mak et al., 12 Jul 2025).

The later paper adopts a specific product-and-order prior: $$\{\xi_i\}_{i=1}^n \in \arg\min_{x_1,\dots,x_n} E(F,F_n),$$6 This is intended to encode hierarchy, heredity, and sparsity directly into the design criterion (Mak et al., 12 Jul 2025). In that formulation, PSPs are not merely “support points with a different kernel”; they are projection-aware designs informed by prior beliefs about which low-dimensional effects are likely to matter.

A major structural result connects PSPs to MaxPro designs. Under a simplified kernel/prior choice in which $$\{\xi_i\}_{i=1}^n \in \arg\min_{x_1,\dots,x_n} E(F,F_n),$$7 for singleton subsets and $$\{\xi_i\}_{i=1}^n \in \arg\min_{x_1,\dots,x_n} E(F,F_n),$$8 otherwise, with $$\{\xi_i\}_{i=1}^n \in \arg\min_{x_1,\dots,x_n} E(F,F_n),$$9, the PSP objective becomes proportional to a MaxPro-type term plus a boundary correction term involving

$$E(F,F_n) = \frac{2}{n}\sum_{i=1}^n \mathbb E\|x_i-Y\|_2 - \frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n \|x_i-x_j\|_2 - \mathbb E\|Y-Y'\|_2.$$0

The MaxPro-type repulsion term preserves projected separation, while the boundary correction pushes points away from the boundaries, correcting a well-known boundary attraction of MaxPro designs (Mak et al., 12 Jul 2025).

This projection-aware design criterion is optimized with the psp.sccp algorithm, based on the convex-concave procedure. The paper states that the iterative PSP algorithm converges to a local optimum of the criterion, and recommends two practical modifications: warm-starting from a user’s design of choice—specifically MaxPro for PSPs—and replacing ordinary Monte Carlo from $$E(F,F_n) = \frac{2}{n}\sum_{i=1}^n \mathbb E\|x_i-Y\|_2 - \frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n \|x_i-x_j\|_2 - \mathbb E\|Y-Y'\|_2.$$1 with randomized Sobol' sequences (Mak et al., 12 Jul 2025).

In GP emulation, the practical guidance is explicit. The later paper states that SPs should be used when all factors are known to be active, whereas PSPs should be used when some factors are known to be less active (Mak et al., 12 Jul 2025). In its robustness experiments, when $$E(F,F_n) = \frac{2}{n}\sum_{i=1}^n \mathbb E\|x_i-Y\|_2 - \frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n \|x_i-x_j\|_2 - \mathbb E\|Y-Y'\|_2.$$2 of factors are active, SPs have the best efficiency; when only $$E(F,F_n) = \frac{2}{n}\sum_{i=1}^n \mathbb E\|x_i-Y\|_2 - \frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n \|x_i-x_j\|_2 - \mathbb E\|Y-Y'\|_2.$$3 are active, PSPs have the best efficiency, and this pattern holds for both smoother and rougher Gaussian-correlation regimes (Mak et al., 12 Jul 2025).

5. Empirical behavior and applications

The original PSP paper evaluates the method on simulation problems, kernel learning, and MCMC chain reduction. In synthetic experiments, PSPs are compared against Monte Carlo random sampling, herding with a standard Gaussian kernel, support points, and inverse-Sobol' points. For 10-dimensional Gaussian data reduced to $$E(F,F_n) = \frac{2}{n}\sum_{i=1}^n \mathbb E\|x_i-Y\|_2 - \frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n \|x_i-x_j\|_2 - \mathbb E\|Y-Y'\|_2.$$4, the paper reports near-perfect 1-dimensional marginals for PSPs, whereas herding and support points gave poor 1-dimensional projection preservation (Mak et al., 2017). On quantitative integration tests with sparse Gaussian-peak and additive-Gaussian functions, one-shot PSPs gave lower errors than random sampling, support points, and in some settings inverse-Sobol' points; sequential PSPs strongly outperformed random sampling, herding, and inverse-Sobol' in several high-dimensional sparse settings (Mak et al., 2017).

In kernel ridge regression on the Million Song Dataset, the reduction task uses $$E(F,F_n) = \frac{2}{n}\sum_{i=1}^n \mathbb E\|x_i-Y\|_2 - \frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n \|x_i-x_j\|_2 - \mathbb E\|Y-Y'\|_2.$$5 training songs with $$E(F,F_n) = \frac{2}{n}\sum_{i=1}^n \mathbb E\|x_i-Y\|_2 - \frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n \|x_i-x_j\|_2 - \mathbb E\|Y-Y'\|_2.$$6 continuous features and a reduced set of $$E(F,F_n) = \frac{2}{n}\sum_{i=1}^n \mathbb E\|x_i-Y\|_2 - \frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n \|x_i-x_j\|_2 - \mathbb E\|Y-Y'\|_2.$$7. The paper reports that PSPs give noticeably better predictive performance than random sampling and herding, with running times of 1583 seconds for random sampling, 3423 seconds for herding, and 3965 seconds for PSPs; full KRR without reduction would require roughly 78 years and about 1720 GB memory (Mak et al., 2017). The computational message is that PSP reduction remains feasible precisely where full kernel learning does not.

In MCMC chain reduction, the method is applied to posterior samples from a Bayesian Gaussian process emulator. With $$E(F,F_n) = \frac{2}{n}\sum_{i=1}^n \mathbb E\|x_i-Y\|_2 - \frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n \|x_i-x_j\|_2 - \mathbb E\|Y-Y'\|_2.$$8 posterior samples per time step, PSPs are compared with thinning, herding, and ordinary support points. For one-shot reduction with $$E(F,F_n) = \frac{2}{n}\sum_{i=1}^n \mathbb E\|x_i-Y\|_2 - \frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n \|x_i-x_j\|_2 - \mathbb E\|Y-Y'\|_2.$$9, PSPs have the smallest error for both posterior prediction and posterior uncertainty quantification; for sequential reduction with $\mathbb E\|Y-Y'\|_2$0, PSPs again have the best error, while herding can be much worse, especially for UQ (Mak et al., 2017). The interpretation given in the paper is that PSPs preserve posterior structure relevant to downstream prediction and uncertainty better than thinning and standard kernel methods.

The later GP-design paper studies a different empirical regime. It compares PSPs with minimax, MmLHD, uniform designs, MaxPro, and miniMaxPro. In $\mathbb E\|Y-Y'\|_2$1, PSPs improve the 1-dimensional projections substantially relative to SPs while sacrificing some 2-dimensional full-space filling; in $\mathbb E\|Y-Y'\|_2$2, they are a strong compromise between MaxPro and miniMaxPro, with especially strong low-dimensional projected maximin behavior for $\mathbb E\|Y-Y'\|_2$3 due to the boundary correction (Mak et al., 12 Jul 2025). On four benchmark test functions—exponential, Friedman, an 8-dimensional function, and wing weight—SPs and PSPs outperform the competing designs for nearly all design sizes, with PSPs especially strong for wing weight, interpreted as smooth with only some active factors (Mak et al., 12 Jul 2025).

A plausible implication is that the two PSP literatures solve closely related but not identical problems. The 2017 formulation targets empirical data reduction for downstream integration and learning, whereas the 2025 formulation targets experimental designs for GP emulation. What they share is a projection-aware discrepancy that privileges low-dimensional structure over ambient-distance fidelity (Mak et al., 2017, Mak et al., 12 Jul 2025).

6. Scope, neighboring concepts, and terminological boundaries

The term “projected support points” is used in the high-dimensional data-reduction and design-of-experiments sense described above, but several nearby literatures use “support points” or “projection” differently.

Context	Object	Meaning of projection
High-dimensional data reduction	PSPs with SpIn kernel	Preservation of low-dimensional coordinate features (Mak et al., 2017)
GP experimental design	$\mathbb E\|Y-Y'\|_2$4-projected support points	Kernel discrepancy over coordinate subspaces (Mak et al., 12 Jul 2025)
Low-rank GP prediction	Support points as knots	Projection of a GP onto a knot-induced subspace, not formal PSPs (2207.12804)

In large-scale low-rank GP prediction, support points are used as knots for predictive processes, and the latent GP is projected onto the span of basis functions induced by those knots. The paper is explicit that this is not a projected-support-points method in the design-of-experiments sense; the “projection” there is the predictive-process projection

$\mathbb E\|Y-Y'\|_2$5

not a projection-aware discrepancy over low-dimensional coordinate subspaces (2207.12804).

Likewise, imbalanced-classification undersampling via support points uses classical energy-distance support points in the original feature space, together with clustering and nearest-neighbor mapping back to observed data. The paper states directly that it does not use projected support points, low-dimensional projection objectives, sliced criteria, or projection-robust representative-point selection (Mak et al., 2024). In discrete Wasserstein barycenters, “support points” are weighted averages of tuples of source support points, and the paper emphasizes that these are not literal projections; they are tuple-induced barycenter points discovered by a pricing MIP (Borgwardt et al., 2022). In projected and incremental pseudo-Boolean model counting, “projection” refers to counting satisfying assignments modulo a projection variable set $\mathbb E\|Y-Y'\|_2$6, not to representative-point design (Yang et al., 2024). In quantum information geometry, “support projection” refers to spectral projection of a pulled-back metric tensor onto its active support, again unrelated to design-theoretic support points (Cho et al., 18 Sep 2025).

Older and adjacent complex-analysis papers also use “support point” in a completely different extremal-functional sense. For compact families of normalized univalent mappings, a support point is an extremizer of a continuous linear functional on a function space, not a representative finite point set for a probability distribution (Bracci et al., 2016, Peretz, 2018, Hamada et al., 2020). These works are relevant to the abstract geometry of support points, to slicing, and to finite-dimensional restrictions, but not to PSPs as used in data reduction and experimental design.

Taken together, the literature supports a precise usage. Projected support points are representative designs or reduced datasets obtained by minimizing a projection-aware discrepancy, typically through a kernel that encodes low-dimensional coordinate structure. Ordinary support points minimize full-space energy distance (Mak et al., 2016). PSPs modify that logic so that low-dimensional projections, sparse active variables, and low-order interactions are preserved explicitly rather than only indirectly (Mak et al., 2017, Mak et al., 12 Jul 2025).