Volume Sampling Distribution

• Volume sampling distribution is defined as a probabilistic mechanism that selects subsets with probability proportional to the squared volume, promoting diversity and robust geometric coverage.
• It underpins applications like matrix approximation, regression, and experimental design by ensuring unbiased estimates and leveraging negative dependence properties.
• Advanced algorithmic strategies, including polynomial time methods and determinantal rejection sampling, enable efficient sampling in high-dimensional and continuous settings.

A volume sampling distribution is a probabilistic mechanism that selects sets, tuples, or locations in geometric, algebraic, or analytic spaces with probability proportional to the (squared) volume spanned by those selected elements. This class of distributions arises in randomized algorithms for matrix approximation, regression, experimental design, optimization, and computational geometry. Sampling "by volume"—whether in discrete, continuous, or manifold settings—exploits the geometry of the underlying space to achieve diversity, unbiasedness, and strong concentration or approximation properties. The mathematical notion typically centers on the determinant of submatrices or kernel matrices and is closely linked to determinantal point processes (DPPs), negative dependence, and optimal design.

1. Foundational Principles: Definition and Geometric Interpretation

At the core of volume sampling is the principle of biasing selection towards maximally "spread out" or "diverse" configurations. In the case of linear spaces, this is formalized as follows: given a data matrix $X \in \mathbb{R}^{n \times d}$, sampling a subset $S$ of $k$ rows (with $k \leq d$) with probability proportional to $\det(X_S^\top X_S)$ corresponds to the squared $k$-dimensional volume of the parallelotope spanned by the chosen rows. For columns, the dual volume sampling distribution uses $\det(A_S A_S^\top)$ for rectangular $A$ with $n \leq k \leq m$ (Li et al., 2017).

For continuous spaces, such as in kernel interpolation, nodes $(x_1, ..., x_N) \in \mathcal{X}^N$ are drawn with density proportional to $\det K(x)$, where $K(x)$ is the kernel matrix evaluated at these points (Belhadji et al., 2020). In geometric measure theory and manifold sampling, the Cauchy–Crofton formula shows that sampling points via the intersection of random lines with a surface yields a point cloud whose empirical density matches the area measure—thereby realizing a "uniform volume sampling" (Palais et al., 2016).

Volume sampling thus generalizes:

• Discrete settings: Matrix row/column selection, least squares design.
• Continuous or manifold settings: Uniform sampling with respect to volume/area for integration or simulation.
• Kernel and function spaces: Node selection for kernel interpolation/quadrature based only on kernel evaluations.

2. Mathematical Formulations and Distribution Properties

The probability measure assigned by volume sampling is explicitly determinant-based:

• In discrete fixed-design (matrix) settings:

$P(S) \propto \det(X_S^\top X_S)$

where $S$ is a size-$k$ subset of container indices, and $X_S$ is the restriction of $X$ to those indices (Dereziński et al., 2018, Dereziński et al., 2018, Dereziński et al., 2017).

• In the "dual" setting:

$P(S) \propto \det(A_S A_S^\top)$

for $n \leq k \leq m$ (columns from an $n \times m$ matrix) (Li et al., 2017).

• In continuous or kernelized volume sampling:

$f_\mathrm{VS}(x_1,\dots,x_N) \propto \det[K(x)]$

with $K(x)$ the kernel Gram matrix (Belhadji et al., 2020).

Volume-rescaled sampling extends the discrete model to the random design case, yielding joint laws on $k$-tuples that are proportional to $\det(\sum_{i=1}^k x_i x_i^\top)$ for vectors $x_i$ drawn from a distribution $D$ (Dereziński et al., 2018).

Key probabilistic properties:

• Negative dependence: Volume sampling distributions are often "strongly Rayleigh," exhibiting negative dependence and yielding powerful concentration results (Li et al., 2017).
• Unbiasedness: For least squares regression, the estimator computed from a volume-sampled subset is unbiased for the population solution, both in fixed and random design settings (Dereziński et al., 2018, Dereziński et al., 2018).
• Diversity promotion: The determinant penalizes near-linear dependence among the selected elements, ensuring near-optimal coverage or geometric spread.

3. Algorithmic Strategies and Computational Complexity

Exact sampling from the volume sampling distribution is challenging due to the combinatorial explosion in the number of subsets or node configurations, but several algorithmic advances have made it tractable in applied contexts:

• Polynomial Time Algorithms: For dual volume sampling (DVS) and regularized variants, polynomial time algorithms exist based on marginal computation and conditional sampling (Li et al., 2017, Dereziński et al., 2017).
• Efficient Iterative Updates: Backward elimination with fast rejection sampling exploits randomized selection and determinant updates for runtime improvement—e.g., FastRegVol achieves $O((n+d)d^2)$ for regularized volume sampling (Dereziński et al., 2017).
• Determinantal Rejection Sampling: Leveraged volume sampling incorporates rescaling by leverage scores and efficiently samples using determinantal rejection (Dereziński et al., 2018).
• Approximate and MCMC Approaches: For kernel and continuous settings, Markov chain Monte Carlo (MCMC) sampling is facilitated by the property that the density function is simply the determinant of a kernel Gram matrix (Belhadji et al., 2020).
• Projection DPPs and Blockwise Sampling: Volume sampling can be realized as sampling from projection DPPs, especially in function approximation and continuous settings (Nouy et al., 2023).

Complexity results are often parameterized in terms of the data dimension $d$, sample size $k$, or problem-specific parameters:

• For Gaussian volume estimation, $O^*(n^3)$ time for integration and $O^*(n^3)$ for the first sample ($O^*(n^2)$ for each thereafter), an improvement of a factor $n$ over previous bounds, exploiting better isoperimetry, smoother annealing, and the speedy walk technique (Cousins et al., 2013, Cousins et al., 2014).
• For dual volume sampling of an $n \times m$ matrix, the randomized algorithm for drawing a size-$k$ sample runs in $O(k m^4)$ (Li et al., 2017).

4. Statistical and Optimization Applications

Volume sampling and its generalizations have substantial impact in statistics, optimization, and machine learning due to their connections with:

• Experimental Design: Selecting rows/columns by volume maximizes the determinant of the design or information matrix, yielding optimality in linear models and control of mean squared prediction error (MSPE), often in terms proportional to the "statistical dimension" $d_\lambda$ (Dereziński et al., 2017).
• Subset Selection: For column/row subset selection, algorithms based on volume sampling achieve best-possible $(k+1)$-factor approximation in spectral error, and volume-rescaled sampling yields near-optimal low-rank approximations (Epperly, 2 Oct 2025).
• Active Learning for Regression: Unbiased estimates for the population least squares solution can be constructed by augmenting arbitrary i.i.d. samples with a small volume-rescaled sample (Dereziński et al., 2018).
• Randomized Coordinate Descent and Block Kaczmarz: Subsets of variables or blocks are selected proportional to the determinant of principal submatrices (volume sampling), yielding provably accelerated convergence rates when variable correlations are pronounced (Rodomanov et al., 2019, Xiang et al., 18 Mar 2025).
• Kernel Quadrature and Interpolation: Continuous volume sampling achieves nearly optimal error rates for kernel quadrature and interpolation, depending only on the spectrum of the integral operator, applicable for arbitrary Mercer kernels; see error bounds such as $\epsilon_1 \leq \sigma_N (1 + \beta_N)$ (Belhadji et al., 2020).
• Weighted Least Squares Function Approximation: Volume-rescaled sampling and projection DPPs provide quasi-optimal weights and sample selection, maintaining Gram matrix conditioning and near-optimal $L^2$ or RKHS error, often with $n = O(m \log m)$ samples for a target $m$-dimensional subspace (Nouy et al., 2023).

5. Role in Computational Geometry and Manifold Sampling

In geometric contexts, the volume sampling distribution shapes algorithms for random polytope volume approximation, convex hull estimation, and equidistributed point cloud construction:

• Convex Body Volume and Isotropic Transformation: Annealing (Gaussian cooling), better isoperimetry, and walk-based sampling schemes exploit volume sampling to enable efficient, high-dimensional integration and sampling from convex polytopes (Cousins et al., 2013, Cousins et al., 2014, Mangoubi et al., 2019, Chalkis et al., 2020).
• Point Cloud Generation on Manifolds: Via the Cauchy–Crofton formula, sampling is performed by analyzing the measure induced by random lines or $k$-planes intersecting the surface, thus achieving uniform measure with respect to area or (more generally) volume elements (Palais et al., 2016).
• Random Polytope Volume Thresholds: For log-concave measures, exponentially many samples (in dimension) are required for the convex hull of random samples to achieve significant volume; for the Euclidean ball, the threshold is superexponential, illustrating "thinness" or inefficiency of random sampling for high-dimensional convex hull volume estimation (Chakraborti et al., 2020).

6. Extensions, Generalizations, and Modern Developments

The concept of volume sampling has been extended in numerous directions:

• Regularized Volume Sampling: Incorporating a ridge term $(X_S X_S^\top + \lambda I)$ permits sampling robustly below full-rank and controls estimator variance (Dereziński et al., 2017).
• Leveraged Volume Sampling: Sampling is performed jointly with scaling by leverage scores, correcting deficiencies in the tail behavior of standard volume sampling for large $k$ (Dereziński et al., 2018).
• Continuous and Manifold Volume Sampling: Continuous analogues select tuples of points on domains or manifolds with density given by the determinant of the kernel matrix, and projection DPPs enforce diversity in point clouds and function approximation (Belhadji et al., 2020, Nouy et al., 2023).
• Volume-Rescaled Augmentation: Correcting the bias of i.i.d. least squares regression solutions by augmenting with a small joint sample according to the volume-rescaled law (Dereziński et al., 2018).
• Fast Algorithmic Implementations: Efficient implementations exploit cumulative quantities, leverage score–based proposal distributions, and binary search techniques, dramatically reducing computational cost for large sparse matrices or for block selection (Xiang et al., 18 Mar 2025, Epperly, 2 Oct 2025).

7. Connections to Negative Dependence and Repulsive Processes

Volume sampling distributions are intimately linked to determinantal point processes (DPPs) and negative dependence structures:

• Negative Association and Concentration: The strong Rayleigh property, established for dual volume sampling and projection DPPs, yields negative dependence and therefore strong concentration of measure and anti-correlation among sampled points or features (Li et al., 2017, Nouy et al., 2023).
• Repulsion and Diversity: In both discrete and continuous settings, the determinant factor in the distribution penalizes clustering or collinearity, encoding geometric diversity among the samples. This feature underpins improved conditioning of Gram matrices and approximation guarantees in least-squares and kernel-based learning.

---

Volume sampling distributions thus provide a unified, geometrically principled framework for subset selection, diversity enforcement, and robust statistical approximation across discrete, continuous, and manifold settings. Their algorithmic tractability, optimality properties, and deep connections to convex geometry, determinantal processes, and randomized numerical linear algebra have established volume sampling as a foundational tool in contemporary high-dimensional computation and inference.