Adaptive Random Sampling

Updated 1 April 2026

Adaptive random sampling is a dynamic method that adjusts sample selection based on observed data to reduce estimator variance and minimize sample complexity.
It employs strategies such as adaptive Metropolis sampling, importance sampling, and reinforcement learning to target informative regions and improve computational efficiency.
The approach offers theoretical guarantees and practical benefits, including error reduction in high-dimensional approximations and efficient resource allocation in risk-averse optimization.

Adaptive random sampling refers to algorithms and strategies that dynamically steer the selection of random samples or features in response to observed data, evolving model parameters, or theoretical criteria. This paradigm contrasts with “oblivious” or fixed random sampling, in which the sampling rule is independent of the adaptively acquired information or error signal. Adaptive random sampling is central to high-dimensional approximation, numerical linear algebra, compressed sensing, statistical inference, and stochastic optimization, enabling more efficient allocation of resources and superior statistical or computational guarantees relative to nonadaptive methods.

1. Core Principles and Motivations

Adaptive random sampling exploits real-time feedback to concentrate sampling effort where it is most informative or most likely to reduce error. The canonical motivations are:

Variance Reduction: By biasing sample allocations or selection densities based on estimated gradients, response values, or tails of the input distribution, the method can sharply reduce estimator variance (Pieraccini et al., 14 Feb 2025, Zhu, 2018).
Sample Complexity Minimization: Adaptively targeting “difficult” regions (e.g., high-frequency bands in Fourier expansions, risk regions in CVaR minimization) often achieves a desired error using far fewer samples than uniform or fixed-density schemes (Kammonen et al., 2020, Pieraccini et al., 14 Feb 2025, Adcock et al., 18 Mar 2026).
Data-Efficient Exploration: By learning or adjusting where to sample (spatially, in feature space, or in terms of data batches), adaptive random sampling efficiently balances exploration and exploitation (Gautier et al., 2021, Dou et al., 2022).
Online Estimation and Streaming: In streaming or massive-data environments, the adaptivity allows continual refinement of sampling distributions or cache contents, improving estimation throughput and accuracy (Drescher et al., 2018).

2. Methodological Frameworks

Adaptive random sampling encompasses a variety of concrete algorithmic instantiations:

A. Adaptive Metropolis Sampling for Random Features

In kernel methods and neural approximations, adaptive random Fourier features with Metropolis sampling iteratively update Fourier frequencies $\omega_k$ according to a coordinate-wise Metropolis-Hastings scheme. The acceptance probability depends on the ratio of corresponding amplitude magnitudes raised to a calibrated exponent $\gamma$ , which is set by a work-error trade-off and scales as $\gamma \approx 3d-2$ , where $d$ is the data dimension (Kammonen et al., 2020).

$\gamma \approx 3d-2$ 0

This adapts the distribution of frequencies, driving the empirical marginal distribution toward the theoretical optimal that matches the modulus of the target Fourier transform (Kammonen et al., 2020).

B. Adaptive Importance and Tail-Focused Sampling

For risk-averse objectives such as Conditional Value-at-Risk (CVaR), adaptive random sampling modifies both the sample size and the sampling density per iteration. Importance weights are constructed to bias sampling toward "risk regions" (e.g., input configurations where rare but high losses occur). Sample size adaptation is computed by enforcing a variance-control condition, and the sampling density is updated using a reduced-order surrogate to efficiently characterize the risk region (Pieraccini et al., 14 Feb 2025).

C. Reinforcement Learning and Active Policy Selection

In high-dimensional or sequential tasks (e.g., representation learning, compressed sensing, or exploration in reinforcement learning), sampling actions are chosen according to a learned policy. The policy is trained to maximize cumulative reward, which is a proxy for downstream metrics such as information gain or task-specific performance. For example, Adaptive Sample with Reward (ASR) in metric learning frames the sampling process as an MDP and optimizes sample (triplet) selection using Proximal Policy Optimization (PPO) (Dou et al., 2022), while in sparse signal recovery, PPO learns a sampling policy that, in conjunction with a variational autoencoder prior, targets pixels or measurements that improve image reconstruction quality (Rasheed et al., 3 Dec 2025).

D. Adaptive Randomized Pivoting for Matrix Subset Selection

In randomized linear algebra, adaptive randomized pivoting (ARP) solves the column (or row) subset selection problem by iteratively sampling indices with adaptive leverage scores that are updated at each step to reflect information already contained in the subset. This results in a sampling law that is equivalent to volume sampling, delivering near-optimal error guarantees (Cortinovis et al., 2024, Epperly, 2 Oct 2025). The following formula dictates the probability at step $k$ :

$\gamma \approx 3d-2$ 1

where $\mathbf{V}_{k-1}$ is the matrix with the $j$ -th row at stage $k-1$ , and $r$ is the subset size.

3. Theoretical Guarantees and Asymptotics

Adaptive random sampling methods are frequently equipped with nonasymptotic and asymptotic optimality guarantees.

Equidistribution and Optimality in Random Features: Metropolis adaptation drives feature amplitudes toward equidistribution in the rescaled variable $K p(\omega) \hat\beta_k$ , producing an empirical measure that converges (in the $\gamma$ 0 limit) to the optimal sampling density for independent features. This density minimizes the mean-square approximation error, yielding orders-of-magnitude improvement over nonadaptive methods in high-dimensional problems (Kammonen et al., 2020).
Optimal Error Bounds for Matrix Approximation: Adaptive randomized pivoting provides error bounds in expectation that match the fundamental “existence” bounds for column subset selection, i.e.,

$\gamma \approx 3d-2$ 2

where $\gamma$ 1 is the input matrix, $\gamma$ 2 the projection onto the selected columns, and $\gamma$ 3 the singular values (Cortinovis et al., 2024, Epperly, 2 Oct 2025).

Variance Control in Adaptive Importance Sampling: By dynamically constructing sampling densities focused on risk regions and tuning sample sizes via variance conditions, adaptive algorithms for risk-averse optimization guarantee that gradient variances remain bounded independently of the risk level. The resulting stochastic gradient schemes retain linear convergence rates with a much lower overall sample and computational budget (Pieraccini et al., 14 Feb 2025).
Instance-Optimal Subsampling for Least Squares: Gradient-based adaptive sampling in large-scale least-squares regression provably achieves $\gamma$ 4 convergence rates with smaller multiplicative constants than leverage-score or uniform random sampling (Zhu, 2018).

4. Representative Applications and Algorithmic Variants

A spectrum of domains has motivated tailored adaptive random sampling schemes.

Domain	Key Adaptive Feature	Reference
Random Fourier Feature Approximation	Metropolis adaptation, $\gamma$ 5-scaling	(Kammonen et al., 2020)
Risk-Averse Optimization (CVaR)	ROM-based IS, tail focusing, norm-test	(Pieraccini et al., 14 Feb 2025)
Matrix Subset Selection	Adaptive leverage scoring, volume law	(Cortinovis et al., 2024, Epperly, 2 Oct 2025)
Bayesian MCMC Inference	Copula-based, mixture adaptation	(Silva et al., 2010)
Compressed Sensing/Image Recovery	Patch-wise texture/gradient analysis	(Taimori et al., 2017)
Sparse Random Feature Expansion	Christoffel function-driven active sampling	(Adcock et al., 18 Mar 2026)
Deep Embedding/Representation Learning	RL-based selection (ASR/PPO)	(Dou et al., 2022, Rasheed et al., 3 Dec 2025)

Specialized variants include:

Copula- and Clustering-Based Proposals: In Bayesian inference, adaptive independent proposals with t-copula structure, optionally with antithetic coupling, achieve higher acceptance and lower inefficiency than adaptive random walk proposals (Silva et al., 2010).
Patchwise Space-Frequency-Gradient Analysis: In adaptive image sampling, each patch’s sampling quota is determined by local texture, DCT sparsity, and edge orientation, integrating uniform, random, and nonuniform schemes, followed by cellular-automaton recovery (Taimori et al., 2017).
Christoffel-Driven Sample Allocation: For sparse random feature expansions, the Christoffel function for the current approximation subspace is used to compute the next batch’s sampling density, ensuring that high leverage regions are adequately covered and stable least-squares solutions are obtained with minimal oversampling (Adcock et al., 18 Mar 2026).

5. Computational Complexity and Practical Efficiency

Adaptive random sampling algorithms often match or outperform static random or greedy algorithms, both in statistical metrics (estimation error, variance, generalization) and computational efficiency (runtime, sample size).

The complexity per adaptation is typically dominated by the cost of recomputing scores, feature amplitudes, or subspace projections. For instance, adaptive random Fourier feature Metropolis updates scale as $\gamma$ 6 per step, but achieve much lower total error for a given computational budget than nonadaptive schemes (Kammonen et al., 2020).
In subset selection, adaptive randomized pivoting post-sketching requires only $\gamma$ 7 arithmetic, a significant saving over $\gamma$ 8 for exact volume sampling, with state-of-the-art accuracy (Cortinovis et al., 2024).
For high-dimensional convex optimization, adaptive sketching transforms (using correlated random subspaces) yield much sharper error decay in solution approximation with $\gamma$ 9-sized subspaces aligned to the problem structure, backed by convex-analytic proofs and empirical benchmarks (Lacotte et al., 2019).
In adaptive patchwise image sampling and recovery, the hybrid measurement design plus local CA imputation achieves high-quality reconstruction well below the naive (Gaussian CS) sampling rate at negligible computational cost (Taimori et al., 2017).

6. Extensions and Limitations

Adaptive random sampling continues to be refined for broader classes of problems.

Loss Function Generality: Extensions to non-quadratic and non-convex losses involve incorporating second-order information, or dynamically learning sampling policies via RL or surrogate models (Zhu, 2018, Rasheed et al., 3 Dec 2025).
RL-Driven Sensing: Sequential adaptive sensing via reinforcement learning leverages neural generative priors, enabling content-adaptive selection in high-dimensional signals and images (Rasheed et al., 3 Dec 2025).
Goal-Oriented and Statistical Field Modelling: In systems with response distributions rather than point evaluations, adaptive sampling is guided by field models over space-feature tuples and acquisition functions targeting metrics like median or risk functional reduction (Gautier et al., 2021).
Known Challenges: Computational overhead of repeated score or subspace update, potential policy collapse in RL-driven settings, and requirement for reliable surrogates can limit scalability or flexibility in extreme large-scale settings (Rasheed et al., 3 Dec 2025).

Ongoing research focuses on further theoretical characterizations (instance-wise oracle inequalities, minimax bounds), distributed and multi-agent adaptive sampling, integration into deep iterative pipelines, and efficient surrogate construction for risk and field-based objectives.