Stochastic Sparse Sampling

• Stochastic Sparse Sampling is a design principle that harnesses sparse representations and randomized sampling to reduce measurements and computation.
• It underpins methodologies in adaptive acquisition, medical time series classification, constrained Bayesian inference, and sparse bandits to optimize performance.
• The approach strategically allocates resources where they matter most, using techniques like thermal noise and random subsampling to enhance robustness and accuracy.

In the cited literature, stochastic sparse sampling does not denote a single standardized algorithm. It appears both as the proper name of a variable-length medical time series classification framework and as a broader descriptor for methods that combine sparsity structure with randomized, probabilistic, or sampling-based operations in order to reduce measurements, computation, or regret. Across these usages, the sparse object may be a signal, a simplex-valued latent variable, a linear bandit parameter, a matrix or graph, a topic model, a PDE expansion, or a CNN feature map, while the stochastic component may come from thermal fluctuations, minibatch noise, posterior sampling, random masks, or random input evaluation (Mootoo et al., 2024, Salehi et al., 2019, Baker et al., 2018, Hao et al., 2021).

1. Scope and recurring design

A recurring structure is visible across the literature. First, the target object is assumed to be sparse, compressible, low-rank, or locally informative only on a small subset of coordinates, windows, edges, coefficients, or spatial locations. Second, a stochastic mechanism chooses where to spend sensing, inference, or computation. Third, a recovery, aggregation, or decision rule reconstructs a dense estimate, a global prediction, or an action policy from the sparse stochastic representation. This suggests an abstract template rather than a single method family in the narrow algorithmic sense.

Domain	Sparse object	Stochastic mechanism
Adaptive acquisition	Sparse signal representation, typically $5\%$ to $15\%$ nonzero coefficients	MRAM-based stochastic oscillator controlled by sparsity estimate (Salehi et al., 2019)
Medical VTSC	Fixed windows from long variable-length recordings	Length-proportional window sampling and probability-space aggregation (Mootoo et al., 2024, Javadov et al., 3 Oct 2025)
Constrained Bayesian inference	Simplex coordinates near zero; $\ell_1$-regularized unknowns	Exact CIR transitions; Hadamard-parameterized Langevin dynamics (Baker et al., 2018, Cheltsov et al., 2024)
Sparse bandits	Sparse linear parameters in high dimension	Sparse posterior sampling, IDS, optimistic posterior sampling (Hao et al., 2021, Schwartz et al., 28 Oct 2025, Chakraborty et al., 2022)
Large-scale approximation	Rows, edges, topic assignments, PDE coefficients, feature-map locations	Row subsampling, edge sampling, Gibbs-in-minibatches, random PC sampling, Gumbel-Softmax masks (Mensch et al., 2017, Rohe et al., 2017, Mimno et al., 2012, Doostan et al., 2010, Xie et al., 2020)

One common misconception is that sparse sampling is necessarily deterministic thresholding or uniform subsampling. The cited work shows otherwise: stochasticity may be introduced specifically to improve robustness, reduce bias, or make constrained inference tractable. Another misconception is that the randomness must be algorithmic pseudorandomness. In hardware acquisition, it can arise from physical stochastic spin dynamics in a low-barrier nanomagnet (Salehi et al., 2019).

2. Adaptive acquisition and sparse signal sampling

In compressive sensing hardware, stochastic sparse sampling is instantiated by AQuRate, a hardware mechanism for adaptive, non-uniform sampling of sparse signals. The architecture estimates the sparsity rate of the previous frame, converts that estimate to a control voltage $V_{SR}$, uses $V_{SR}$ to tune an MRAM-based stochastic oscillator, and then converts the oscillator output into an asynchronous non-uniform sampling clock for the next frame. The target setting is sparse signals whose sparse representation has only a small fraction of nonzero coefficients, typically in the range $5\%$ to $15\%$ (Salehi et al., 2019).

The oscillator is a 2-terminal magnetic tunnel junction with a low energy barrier nanomagnet satisfying

$E_B \ll 40kT.$

Because the free layer is thermally unstable, the magnetization $m_z$ fluctuates stochastically, and the conductance obeys

$$G_{MTJ} = G_0 \left[ 1 + m_z \frac{TMR}{(2+TMR)}\right].$$

The drain voltage of the NMOS/MTJ stage is

$15\%$0

Maximum fluctuations occur when $15\%$1, in practice near $15\%$2. A CMOS inverter amplifies the noisy drain node into a full-swing stochastic bitstream, a D flip-flop samples that bitstream, and a NAND gate combines it with the system clock to generate the asynchronous clock $15\%$3-Clk (Salehi et al., 2019).

This mechanism is sparse because the effective sampling rate is matched to the estimated sparsity rather than fixed by Nyquist-style uniform sampling. It is stochastic because the clock source is driven by thermal fluctuations, not merely by an LFSR-like digital pseudorandom circuit. Relative to prior non-uniform clock generators based on CMOS LFSR + ring oscillator structures and VCMA-MTJ-based approaches, the AQR generator reports ~25-fold reduction in area, ~6-fold reduced power dissipation, requires only 23 FinFET transistors, and has simulated average power dissipation

$15\%$4

Embedded in OMP and CoSaMP recovery, the reported mean normalized reconstruction errors are $15\%$5 for OMP and $15\%$6 for CoSaMP at sparsity levels $15\%$7, respectively (Salehi et al., 2019).

3. Variable-length medical time series

As a named framework, Stochastic Sparse Sampling (SSS) was introduced for variable-length time series classification in medical settings, especially seizure onset zone localization from intracranial EEG. The central idea is to decompose each sequence into fixed-length windows, sample windows stochastically during training, compute local predictions with a fixed-length backbone, and aggregate those local predictions into a series-level probability vector. For a series of length $15\%$8, window length $15\%$9, and stride $\ell_1$0, the number of extracted windows is

$\ell_1$1

and the training-time sampling probability is

$\ell_1$2

If a batch has size $\ell_1$3, then $\ell_1$4 and $\ell_1$5 (Mootoo et al., 2024).

For each sampled window $\ell_1$6, the backbone produces

$\ell_1$7

Series-level prediction is obtained by convex aggregation,

$\ell_1$8

The reported experiments use mean aggregation. At inference time, SSS applies calibration before aggregation, using isotonic regression or Venn-Abers predictors. The calibration ablation reports F1/AUC/accuracy of $\ell_1$9 for isotonic regression, $V_{SR}$0 for Venn-Abers, and $V_{SR}$1 without calibration. On the Epilepsy iEEG Multicenter Dataset, SSS is reported to outperform modern finite- and infinite-context baselines overall and to outperform all baselines on every unseen medical center in the out-of-distribution evaluation (Mootoo et al., 2024).

RAxSS generalizes this framework by replacing uniform aggregation with retrieval-informed weighting. For each window $V_{SR}$2, it retrieves the top-$V_{SR}$3 most similar nonidentical windows from the same series using Pearson or cosine similarity,

$V_{SR}$4

and forms the summary support

$V_{SR}$5

Window weights are then

$V_{SR}$6

and the series-level prediction is the convex combination

$V_{SR}$7

The derivative

$V_{SR}$8

makes the influence of retrieved neighbors explicit. RAxSS reports an evidence trail consisting of a window’s mean support, its ranked neighbor list, and its resulting weight. On the multicenter iEEG evaluation, RAxSS (cosine) reports AUC $V_{SR}$9, RAxSS (Pearson) reports F1 $V_{SR}$0 and accuracy $V_{SR}$1, and the SSS reproduction reports F1 $V_{SR}$2, AUC $V_{SR}$3, accuracy $V_{SR}$4 (Javadov et al., 3 Oct 2025).

4. Constrained Bayesian inference and sparse priors

In Bayesian computation, stochastic sparse sampling often addresses constrained or sparsity-inducing latent spaces. On the probability simplex, the key difficulty is that standard SGMCMC methods inherit an $V_{SR}$5 discretization bias, which becomes especially damaging near the boundary where many coordinates are close to zero. The SCIR method resolves this by reparameterizing Dirichlet variables through gamma variables and then using the exact transition law of the Cox–Ingersoll–Ross process,

$V_{SR}$6

With minibatch estimator

$V_{SR}$7

SCIR applies the exact CIR transition with $V_{SR}$8 plugged in. Because the transition is exact, the method removes time discretization error, and the paper proves asymptotic unbiasedness: $V_{SR}$9 It also derives explicit variance inflation from minibatch noise. Empirically, SCIR is reported to perform substantially better than existing SGMCMC approaches on a sparse simplex experiment, a topic model, and a Dirichlet process mixture model (Baker et al., 2018).

A different construction appears in sampling with $5\%$0-priors. Hadamard Langevin dynamics starts from the observation that the Laplace prior is non-smooth and that proximal Langevin methods introduce Moreau-envelope bias. The paper introduces latent variables $5\%$1 and the parameterization

$5\%$2

An expanded smooth target density $5\%$3 is constructed so that the pushforward through $5\%$4 recovers the original sparse prior exactly. Standard overdamped Langevin dynamics is then run in the augmented space,

$5\%$5

with Euler–Maruyama discretization

$5\%$6

The continuous-time process is proved well posed and geometrically ergodic, and the discretization is proved convergent as $5\%$7. The conceptual distinction from proximal approaches is exactness by overparameterization rather than smoothing of the original $5\%$8 potential (Cheltsov et al., 2024).

5. Sparse posterior sampling and information-directed decision making

In stochastic sparse linear bandits, stochastic sparse sampling refers to action selection driven by sampling from sparse posterior or optimistic-posterior distributions. Information Directed Sampling for Sparse Linear Bandits studies

$5\%$9

and shows that optimism can be suboptimal because informative actions may be temporarily low reward. The proposed sparse IDS combines an information-ratio policy with an empirical Bayes sparse posterior sampler based on a spike-and-slab Gaussian-Laplace prior. The Bayesian regret bound is

$15\%$0

with

$15\%$1

which nearly matches lower bounds in both data-rich and data-poor regimes (Hao et al., 2021).

Sparse Optimistic Information-Directed Sampling (SOIDS) removes Bayesian assumptions and works in the frequentist sparse linear bandit model

$15\%$2

Its optimistic posterior is

$15\%$3

and the policy minimizes a surrogate 2-information ratio while inheriting control of higher-order information ratios. Under the sparse optimal action condition, the regret is summarized as

$15\%$4

with the appropriate $15\%$5-dependence in the data-poor regime. The paper presents this as the first frequentist algorithm that simultaneously achieves optimal worst-case regret in both regimes (Schwartz et al., 28 Oct 2025).

A related posterior-sampling line is Thompson Sampling for High-Dimensional Sparse Linear Contextual Bandits, where the unknown parameter satisfies $15\%$6. The prior samples a support $15\%$7, sets $15\%$8, and places a product Laplace slab on active coordinates. To avoid MCMC cost, the posterior is approximated by mean-field variational Bayes,

$15\%$9

The paper proves posterior contraction at a sparse-regression rate and a regret bound whose dependence on ambient dimension is only logarithmic or polylogarithmic, with the leading dependence on $E_B \ll 40kT.$0 rather than $E_B \ll 40kT.$1 (Chakraborty et al., 2022).

6. Related usages in large-scale computation and approximation

In large-scale graph simulation, fastRG samples a sparse random matrix with low-rank expectation

$E_B \ll 40kT.$2

Instead of element-wise $E_B \ll 40kT.$3 sampling, it first draws the total number of edges $E_B \ll 40kT.$4, then samples latent block pairs from $E_B \ll 40kT.$5 and endpoints from the corresponding columns of $E_B \ll 40kT.$6 and $E_B \ll 40kT.$7. In sparse graphs with $E_B \ll 40kT.$8, ignoring log terms, the runtime is $E_B \ll 40kT.$9. The paper reports simulations up to $m_z$0 nodes with $m_z$1 edges, and on $m_z$2, $m_z$3, fastRG runs in less than one second on a 3.5 GHz Intel i5 (Rohe et al., 2017).

In matrix factorization, subsampled online matrix factorization (somf) streams columns but randomly subsamples rows using a diagonal mask $m_z$4, keeping only a fraction $m_z$5 of coordinates so that

$m_z$6

Only the sampled rows are used to approximate code computation, update surrogate statistics, and update dictionary rows. The dominant $m_z$7-dependent costs of online matrix factorization are reduced to $m_z$8, and the method is proved to converge to a stationary point of the original empirical risk. Reported times to reach within $m_z$9 of convergence include 6 min $$G_{MTJ} = G_0 \left[ 1 + m_z \frac{TMR}{(2+TMR)}\right].$$0 28 s on ADHD200 and 3 h 50 $$G_{MTJ} = G_0 \left[ 1 + m_z \frac{TMR}{(2+TMR)}\right].$$1 17 min on HCP, corresponding to speed-ups $$G_{MTJ} = G_0 \left[ 1 + m_z \frac{TMR}{(2+TMR)}\right].$$2 and $$G_{MTJ} = G_0 \left[ 1 + m_z \frac{TMR}{(2+TMR)}\right].$$3, respectively (Mensch et al., 2017).

In computer vision, spatially adaptive inference with stochastic feature sampling and interpolation computes CNN features only at sparsely sampled spatial locations, chosen probabilistically from predicted activation responses via a Gumbel-Softmax mask,

$$G_{MTJ} = G_0 \left[ 1 + m_z \frac{TMR}{(2+TMR)}\right].$$4

Missing locations are reconstructed by interpolation, with the learned RBF kernel reported as the best overall in FLOPs/accuracy tradeoff. On COCO object detection the baseline is 43.4 mAP, 289.5 GFLOPs, while the method reports 43.3 mAP, 160.4 GFLOPs at sparse loss weight $$G_{MTJ} = G_0 \left[ 1 + m_z \frac{TMR}{(2+TMR)}\right].$$5, and on Cityscapes semantic segmentation the baseline is 80.8 mIoU, 920.6 GFLOPs while the method reports 80.6 mIoU, 373.2 GFLOPs at sparse loss weight $$G_{MTJ} = G_0 \left[ 1 + m_z \frac{TMR}{(2+TMR)}\right].$$6 (Xie et al., 2020).

In Bayesian latent-variable models, Sparse Stochastic Inference for LDA combines collapsed Gibbs sampling for document-level latent assignments with online stochastic variational inference for global topic-word parameters. The variational family keeps $$G_{MTJ} = G_0 \left[ 1 + m_z \frac{TMR}{(2+TMR)}\right].$$7 structured rather than token-factorized, and minibatch-based natural-gradient updates take the form

$$G_{MTJ} = G_0 \left[ 1 + m_z \frac{TMR}{(2+TMR)}\right].$$8

The method was used on 1.2 million books (33 billion words) with thousands of topics, and at $$G_{MTJ} = G_0 \left[ 1 + m_z \frac{TMR}{(2+TMR)}\right].$$9 the paper reports that less than 1% of the $15\%$00 topic-word parameters were nonzero (Mimno et al., 2012).

In stochastic PDE approximation, the phrase denotes a non-intrusive sparse-compressive sampling strategy: draw i.i.d. random inputs $15\%$01, evaluate the deterministic solver, build a measurement matrix $15\%$02 from polynomial chaos basis functions, and recover sparse coefficients by BPDN or OMP. The method converges in probability with probabilistic error bounds, relying on sparsity of the PC representation and concentration of measure for empirical correlations (Doostan et al., 2010). In sparse regression with limited attribute observation, the learner observes only $15\%$03 coordinates of each example, constructs an unbiased gradient estimator from the current support and sampled coordinate blocks, and updates by iterative hard thresholding,

$15\%$04

The exploration method achieves essentially $15\%$05 sample complexity, and the Hybrid algorithm can reach $15\%$06 when the minimum nonzero coefficient is not too small (Murata et al., 2018).

Taken together, these works show that stochastic sparse sampling is best understood as a design principle: exploit a sparse latent structure, randomize where information is acquired or computation is spent, and then use model-based recovery, aggregation, or decision rules to preserve global performance. The principle is realized very differently in hardware clocks, medical VTSC, constrained MCMC, sparse bandits, graph simulation, matrix factorization, topic models, PDE approximation, and CNN acceleration, but the underlying economy is the same: computation is concentrated on a sparse subset selected stochastically rather than processed densely everywhere.