--- title: Stochastic Sparse Sampling url: https://www.emergentmind.com/topics/stochastic-sparse-sampling type: topic --- # Stochastic Sparse Sampling In the cited literature, **stochastic sparse sampling** does not denote a single standardized algorithm. It appears both as the proper name of a variable-length medical time series classification framework and as a broader descriptor for methods that combine **sparsity structure** with **randomized, probabilistic, or sampling-based operations** in order to reduce measurements, computation, or regret. Across these usages, the sparse object may be a signal, a simplex-valued latent variable, a linear bandit parameter, a matrix or graph, a topic model, a PDE expansion, or a CNN feature map, while the stochastic component may come from thermal fluctuations, minibatch noise, posterior sampling, random masks, or random input evaluation [2410.06412] [1903.00971] [1806.07137] [2105.14267]. ## 1. Scope and recurring design A recurring structure is visible across the literature. First, the target object is assumed to be sparse, compressible, low-rank, or locally informative only on a small subset of coordinates, windows, edges, coefficients, or spatial locations. Second, a stochastic mechanism chooses where to spend sensing, inference, or computation. Third, a recovery, aggregation, or decision rule reconstructs a dense estimate, a global prediction, or an action policy from the sparse stochastic representation. This suggests an abstract template rather than a single method family in the narrow algorithmic sense. | Domain | Sparse object | Stochastic mechanism | |---|---|---| | Adaptive acquisition | Sparse signal representation, typically \(5\%\) to \(15\%\) nonzero coefficients | MRAM-based stochastic oscillator controlled by sparsity estimate [1903.00971] | | Medical VTSC | Fixed windows from long variable-length recordings | Length-proportional window sampling and probability-space aggregation [2410.06412] [2510.02936] | | Constrained Bayesian inference | Simplex coordinates near zero; \(\ell_1\)-regularized unknowns | Exact CIR transitions; Hadamard-parameterized Langevin dynamics [1806.07137] [2411.11403] | | Sparse bandits | Sparse linear parameters in high dimension | Sparse posterior sampling, IDS, optimistic posterior sampling [2105.14267] [2510.24234] [2211.05964] | | Large-scale approximation | Rows, edges, topic assignments, PDE coefficients, feature-map locations | Row subsampling, edge sampling, Gibbs-in-minibatches, random PC sampling, Gumbel-Softmax masks [1701.05363] [1703.02998] [1206.6425] [1006.2151] [2003.08866] | One common misconception is that sparse sampling is necessarily deterministic thresholding or uniform subsampling. The cited work shows otherwise: stochasticity may be introduced specifically to improve robustness, reduce bias, or make constrained inference tractable. Another misconception is that the randomness must be algorithmic pseudorandomness. In hardware acquisition, it can arise from physical stochastic spin dynamics in a low-barrier nanomagnet [1903.00971]. ## 2. Adaptive acquisition and sparse signal sampling In compressive sensing hardware, stochastic sparse sampling is instantiated by **AQuRate**, a hardware mechanism for adaptive, non-uniform sampling of sparse signals. The architecture estimates the sparsity rate of the previous frame, converts that estimate to a control voltage \(V_{SR}\), uses \(V_{SR}\) to tune an MRAM-based stochastic oscillator, and then converts the oscillator output into an asynchronous non-uniform sampling clock for the next frame. The target setting is sparse signals whose sparse representation has only a small fraction of nonzero coefficients, typically in the range \(5\%\) to \(15\%\) [1903.00971]. The oscillator is a 2-terminal magnetic tunnel junction with a low energy barrier nanomagnet satisfying \[ E_B \ll 40kT. \] Because the free layer is thermally unstable, the magnetization \(m_z\) fluctuates stochastically, and the conductance obeys \[ G_{MTJ} = G_0 \left[ 1 + m_z \frac{TMR}{(2+TMR)}\right]. \] The drain voltage of the NMOS/MTJ stage is \[ V_{DRAIN}/V_{DD} = \frac{( 2+TMR)+ TMR \ m_z}{(2+TMR)(1+\alpha)+TMR \ m_z}, \qquad \alpha = \frac{G_T}{G_0}. \] Maximum fluctuations occur when \(\alpha \approx 1\), in practice near \(V_{IN} = 0.5 V_{DD}\). A CMOS inverter amplifies the noisy drain node into a full-swing stochastic bitstream, a D flip-flop samples that bitstream, and a NAND gate combines it with the system clock to generate the asynchronous clock \(A\)-Clk [1903.00971]. This mechanism is sparse because the effective sampling rate is matched to the estimated sparsity rather than fixed by Nyquist-style uniform sampling. It is stochastic because the clock source is driven by thermal fluctuations, not merely by an LFSR-like digital pseudorandom circuit. Relative to prior non-uniform clock generators based on CMOS LFSR + ring oscillator structures and VCMA-MTJ-based approaches, the AQR generator reports **~25-fold reduction in area**, **~6-fold reduced power dissipation**, requires only **23 FinFET transistors**, and has simulated average power dissipation \[ 22.64 \ \mu W. \] Embedded in OMP and CoSaMP recovery, the reported mean normalized reconstruction errors are \(0.0504, 0.0446, 0.0252\) for OMP and \(0.0487, 0.0304, 0.0245\) for CoSaMP at sparsity levels \(5\%, 10\%, 15\%\), respectively [1903.00971]. ## 3. Variable-length medical time series As a named framework, **Stochastic Sparse Sampling (SSS)** was introduced for variable-length time series classification in medical settings, especially seizure onset zone localization from intracranial EEG. The central idea is to decompose each sequence into fixed-length windows, sample windows stochastically during training, compute local predictions with a fixed-length backbone, and aggregate those local predictions into a series-level probability vector. For a series of length \(T_i\), window length \(L\), and stride \(S\), the number of extracted windows is \[ A_i = \left\lfloor \frac{T_i - L}{S} \right\rfloor + 1, \] and the training-time sampling probability is \[ p_i = \frac{A_i}{\sum_{j=1}^n A_j} \approx \frac{T_i}{\sum_{j=1}^n T_j}. \] If a batch has size \(B\), then \(N_i \sim \mathrm{Binomial}(B,p_i)\) and \(\mathbb{E}[N_i] = Bp_i\) [2410.06412]. For each sampled window \(\bw_b\), the backbone produces \[ \hat{y}_b = f_{\theta}(\bw_b) \in [0,1]^K. \] Series-level prediction is obtained by convex aggregation, \[ \hat{y}^{(i)} = \sum_{\hat{y} \in \mathcal{Y}_i} \alpha(\hat{y})\hat{y}, \qquad \sum_{\hat{y}\in \mathcal{Y}_i}\alpha(\hat{y}) = 1. \] The reported experiments use mean aggregation. At inference time, SSS applies calibration before aggregation, using **isotonic regression** or **Venn-Abers predictors**. The calibration ablation reports F1/AUC/accuracy of \(0.7629/0.7999/72.35\%\) for isotonic regression, \(0.7637/0.8003/72.47\%\) for Venn-Abers, and \(0.7291/0.7830/69.93\%\) without calibration. On the Epilepsy iEEG Multicenter Dataset, SSS is reported to outperform modern finite- and infinite-context baselines overall and to outperform all baselines on every unseen medical center in the out-of-distribution evaluation [2410.06412]. **RAxSS** generalizes this framework by replacing uniform aggregation with retrieval-informed weighting. For each window \(w_k\), it retrieves the top-\(m\) most similar nonidentical windows from the same series using Pearson or cosine similarity, \[ s_k^{(j)} = \phi(w_k,w_j), \qquad j \in N_k,\quad |N_k| = m, \] and forms the summary support \[ \bar s_k = \frac{1}{m}\sum_{j\in N_k} s_k^{(j)}. \] Window weights are then \[ \alpha_k = \frac{\exp(\bar s_k/\tau)}{\sum_{t\in K_i}\exp(\bar s_t/\tau)}, \qquad \sum_{k\in K_i}\alpha_k = 1, \] and the series-level prediction is the convex combination \[ \hat p^{(i)}=\sum_{k\in K_i}\alpha_k\,p_k. \] The derivative \[ \frac{\partial \alpha_k}{\partial s_k^{(j)}} = \frac{1}{m\tau}\,\alpha_k(1-\alpha_k) > 0 \] makes the influence of retrieved neighbors explicit. RAxSS reports an evidence trail consisting of a window’s mean support, its ranked neighbor list, and its resulting weight. On the multicenter iEEG evaluation, **RAxSS (cosine)** reports AUC \(= 0.8046 \pm 0.0346\), **RAxSS (Pearson)** reports F1 \(= 0.7275 \pm 0.0489\) and accuracy \(= 70.51 \pm 3.59\), and the **SSS reproduction** reports F1 \(= 0.7437 \pm 0.0537\), AUC \(= 0.8035 \pm 0.0686\), accuracy \(= 71.14 \pm 6.31\) [2510.02936]. ## 4. Constrained Bayesian inference and sparse priors In Bayesian computation, stochastic sparse sampling often addresses constrained or sparsity-inducing latent spaces. On the probability simplex, the key difficulty is that standard SGMCMC methods inherit an \(O(h)\) discretization bias, which becomes especially damaging near the boundary where many coordinates are close to zero. The **SCIR** method resolves this by reparameterizing Dirichlet variables through gamma variables and then using the exact transition law of the Cox–Ingersoll–Ross process, \[ d\theta_t = (a-\theta_t)\,dt + \sqrt{2\theta_t}\,dW_t. \] With minibatch estimator \[ \hat a_m = \alpha + \frac{N}{n}\sum_{i\in S_m} z_i, \] SCIR applies the exact CIR transition with \(\hat a_m\) plugged in. Because the transition is exact, the method removes time discretization error, and the paper proves asymptotic unbiasedness: \[ \lim_{M\to\infty}\left|\frac{1}{M}\sum_{m=1}^M E[\hat\theta_m]-E_\pi[\theta]\right|=0. \] It also derives explicit variance inflation from minibatch noise. Empirically, SCIR is reported to perform substantially better than existing SGMCMC approaches on a sparse simplex experiment, a topic model, and a Dirichlet process mixture model [1806.07137]. A different construction appears in sampling with \(\ell_1\)-priors. **Hadamard Langevin dynamics** starts from the observation that the Laplace prior is non-smooth and that proximal Langevin methods introduce Moreau-envelope bias. The paper introduces latent variables \(u,v \in \mathbb{R}^d\) and the parameterization \[ x = u \odot v. \] An expanded smooth target density \(\widetilde \pi(u,v)\) is constructed so that the pushforward through \((u,v)\mapsto u\odot v\) recovers the original sparse prior exactly. Standard overdamped Langevin dynamics is then run in the augmented space, \[ d\theta_t = -\nabla \widetilde V(\theta_t)\,dt + \sqrt{2}\,dW_t, \] with Euler–Maruyama discretization \[ \theta_{k+1} = \theta_k - h\,\nabla \widetilde V(\theta_k) + \sqrt{2h}\,\xi_{k+1}. \] The continuous-time process is proved well posed and geometrically ergodic, and the discretization is proved convergent as \(h \to 0\). The conceptual distinction from proximal approaches is exactness by overparameterization rather than smoothing of the original \(\ell_1\) potential [2411.11403]. ## 5. Sparse posterior sampling and information-directed decision making In stochastic sparse linear bandits, stochastic sparse sampling refers to action selection driven by sampling from sparse posterior or optimistic-posterior distributions. **Information Directed Sampling for Sparse Linear Bandits** studies \[ Y_t = \langle A_t,\theta^*\rangle + \eta_t, \qquad \Theta=\left\{\theta\in\mathbb R^d:\sum_{j=1}^d \mathbf 1\{\theta_j\neq 0\}\le s,\ \|\theta\|_2\le 1\right\}, \] and shows that optimism can be suboptimal because informative actions may be temporarily low reward. The proposed sparse IDS combines an information-ratio policy with an empirical Bayes sparse posterior sampler based on a **spike-and-slab Gaussian-Laplace prior**. The Bayesian regret bound is \[ BR(n;\pi^{\text{IDS}}) \le \min\left\{ \sqrt{\frac{1}{2}nd\Delta}, \ \frac{s^{2/3}n^{2/3}\Delta^{1/3}}{(2C_{\min}(\mathcal A))^{1/3}} \right\}, \] with \[ \Delta=\min\!\left(\log K,\ 2s\log(Cdn^{1/2}/s)\right), \] which nearly matches lower bounds in both data-rich and data-poor regimes [2105.14267]. **Sparse Optimistic Information-Directed Sampling (SOIDS)** removes Bayesian assumptions and works in the frequentist sparse linear bandit model \[ Y_t = \theta_0^\top A_t + \epsilon_t, \qquad \sum_{j=1}^d \mathbf 1\{\theta_j \neq 0\} \le s,\ \|\theta_0\|_1 \le 1. \] Its optimistic posterior is \[ \frac{dQ_{t+1}^+}{dQ_1^+}(\theta) \propto \prod_{s=1}^t p(Y_s\mid \theta,A_s)^\eta \cdot \exp\!\left(\lambda_t \sum_{s=1}^t \Delta(A_s,\theta)\right), \] and the policy minimizes a surrogate 2-information ratio while inheriting control of higher-order information ratios. Under the sparse optimal action condition, the regret is summarized as \[ R_T = \widetilde O\!\left(\min\big(\sqrt{sdT}, (sT)^{2/3}\big)\right), \] with the appropriate \(C_{\min}\)-dependence in the data-poor regime. The paper presents this as the first frequentist algorithm that simultaneously achieves optimal worst-case regret in both regimes [2510.24234]. A related posterior-sampling line is **Thompson Sampling for High-Dimensional Sparse Linear Contextual Bandits**, where the unknown parameter satisfies \(\|\beta^*\|_0 = s^* \ll d\). The prior samples a support \(S\subset[d]\), sets \(\beta_{S^c}=0\), and places a product Laplace slab on active coordinates. To avoid MCMC cost, the posterior is approximated by mean-field variational Bayes, \[ \bigotimes_{j=1}^d \big[ \gamma_j\,\mathcal{N}(\mu_j,\sigma_j^2) + (1-\gamma_j)\delta_0 \big]. \] The paper proves posterior contraction at a sparse-regression rate and a regret bound whose dependence on ambient dimension is only logarithmic or polylogarithmic, with the leading dependence on \(s^*\) rather than \(d\) [2211.05964]. ## 6. Related usages in large-scale computation and approximation In large-scale graph simulation, **fastRG** samples a sparse random matrix with low-rank expectation \[ \mathbb{E}(A)=XSY^T, \qquad A_{ij}\sim \mathrm{Poisson}(x_i^T S y_j). \] Instead of element-wise \(O(n^2)\) sampling, it first draws the total number of edges \(m\), then samples latent block pairs from \(\tilde S\) and endpoints from the corresponding columns of \(\tilde X\) and \(\tilde Y\). In sparse graphs with \(m=O(n)\), ignoring log terms, the runtime is \(O(n)\). The paper reports simulations up to \(n=10{,}000{,}000\) nodes with \(m=100{,}000{,}000\) edges, and on \(n=500{,}000\), \(m=5{,}000{,}000\), fastRG runs in less than one second on a 3.5 GHz Intel i5 [1703.02998]. In matrix factorization, **subsampled online matrix factorization (somf)** streams columns but randomly subsamples rows using a diagonal mask \(M_t\), keeping only a fraction \(1/r\) of coordinates so that \[ \mathbb{E}\|M_t x_t\|_0 = q = \frac{p}{r}. \] Only the sampled rows are used to approximate code computation, update surrogate statistics, and update dictionary rows. The dominant \(p\)-dependent costs of online matrix factorization are reduced to \(q=p/r\), and the method is proved to converge to a stationary point of the original empirical risk. Reported times to reach within \(1\%\) of convergence include **6 min \(\to\) 28 s** on ADHD200 and **3 h 50 \(\to\) 17 min** on HCP, corresponding to speed-ups \(11.8\times\) and \(13.31\times\), respectively [1701.05363]. In computer vision, **spatially adaptive inference with stochastic feature sampling and interpolation** computes CNN features only at sparsely sampled spatial locations, chosen probabilistically from predicted activation responses via a Gumbel-Softmax mask, \[ \mathcal{M}(p)= \frac{\exp\left((-\log(\pi_1(p))+g_1(p))/\tau\right)} {\sum_{j\in\{0,1\}}\exp\left((-\log(\pi_j(p))+g_j(p))/\tau\right)}. \] Missing locations are reconstructed by interpolation, with the learned RBF kernel reported as the best overall in FLOPs/accuracy tradeoff. On COCO object detection the baseline is **43.4 mAP, 289.5 GFLOPs**, while the method reports **43.3 mAP, 160.4 GFLOPs** at sparse loss weight \(0.02\), and on Cityscapes semantic segmentation the baseline is **80.8 mIoU, 920.6 GFLOPs** while the method reports **80.6 mIoU, 373.2 GFLOPs** at sparse loss weight \(0.05\) [2003.08866]. In Bayesian latent-variable models, **Sparse Stochastic Inference for LDA** combines collapsed Gibbs sampling for document-level latent assignments with online stochastic variational inference for global topic-word parameters. The variational family keeps \(q(z_d)\) structured rather than token-factorized, and minibatch-based natural-gradient updates take the form \[ \lambda_{kw}^{(t)} = (1-\rho_t)\lambda_{kw}^{(t-1)} + \rho_t\left( \eta + \frac{D}{|B|}\sum_{d\in B}\hat N_{dkw} \right). \] The method was used on **1.2 million books (33 billion words)** with thousands of topics, and at \(K=2000\) the paper reports that **less than 1%** of the \(2000 \times 216\) topic-word parameters were nonzero [1206.6425]. In stochastic PDE approximation, the phrase denotes a **non-intrusive sparse-compressive sampling** strategy: draw i.i.d. random inputs \(\bm y_i\), evaluate the deterministic solver, build a measurement matrix \(\bm \Psi\) from polynomial chaos basis functions, and recover sparse coefficients by BPDN or OMP. The method converges in probability with probabilistic error bounds, relying on sparsity of the PC representation and concentration of measure for empirical correlations [1006.2151]. In sparse regression with **limited attribute observation**, the learner observes only \(s'(