Autoregressive Row-Wise Sampling Algorithm

• Autoregressive row-wise sampling algorithms are methods that use row or block sampling guided by leverage scores and autoregressive structures to ensure scalable inference and preserve key spectral properties.
• They are applied across matrix spectral approximation, time-series analysis, neural sequence generation, and quantum simulation, offering theoretical guarantees and practical efficiency.
• Empirical studies demonstrate accelerated computation, reduced memory overhead, and improved model fitting in large-scale and streaming data environments.

An autoregressive row-wise sampling algorithm is a class of data reduction and generation methods that leverages row-wise or block-wise sampling strategies—often guided by leverage scores or autoregressive model structure—to accelerate computation, enable scalable inference, and preserve critical spectral or statistical properties in high-dimensional or large-scale matrix, time-series, generative, or quantum many-body settings. These algorithms select or generate rows or blocks either for subsampling (in regression, streaming, or inference contexts) or for structured parallel generation (in neural and quantum models), typically with rigorous guarantees of statistical consistency or spectral approximation.

1. Row-Wise Sampling for Spectral Approximation of Matrices

Given a tall matrix $A \in \mathbb R^{n \times d}$ with $n \gg d$, the goal is to construct a reduced matrix $\tilde A \in \mathbb R^{m \times d}$ comprising a subset of rows of $A$ such that the spectral norm of $A^T A$ is closely approximated by $\tilde A^T \tilde A$:

$$(1-\epsilon)\,A^T A - \delta I_d \preceq \tilde A^T \tilde A \preceq (1+\epsilon)\,A^T A + \delta I_d$$

with error parameters $\epsilon\in(0,1)$ and $\delta > 0$ (Cohen et al., 2016).

Key to such sampling is the computation of (ridge) leverage scores for each row:

$$\ell_i^\lambda(A) = a_i^T (A^T A + \lambda I)^{-1} a_i$$

where $a_i$ designates row $i$ and $\lambda$ is a regularization parameter. Rows are sampled with probabilities proportional to (an overestimate of) these scores, and reweighted as $a_i / \sqrt{p_i}$.

Several algorithmic variants exist:

• Online-Sample: processes rows one by one, selects rows based on current leverage approximations, and provides high-probability spectral approximation with $$O(d\log d\log(\epsilon\|A\|_2^2/\delta)/\epsilon^2)$$ sampled rows and minimal memory overhead.
• Slim-Sample: leverages a secondary, lower-precision sketch to further reduce memory without weakening the spectral guarantee.
• Online-BSS: optimal sample complexity (removing extra logarithmic factors) relying on $O(d^2)$ memory via Batson–Spielman–Srivastava (BSS) barrier maintenance.

Fundamental lower bounds show no online algorithm can improve upon $O(d\log(\epsilon||A||_2^2/\delta)/\epsilon^2)$ sampled rows while achieving the same spectral guarantee (Cohen et al., 2016).

2. Leverage Score–Based Row Sampling for Autoregressive Time Series

For AR($p$) models (e.g., $$Y_t = \phi_1 Y_{t-1} + \ldots + \phi_p Y_{t-p} + W_t$$), LSAR (Eshragh et al., 2019) and SLS (Xie et al., 25 Sep 2025) introduce methods to sample rows (data points) efficiently for model fitting and inference by leveraging randomized numerical linear algebra (RandNLA) and online leverage scoring.

In LSAR, leverage scores for each row of the lagged design matrix $X$ are computed via recursive formulas exploiting Hankel/Toeplitz structure:

$$\ell_{n,p}(i) = \ell_{n-1,p-1}(i) + \frac{r_{n-1,p-1}(i)^{2}}{\|r_{n-1,p-1}\|_2^2}$$

where $r_{n-1,p-1}(i)$ is a model residual. Instead of full OLS, small row subsets are drawn proportional to leverage estimates, resulting in worst-case time $O(n p^{*} + p^{*4} \log p^{*}/ \epsilon^2)$ for order selection and coefficient estimation, rather than $O(n p^{*2})$ (Eshragh et al., 2019).

SLS extends this to streaming data: it computes streaming leverage scores using a pilot window, selects the block start via Bernoulli trials on these scores, and, upon trigger, expands the block sequentially until a sufficient Fisher information threshold is reached. The full block is then used for OLS. This method achieves asymptotic normality for $p$-dimensional parameter inference in both linear and nonlinear AR settings, is memory-light ($O(p^2)$), and operates on massive data streams efficiently (Xie et al., 25 Sep 2025).

3. Autoregressive Row-Wise Sampling for Accelerated Neural Generation

Chunk-wise or row-wise structured sampling accelerates autoregressive sequence and image generation by breaking the strictly sequential process. In the Neural Approximation of an Auto-Regressive Process (NARA) (Yoo et al., 2019), the future chunk of $M$ samples is proposed in parallel using a fast prior predictor $q_W(\cdot | x_{<t})$. These are post-processed by the mother AR model, and a learned confidence network $g_V$ determines whether each proposal is accepted or needs sequential resampling.

Algorithmically, for each chunk:

• Parallel proposals $m_{t:t+M-1}$ are drawn.
• The AR model post-processes each proposal, a confidence score $\sigma_l$ is computed, and positions with $\sigma_l$ above a threshold $\tau$ are accepted; others are resampled sequentially.
• Theoretically, this factorizes as $$q_{θ,W}(x_{i+1:j} \mid x_{\leq i}, m) = p_θ(x_{i+1:j} \mid x_{\leq i}) \cdot \mathcal{R}(p_θ, q_W, m)$$, where $\mathcal{R}$ goes to $1$ when priors and AR coincide.

Empirical studies on LSTMs and PixelCNN++ models reveal $5-10\times$ acceleration on images, with high-quality generation achieved for intermediate values of the acceptance threshold, as measured by FID scores and $\ell_1$ reconstruction errors (Yoo et al., 2019).

4. Collective Row Updates in Quantum Many-Body Monte Carlo

In variational Monte Carlo (VMC) for projected entangled-pair states (PEPS), autoregressive row-wise sampling (Chen et al., 28 Jan 2026) updates entire rows of spins in a two-dimensional lattice configuration. The joint probability $P_\theta(\sigma)$ is factorized over rows and then columns:

$$P_\theta(\sigma) = \prod_{r=1}^{R} P_\theta\big(\sigma_{\mathrm{row}^{(r)}} | \sigma_{<\mathrm{row}^{(r)}}\big)$$

$$P_\theta(\sigma_{r,1:C} | \sigma_{<r,\cdot}) = \prod_{c=1}^{C} P_\theta(\sigma_{r,c} | \sigma_{r,<c}, \sigma_{<r,\cdot})$$

Algorithm steps for updating a row:

• Boundary environments (matrix product states, MPS) above and below the target row are contracted.
• For each column position, the conditional probability for possible spins is evaluated by contracting the network with the current configuration, generating samples sequentially within the row.
• Replacement of the current row by newly sampled values yields non-local moves with rejection-free sampling.

Benchmarks demonstrate profound reductions in autocorrelation time ($\tau \approx O(1)$ vs $\tau \propto L^z$ for Metropolis), mitigation of critical slowing down near quantum phase transitions, and improved variational optimization stability in spin glass benchmarking (Chen et al., 28 Jan 2026).

5. Sampling Probabilities and Theoretical Guarantees

A central aspect across these algorithms is the use of leverage or confidence-based probabilities for row/block selection or acceptance:

• In matrix sampling, $p_i = \min(c\cdot \tilde \ell_i, 1)$ with $c=8\log d/\epsilon^2$ for spectral guarantees (Cohen et al., 2016).
• For AR models, rows are subsampled such that $s = O(p \log(p/δ)/(β\epsilon^2))$ suffices, and leverage approximations satisfy $$|\ell_{n,p}(i)-\hat\ell_{n,p}(i)| \le O(p\sqrt{\epsilon})\ell_{n,p}(i)$$ with high probability (Eshragh et al., 2019).
• For streaming SLS, block start is triggered by a Bernoulli trial on approximate streaming leverage scores; block is expanded until Fisher information exceeds threshold $c$, with downstream estimator asymptotically normal (Xie et al., 25 Sep 2025).
• In neural generation, acceptance of proposals reflects learned confidence, balancing acceleration and reconstruction fidelity (Yoo et al., 2019).

Theoretical results encompass matrix-Chernoff bounds for spectral approximation, martingale CLTs for AR estimator distribution, and equivalence/convergence of approximate and exact AR generation under appropriate conditions.

6. Empirical Performance and Practical Applications

Practical applications include:

• Interpretable, compressed representations for large matrices, graph data, and streaming sensor data (Cohen et al., 2016, Xie et al., 25 Sep 2025).
• Rapid fitting and model selection for massive time series (e.g., AR(200) with $n=2\times 10^6$ rows) with accurate PACF recovery and $100\times$ runtime reductions (Eshragh et al., 2019).
• Efficient detection and characterization of macro- and microseismic events in streaming physical datasets using online SLS (Xie et al., 25 Sep 2025).
• Accelerated sequence and image generation in deep neural architectures, with empirical speedups and competitive task-specific measures (Yoo et al., 2019).
• Substantial gains in mixing, optimization stability, and energy minimization in quantum many-body simulation workloads (Chen et al., 28 Jan 2026).

A plausible implication is that autoregressive row-wise sampling principles generalize efficiently across streaming inference, numerical analysis, generative modeling, and quantum simulation domains, yielding both theoretical guarantees and practical computational savings.