Sequential Adaptive Sampling Scheme

• Sequential adaptive sampling schemes are algorithmic frameworks that dynamically adjust data collection based on previous observations to improve decision-making.
• They employ Bayesian updating and techniques like Thompson sampling to balance exploration and exploitation, ensuring robust inference and efficient convergence.
• These methods are applied in diverse fields such as survey design, reinforcement learning, and high-dimensional optimization, demonstrating proven consistency and performance guarantees.

A sequential adaptive sampling scheme refers to any algorithmic framework in which samples (actions, data points, intervention decisions, measurement units, etc.) are collected one at a time with each sampling decision adaptively determined by information acquired in previous steps. These schemes provide dynamic control over sampling policies conditioned on observed data, uncertainty, or reward, and are central in Bayesian decision theory, survey methodology, importance sampling, reinforcement learning, experimental design, and stochastic optimization. The formal structure, guarantees, and application details depend crucially on the underlying modeling assumptions and sequential updating protocol.

1. Bayesian Modeling of Sequential Adaptive Sampling

The canonical probabilistic setup considers an agent operating in an unknown, potentially stochastic environment parameterized by $\theta \in \Theta$, interacting at discrete time points $t=1,\dots,T$. Observational data $D_{t-1}$ collected up to time $t-1$ drives the Bayesian update:

$$P(\theta \mid D_{t-1}) \propto P(\theta) \prod_{s=1}^{t-1} P(o_s \mid a_s, \theta)$$

Each possible environment admits an optimal policy $\pi_\theta(a_t \mid a_{<t}, o_{<t})$. The agent uses the Bayesian mixture over policies to specify an action distribution at each timestep:

$$P(a_t \mid D_{t-1}) = \sum_{\theta \in \Theta} \pi_\theta(a_t \mid a_{<t}, o_{<t})\, P(\theta \mid D_{t-1})$$

Thompson sampling instantiates this mixture, sampling actions sequentially with an automatic exploration/exploitation balance. As observations accumulate, the posterior concentrates on parameters consistent with the observed environment, increasingly favoring exploitation (Ortega et al., 2013).

2. Algorithmic Structures and Representative Mechanisms

Sequential adaptive sampling encompasses a range of algorithmic realizations.

• Thompson Sampling: At each step, sample $\hat\theta_t \sim P(\theta \mid D_{t-1})$, then take $a_t \sim \pi_{\hat\theta_t}$. This two-step rule is both a natural outcome of Bayesian uncertainty modeling and provably consistent under ergodicity (Ortega et al., 2013).
• Population-enrichment in Epidemiology: PoSA (Population-based Sequential Adaptive) intensifies sampling in spatial clusters after detection of rare cases (e.g., TB). Inclusion probabilities $p_i^{(t)}$ are updated adaptively. Estimation is performed via inverse-probability weighting Horvitz–Thompson estimators (Mecatti et al., 2020):

$$\hat{\bar y} = \frac{1}{N} \sum_{i=1}^N \frac{y_i S_i}{p_i^{(i-1)}}$$

• Adaptive Importance Sampling and Multiple Proposals: AMIS (Adaptive Multiple Importance Sampling) adapts proposal distributions sequentially. Weight update schemes, including balance heuristics and discarding-reweighting, allow reuse or selective discarding of samples to control effective sample size and computational burden (Thijssen et al., 2018).
• Structured High-Dimensional Estimation: Adaptive SGD minimizes either the variance of importance weights or a proxy (KL-divergence, $L_2$ distance) to the optimal importance distribution, with gradients estimated sequentially using past samples (Ortiz et al., 2013).

3. Theoretical Guarantees: Consistency, Efficiency, Optimality

Sequential adaptive schemes are often justified by rigorous convergence and regret bounds.

• Consistency: When ergodicity and identifiability conditions are met, the posterior over $\theta$ concentrates on the true parameter $\theta^*$, yielding convergence of the adaptive sampling-induced distribution $P(a_t \mid D_{t-1}) \to \pi_{\theta^*}$. Similar guarantees extend to estimation in rare-disease surveys via unbiasedness and root mean square error control (Ortega et al., 2013, Mecatti et al., 2020).
• Regret Bounds: For $K$-armed bandits under Thompson sampling, expected cumulative regret satisfies

$\mathbb{E}[R(T)] = O(\sqrt{KT \ln T})$

with asymptotic logarithmic regret under suitable priors (Ortega et al., 2013). In survey sampling, PoSA/CPoSA improves efficiency relative to cross-sectional sampling particularly in spatially clustered populations (Mecatti et al., 2020).

• Optimality Criteria: Thompson sampling is Bayes–optimal under sampling constraints, minimizing expected Kullback–Leibler divergence between agent and environment joint laws (Ortega et al., 2013). In adaptive survey schemes, design-unbiasedness is maintained by exact calculation of inclusion probabilities.

4. Extensions: Exploration, Causal Inference, Decision Theory

Sequential adaptive sampling frameworks support several major extensions:

• Game-theoretic Interaction and Multi-Agent Adaptation: Applies when multiple agents adaptively interact; the Bayesian mixture over policies enables a game-theoretic analysis (Ortega et al., 2013).
• Causal Inference via Sequential Interventions: When each $\theta$ indexes a causal model, past actions are treated as interventions; the Bayesian update is performed respecting do-calculus. Sampling $\theta$ then adaptively chooses interventions that facilitate discovery of the true causal structure (Ortega et al., 2013):

$$P(\theta \mid \hat a_{<t}, o_{<t}) \propto P(\theta) \prod_{s=1}^{t-1} P(o_s \mid a_s, \theta)$$

• Adaptive Control and Optimization Under Budget Constraints: Sample size and sampling allocation can be controlled to respect cost or error budgets. For instance, conditional PoSA (CPoSA) maintains fixed minimal sample sizes under logistic constraints while preserving design-unbiasedness via rejective-sampling weights (Mecatti et al., 2020).

5. Practical Implementation and Performance Considerations

Implementation details vary with context and objectives.

• Data Structures: In sequential Monte Carlo, adaptation is performed by fitting parametric families to current weighted samples (e.g., logistic-conditional models for variable selection (Schäfer et al., 2011)).
• Computational Complexity: Flat re-weighting and discarding schemes admit $O(MK)$ cost per iteration, whereas balance heuristics can induce up to $O(MK^2)$ cost, highlighting efficiency trade-offs (Thijssen et al., 2018).
• Empirical Application: Sequential adaptive designs have been shown to double the case-detection rate and cut costs by 20–30% in tuberculosis prevalence surveys, with minimal impact on confidence interval width or RMSE compared to classic designs (Mecatti et al., 2020).

6. Generalizations and Domain-Specific Optimizations

Sequential adaptive sampling is generalized to wider classes of problems:

• Stochastic Optimization under Unknown Cost Functions: Adaptive policies have been constructed for multi-armed bandit models with incomplete information and budget constraints, relying on forced-sampling sequences and certainty-equivalence LP allocation. These guarantee almost sure convergence to full-information optima (Burnetas et al., 2012).
• Rare Event Estimation, Compressed Sensing, and High-Dimensional Modeling: Huffman-coded adaptive compressed sampling provides a deterministic, sequential adaptive scheme for sparse signal recovery with $O(s \log n)$ measurement complexity (0810.4916); sequential double sampling strategies leverage auxiliary variables for cluster and rare event estimation with analytic variance formulas (Panahbehagh et al., 2018); sequential directional importance sampling (SDIS) enables adaptive rare event probability estimation with controlled coefficient of variation across intermediate steps (Cheng et al., 2022).

---

Sequential adaptive sampling integrates Bayesian policy uncertainty, optimal allocation, and dynamic updating. Its utility is revealed in environments requiring real-time learning, cost-effectiveness, optimal inference, and robust estimation under uncertainty. The general theory has enabled a rich array of domain-specific algorithms, each leveraging the defining principle: sample adaptively, calibrate to evidence, and guarantee statistical efficiency and consistency.