AB-SID-iVAR: Adaptive GP Active Learning

• The paper introduces AB-SID-iVAR as a novel active-learning acquisition strategy that minimizes integrated posterior variance using a closed-form self-induced Boltzmann surrogate.
• It leverages Gaussian process models and variance reduction techniques to focus query selection on thermodynamically or decision-relevant regions rather than across the entire domain.
• Empirical results show that AB-SID-iVAR achieves significantly lower weighted MSE in applications like potential energy surface modeling and molecular screening compared to baseline methods.

Searching arXiv for the primary paper and closely related active-learning work mentioned in the source material. AB-SID-iVAR is a Gaussian-process active-learning acquisition strategy for regression problems in which predictive accuracy is evaluated under a self-induced Boltzmann distribution, rather than under a fixed or uniform test measure. In this setting, the target distribution depends on the unknown function itself, so both the weighting measure and its partition function are unavailable during data acquisition. The method introduced in "Active Learning for Gaussian Process Regression Under Self-Induced Boltzmann Weights" (Qing et al., 11 May 2026) addresses this by constructing a closed-form surrogate for the Bayesian marginal of the target distribution and then selecting queries that minimize integrated posterior variance under that surrogate. The resulting framework is designed for expensive-function settings such as potential energy surface modeling and molecular screening, where prediction quality matters primarily in thermodynamically or decision-relevant regions rather than across the entire domain.

1. Problem formulation and self-induced Boltzmann weighting

The method is defined for noisy regression with expensive function evaluations,

$$y = f(x) + \epsilon,\qquad \epsilon \sim \mathcal{N}(0,\tau^2),$$

where $f : \mathcal{X} \to \mathbb{R}$ is unknown. The distinguishing feature is that performance is not measured uniformly over $\mathcal{X}$, but under a self-induced Boltzmann distribution

$$P_f(x) = Z_f^{-1} \exp\bigl(\lambda f(x) + b(x)\bigr),$$

with known $\lambda \neq 0$, known bounded bias $b(x)$, and partition function

$$Z_f = \int_{\mathcal{X}} \exp(\lambda f(x)+b(x))\,dx < \infty.$$

Because both the exponent and the normalizer depend on $f$, the target measure is unknown and its normalization is generally intractable (Qing et al., 11 May 2026).

The loss minimized by active learning is the weighted mean-squared error

$$\mathcal{L}(\hat f) = \mathbb{E}_{x \sim P_f}\bigl[(\hat f(x)-f(x))^2\bigr].$$

With a Gaussian-process surrogate, the final predictor is the posterior mean $\mu_T$ after $f : \mathcal{X} \to \mathbb{R}$0 queries, and the objective is to choose $f : \mathcal{X} \to \mathbb{R}$1 so as to minimize $f : \mathcal{X} \to \mathbb{R}$2 (Qing et al., 11 May 2026).

This formulation arises naturally in applications where downstream relevance is exponentially tilted toward favorable or physically accessible configurations. In potential energy surface modeling,

$f : \mathcal{X} \to \mathbb{R}$3

so low-energy regions dominate thermodynamic behavior. In molecular scoring or drug-discovery settings, one may instead use

$f : \mathcal{X} \to \mathbb{R}$4

to emphasize high-scoring candidates. This suggests that the central difficulty is not merely exploration under uncertainty, but acquisition under a target distribution that is itself latent and function-dependent (Qing et al., 11 May 2026).

2. Gaussian-process model and variance-based objective

AB-SID-iVAR assumes a Gaussian-process prior

$f : \mathcal{X} \to \mathbb{R}$5

with kernel satisfying $f : \mathcal{X} \to \mathbb{R}$6. Given data $f : \mathcal{X} \to \mathbb{R}$7, the posterior predictive mean and covariance are

$f : \mathcal{X} \to \mathbb{R}$8

$f : \mathcal{X} \to \mathbb{R}$9

with posterior variance $\mathcal{X}$0 (Qing et al., 11 May 2026).

A key analytical step is that the weighted MSE under the unknown $\mathcal{X}$1 can be related to integrated posterior variance under a surrogate Boltzmann distribution built from the posterior mean. The paper gives a high-probability bound and an average-case bound showing that control of

$\mathcal{X}$2

is sufficient to control the target loss, where

$\mathcal{X}$3

The average-case bound is

$\mathcal{X}$4

This motivates an integrated variance reduction criterion targeted at an approximation to the self-induced distribution itself (Qing et al., 11 May 2026).

The method therefore belongs to the variance-reduction family of active-learning criteria, but with a weighting measure that is adaptive, posterior-dependent, and specifically designed to approximate a latent Boltzmann target. A plausible implication is that its acquisition behavior differs fundamentally from uniform IMSE: it is driven not simply by uncertainty, but by uncertainty in regions that are likely to carry high Boltzmann mass.

3. Definition of AB-SID-iVAR

SID-iVAR is a one-step look-ahead rule that chooses the next query by minimizing future integrated posterior variance under an unnormalized surrogate density: $\mathcal{X}$5 Here $\mathcal{X}$6 is the fantasy posterior variance after a hypothetical observation at candidate $\mathcal{X}$7, and $\mathcal{X}$8 is a constraint set that enforces exploration (Qing et al., 11 May 2026).

The defining ingredient of AB-SID is an Approximate Bayesian surrogate for the intractable Bayesian self-induced density

$\mathcal{X}$9

Using a zero-order Taylor approximation of $$P_f(x) = Z_f^{-1} \exp\bigl(\lambda f(x) + b(x)\bigr),$$0 around $$P_f(x) = Z_f^{-1} \exp\bigl(\lambda f(x) + b(x)\bigr),$$1, with $$P_f(x) = Z_f^{-1} \exp\bigl(\lambda f(x) + b(x)\bigr),$$2 and $$P_f(x) = Z_f^{-1} \exp\bigl(\lambda f(x) + b(x)\bigr),$$3, the method approximates

$$P_f(x) = Z_f^{-1} \exp\bigl(\lambda f(x) + b(x)\bigr),$$4

Because the posterior is Gaussian, $$P_f(x) = Z_f^{-1} \exp\bigl(\lambda f(x) + b(x)\bigr),$$5 and $$P_f(x) = Z_f^{-1} \exp\bigl(\lambda f(x) + b(x)\bigr),$$6 are available in closed form via the Gaussian MGF, yielding the unnormalized surrogate

$$P_f(x) = Z_f^{-1} \exp\bigl(\lambda f(x) + b(x)\bigr),$$7

The denominator need not be computed, since only relative weights enter the acquisition integral (Qing et al., 11 May 2026).

The term $$P_f(x) = Z_f^{-1} \exp\bigl(\lambda f(x) + b(x)\bigr),$$8 directs attention toward regions favored by the current posterior mean, while the term $$P_f(x) = Z_f^{-1} \exp\bigl(\lambda f(x) + b(x)\bigr),$$9 inflates mass in uncertain regions through the Gaussian MGF. This suggests that AB-SID-iVAR is neither purely exploitative nor purely uncertainty-seeking; its weighting mechanism internalizes posterior uncertainty directly in the surrogate target measure.

Exploration is enforced through the constraint set

$\lambda \neq 0$0

Only candidates whose current variance is at least the surrogate-weighted average variance are admissible. The complete acquisition is therefore

$\lambda \neq 0$1

with $\lambda \neq 0$2 given above (Qing et al., 11 May 2026).

4. Algorithmic realization on discrete and continuous domains

The iterative procedure is as follows. At each round, the Gaussian-process posterior is updated from $\lambda \neq 0$3, the SID surrogate is constructed, the constraint set is formed using the surrogate-weighted mean variance, the acquisition is optimized over admissible candidates, and the resulting query is evaluated to update the dataset. The final output is the posterior mean $\lambda \neq 0$4 (Qing et al., 11 May 2026).

The implementation differs according to the nature of $\lambda \neq 0$5.

Domain	Integral evaluation	Practical consequence
Discrete $\lambda \neq 0$6	Exact finite sums	No Monte Carlo required
Continuous $\lambda \neq 0$7	Approximated by MCMC	Uses SMC with tempering and random-walk Metropolis rejuvenation

In discrete domains, both the acquisition integral and the expectation defining $\lambda \neq 0$8 are exact sums over the candidate set. In continuous domains, they are approximated from samples drawn from $\lambda \neq 0$9 by sequential Monte Carlo with tempering and random-walk Metropolis rejuvenation, and the theoretical analysis explicitly incorporates the resulting Monte Carlo error (Qing et al., 11 May 2026).

The paper reports that, in continuous settings, gradient-based optimizers such as SLSQP and L-BFGS-B via BoTorch are used in practice. Runtime scales roughly linearly with the particle count $b(x)$0, and experiments show practical runtimes of order 10–30 seconds per iteration for dimensions up to 6, using $b(x)$1 particles; smaller values such as $b(x)$2 are also described as adequate in empirical tests (Qing et al., 11 May 2026).

5. Thompson-sampling variant and relation to existing acquisitions

A companion method, TS-SID-iVAR, replaces the analytic AB-SID surrogate with a single posterior draw $b(x)$3, defining

$b(x)$4

The same integrated-variance objective is then optimized under this sampled surrogate (Qing et al., 11 May 2026).

The paper characterizes the distinction as follows: AB-SID-iVAR is a biased, low-variance approximation of $b(x)$5 using analytic Gaussian MGFs, whereas TS-SID-iVAR is an unbiased, high-variance Monte Carlo approximation based on one posterior sample. Empirically, TS-SID-iVAR is described as competitive but more variable across random seeds, especially on multimodal problems (Qing et al., 11 May 2026).

Within the broader active-learning literature, AB-SID-iVAR is positioned against several existing paradigms. Classical IMSE minimizes $b(x)$6 under uniform weighting, whereas AB-SID-iVAR uses a self-induced Boltzmann surrogate. Prediction-oriented active learning and test-distribution-aware active learning assume that the target test distribution is known and fixed; AB-SID-iVAR instead handles the case where the target depends on the unknown function. Distributionally robust active learning, by contrast, optimizes worst-case performance over an ambiguity set and does not exploit the specific Boltzmann structure (Qing et al., 11 May 2026).

The paper also contrasts AB-SID-iVAR with heuristic scientific strategies such as FLARE and HAL, which sample from biased densities of the form

$b(x)$7

and trigger queries based on local uncertainty thresholds. Those methods share the use of Boltzmann-like surrogates and uncertainty modulation, but the paper states that they are not derived from a one-step Bayes criterion and lack the same formal guarantees (Qing et al., 11 May 2026).

6. Theoretical guarantees

The convergence analysis covers both AB-SID-iVAR and TS-SID-iVAR under a Gaussian-process prior with a stationary, four-times differentiable kernel on compact $b(x)$8, noisy observations of variance $b(x)$9, and self-induced Boltzmann weighting of the prescribed form. For continuous domains, Monte Carlo approximation error from MCMC is treated explicitly (Qing et al., 11 May 2026).

Let

$$Z_f = \int_{\mathcal{X}} \exp(\lambda f(x)+b(x))\,dx < \infty.$$0

denote the maximum information gain. The high-probability terminal MSE bound states that, for any $$Z_f = \int_{\mathcal{X}} \exp(\lambda f(x)+b(x))\,dx < \infty.$$1, with probability at least $$Z_f = \int_{\mathcal{X}} \exp(\lambda f(x)+b(x))\,dx < \infty.$$2,

$$Z_f = \int_{\mathcal{X}} \exp(\lambda f(x)+b(x))\,dx < \infty.$$3

where $$Z_f = \int_{\mathcal{X}} \exp(\lambda f(x)+b(x))\,dx < \infty.$$4 for AB-SID-iVAR and $$Z_f = \int_{\mathcal{X}} \exp(\lambda f(x)+b(x))\,dx < \infty.$$5 for TS-SID-iVAR, $$Z_f = \int_{\mathcal{X}} \exp(\lambda f(x)+b(x))\,dx < \infty.$$6, $$Z_f = \int_{\mathcal{X}} \exp(\lambda f(x)+b(x))\,dx < \infty.$$7, $$Z_f = \int_{\mathcal{X}} \exp(\lambda f(x)+b(x))\,dx < \infty.$$8, and

$$Z_f = \int_{\mathcal{X}} \exp(\lambda f(x)+b(x))\,dx < \infty.$$9

For finite $f$0 with exact threshold computation, $f$1 (Qing et al., 11 May 2026).

The paper interprets this as showing that the weighted MSE goes to zero, with rate controlled by $f$2 plus Monte Carlo and confidence-bound terms. The corresponding corollary states that if $f$3 is constant, then $f$4 for any $f$5, whereas if $f$6, then $f$7 (Qing et al., 11 May 2026).

A tighter Bayesian average-case theorem gives

$f$8

again with $f$9 for AB and $$\mathcal{L}(\hat f) = \mathbb{E}_{x \sim P_f}\bigl[(\hat f(x)-f(x))^2\bigr].$$0 for TS. The paper notes that this improves logarithmic factors relative to the high-probability result (Qing et al., 11 May 2026).

The proof strategy combines GP confidence bands, density-ratio control for $$\mathcal{L}(\hat f) = \mathbb{E}_{x \sim P_f}\bigl[(\hat f(x)-f(x))^2\bigr].$$1 without partition-function evaluation, variance-based surrogate bounds, information-gain inequalities, and Freedman’s inequality for cumulative Monte Carlo error. A plausible implication is that the main novelty of the theory lies not in a new generic GP concentration argument, but in making self-induced Boltzmann weighting analytically tractable despite the latent normalizer.

7. Empirical behavior, applications, and limitations

The empirical study compares AB-SID-iVAR and TS-SID-iVAR against random sampling, uncertainty sampling, uniform IMSE, prediction-oriented EPIG, and GHAL on synthetic benchmarks, potential energy surface tasks, and molecular drug discovery problems (Qing et al., 11 May 2026).

On standard synthetic functions from one to six dimensions with target $$\mathcal{L}(\hat f) = \mathbb{E}_{x \sim P_f}\bigl[(\hat f(x)-f(x))^2\bigr].$$2, AB-SID-iVAR is reported to consistently achieve the lowest Boltzmann-weighted MSE. The advantage grows with dimension and with more concentrated targets, that is, larger $$\mathcal{L}(\hat f) = \mathbb{E}_{x \sim P_f}\bigl[(\hat f(x)-f(x))^2\bigr].$$3. Uncertainty sampling is reported to fail badly in some cases because it ignores target relevance under $$\mathcal{L}(\hat f) = \mathbb{E}_{x \sim P_f}\bigl[(\hat f(x)-f(x))^2\bigr].$$4, and IMSE is described as strong but inferior on the weighted metric because it learns uniformly. An ablation further shows that replacing the surrogate with $$\mathcal{L}(\hat f) = \mathbb{E}_{x \sim P_f}\bigl[(\hat f(x)-f(x))^2\bigr].$$5 and omitting the $$\mathcal{L}(\hat f) = \mathbb{E}_{x \sim P_f}\bigl[(\hat f(x)-f(x))^2\bigr].$$6 term leads to failures on multimodal or oscillatory targets such as Branin and Ishigami, while removing the constraint set makes performance more fragile (Qing et al., 11 May 2026).

In potential energy surface modeling, the paper studies four systems: $$\mathcal{L}(\hat f) = \mathbb{E}_{x \sim P_f}\bigl[(\hat f(x)-f(x))^2\bigr].$$7 on Cu(100), H atom on Cu$$\mathcal{L}(\hat f) = \mathbb{E}_{x \sim P_f}\bigl[(\hat f(x)-f(x))^2\bigr].$$8 cluster, Si crystal, and $$\mathcal{L}(\hat f) = \mathbb{E}_{x \sim P_f}\bigl[(\hat f(x)-f(x))^2\bigr].$$9 on Pt(111), with dimensions ranging from 2D to 6D. AB-SID-iVAR is reported to yield the lowest weighted MSE across all systems, while GHAL is sometimes competitive but sensitive to heuristic hyperparameters $\mu_T$0 and exhibits high variance (Qing et al., 11 May 2026).

In molecular drug discovery, experiments are conducted on a discrete pool of approximately $\mu_T$1 GuacaMol molecules using a Gaussian process with Tanimoto kernel over Morgan fingerprints and two scoring functions, Median 1 and Median 2, under targets $\mu_T$2 with $\mu_T$3. The reported evaluation metrics are weighted MSE under $\mu_T$4 and $\mu_T$5 on the top-$\mu_T$6 of molecules ranked by Boltzmann weight. AB-SID-iVAR is reported to achieve 1–2 orders of magnitude lower weighted MSE than baselines, with larger gains for $\mu_T$7. While global $\mu_T$8 is similar across methods, baseline $\mu_T$9 on top-$f : \mathcal{X} \to \mathbb{R}$00 molecules is said to drop to negative values as $f : \mathcal{X} \to \mathbb{R}$01 shrinks, whereas AB-SID-iVAR maintains near-zero or positive $f : \mathcal{X} \to \mathbb{R}$02 even at small $f : \mathcal{X} \to \mathbb{R}$03 (Qing et al., 11 May 2026).

The method’s practical limitations are also explicit. Theoretical bounds contain exponential factors in $f : \mathcal{X} \to \mathbb{R}$04, which may be loose for large $f : \mathcal{X} \to \mathbb{R}$05. Continuous-domain optimization is more expensive than uniform IMSE or uncertainty sampling because of MCMC and constrained optimization. The present analysis assumes a GP surrogate and strictly Boltzmann self-induced distributions, and extending either the modeling framework or the theory beyond those assumptions remains open (Qing et al., 11 May 2026).

Overall, AB-SID-iVAR can be understood as a GP-based experimental design criterion tailored to weighted prediction error under a latent Boltzmann test measure. Its central technical contribution is the closed-form surrogate

$f : \mathcal{X} \to \mathbb{R}$06

which avoids partition-function estimation while preserving a Bayesian connection to the self-induced target distribution. This suggests that its significance lies in bridging Bayesian active learning, integrated variance reduction, and thermodynamically weighted learning in a single framework (Qing et al., 11 May 2026).