BSPMI: Optimal Posterior Mean Incumbent

• BSPMI is a Bayesian optimization method that selects the incumbent as the observed point with the lowest posterior mean, demonstrating efficient computation and proven no-regret learning.
• It achieves sublinear cumulative regret in noisy GP models and extends modularly to multi-objective scenarios and low-rank approximations.
• BSPMI informs robust decision-making in experimental design and information acquisition by bridging theoretical optimality with practical performance.

The Best Sampled Posterior Mean Incumbent (BSPMI) is a principled incumbent selection strategy, predominantly used in Bayesian optimization and bandit frameworks, with foundational connections to Bayesian experimental design and optimal low-rank inference. BSPMI defines the incumbent as the sampled point among those previously observed which achieves the minimal posterior mean, and is motivated by both computational considerations and statistical optimality properties. BSPMI is specifically notable for balancing computational efficiency with strong theoretical guarantees—including no-regret learning in noisy Gaussian process optimization, modularity in multi-objective settings, and optimality in low-rank posterior approximation. Its development is supported by a spectrum of theoretical results in both finite- and infinite-dimensional Bayesian models and guides algorithm design in a range of applied and theoretical contexts.

1. Definition and Context of BSPMI

BSPMI is defined in the canonical Gaussian process (GP) expected improvement (EI) framework as follows: at time $t$, let $S_t$ denote the set of evaluation locations already sampled. The BSPMI incumbent $p_t$ is

$p_t = \min_{x \in S_t} \mu_{t-1}(x)$

where $\mu_{t-1}(x)$ is the posterior mean of the GP at point $x$ given data up to $t-1$. This is distinct from the Best Posterior Mean Incumbent (BPMI), which minimizes $\mu_{t-1}(x)$ over the entire domain (typically requiring global optimization), and the Best Observation Incumbent (BOI), which uses $\min_{x \in S_t} y(x)$, the best observed value.

This selection mechanism defines a “reference level” for the EI acquisition function and has direct efficiency advantages since it only requires evaluating the mean at a (typically small) discrete set. BSPMI appears not only in optimization algorithms (Wang et al., 21 Aug 2025), but also as an analytical tool in experimental design and data-driven information acquisition (Mensch et al., 2023).

2. Regret Analysis and Theoretical Guarantees

The cumulative regret of algorithms using BSPMI as the incumbent has been rigorously analyzed in the context of noisy GP-EI. Lemma 2 and Theorem 3 in (Wang et al., 21 Aug 2025) establish that, under standard assumptions (e.g., squared exponential or Matérn kernels), the GP-EI algorithm with BSPMI achieves a cumulative regret of

$R_T = O(T^{1/2} \log^2 T)$

for the squared exponential kernel, with analogous sublinear bounds for Matérn kernels (with exponents determined by smoothness and dimension). This ensures no-regret learning, meaning the average per-step regret vanishes as $T \to \infty$.

The regret bounds for BPMI are marginally tighter (lower constants and reduced logarithmic factors) because BPMI has global access to the surrogate; however, BSPMI avoids the computational burden of global minimization at each step. BOI, while simple, can suffer in high-noise regimes—they are not guaranteed to be no-regret if the observed best value is corrupted downwards by noise.

Table: Comparison of Common Incumbents in GP-EI

Incumbent	Definition	Regret Bound (SE Kernel)
BPMI	$\min_{x \in \mathcal{X}} \mu_{t-1}(x)$	$O(\sqrt{T} \log T)$
BSPMI	$\min_{x \in S_t} \mu_{t-1}(x)$	$O(\sqrt{T} \log^2 T)$
BOI	$\min_{x \in S_t} y(x)$	$\text{May not be sublinear in noise}$

In regimes where computational tractability and robust regret guarantees are required, BSPMI provides a favorable trade-off.

3. Information Acquisition and Economic Interpretation

BSPMI’s role in decision-making is not limited to optimization; it extends to models of information acquisition where costs are linear or concavifiable in the posterior means (Mensch et al., 2023). When a decision maker (DM) faces a payoff structure depending solely on the distribution of posterior means, the “posterior-mean separable” cost model takes the form

$C(F) = c(z_0) - \int c(z) dF(z)$

where $F$ is the cumulative distribution function of posterior means and $c$ is a generating function.

The optimal information acquisition problem becomes

$$\max_{F \in \mathcal{I}(F_0)} \int [\varphi_A(z) + c(z)] dF(z) - c(z_0)$$

where $\varphi_A(z)$ is the indirect utility under menu $A$ and $z$ is the realized posterior mean. For BSPMI-style problems, this structure enables dimension reduction: optimization and information design problems become concavifications or linear programs over distributions on scalar posterior means, vastly simplifying analysis and calibrations.

Testable revealed preference axioms—No Improving Action Switches (NIAS) and No Improving Posterior-Mean Cycles (NIPMC)—characterize when observed stochastic choice data is consistent with posterior-mean separable costs, and underpin the practical utility of BSPMI in these contexts.

4. Optimality and Bayes Risk in Linear Inverse Problems

In finite-dimensional, linear Gaussian models, the Bayes risk of the posterior mean, under squared loss, is exactly the trace of the posterior covariance matrix (Alexanderian, 2023): $$\mathcal{E}(m^y) = \int \| m - m^y \|^2 \, p(dm) = \operatorname{tr}(\Gamma_{\mathrm{post}})$$ This result directly motivates approaches where the “best” estimator (in expectation) among those available is the posterior mean with the minimal posterior covariance trace. In experimental design, this corresponds to A-optimality.

Extensions to infinite-dimensional inverse problems rely on analogous spectral characterizations (requiring the covariance operator to be trace-class). Thus, strategies such as BSPMI—minimizing the risk by focusing on the most uncertain directions—are directly aligned with foundational Bayesian optimality criteria.

5. Low-Rank Posterior Mean Approximations

The computational challenge of high-dimensional or infinite-dimensional Bayesian inference (e.g., PDE-constrained inversion) is addressed by low-rank approximations to the posterior mean and covariance (Carere et al., 31 Mar 2025). BSPMI in these settings can be understood as projecting the full estimation problem onto the “likelihood-informed subspace” spanned by dominant eigenpairs of the prior-preconditioned Hessian.

For structure-ignoring low-rank approximations, the optimal estimator is constructed as

$$A^{(2)}_r = \mathcal{C}^{1/2} \left[ \sum_{i=1}^r \sqrt{ -\lambda_i (1 + \lambda_i) } w_i \otimes \phi_i \right] \mathcal{C}^{-1/2}$$

with $(\lambda_i, w_i)$ the eigenpairs selecting the “most informed” directions. Approximation errors are measured using information-theoretic divergences (e.g., Renyi, Amari, Hellinger, KL) averaged over the data distribution, ensuring that only the dominant “directions of uncertainty” influence the estimator. Necessary and sufficient conditions for uniqueness reduce to spectral gaps in the eigenvalues—a property that carries over to practical BSPMI constructions.

6. Bias Reduction and Bayesian Estimator Properties

Asymptotic bias of the posterior mean may be eliminated via judicious selection of priors, notably the squared Jeffreys prior in regular statistical models (Yoichi et al., 29 Sep 2024). The first-order (O(1/n)) bias term in posterior mean estimation is removed if the prior $\pi_{BR}(\theta)$ satisfies

$$\partial [\log \pi(\theta)]/\partial \theta_j + \sum_{r,s} \kappa^{(r,s)} ( \kappa_{rsj} + \kappa_{r,js} ) = 0$$

where $\kappa$ encodes Fisher information and higher-order cumulants. The resulting bias-reducing prior is often $\propto |I(\theta)|$, the determinant of the Fisher information matrix (the squared Jeffreys prior).

For BSPMI, this translates to selection of prior distributions such that the posterior mean is asymptotically unbiased, a property that can yield lower-bias incumbents in practice—especially relevant in moderate sample regimes.

7. Multi-Objective and High-Dimensional Settings

Extensions of BSPMI appear in multi-objective pure exploration, where the goal is Pareto set identification (PSI) (Kone et al., 7 Nov 2024). In such settings, posterior sampling approaches generalize the BSPMI principle to multi-objective contexts via sample-based stopping and sampling rules. The PSIPS algorithm leverages BSPMI-related logic by declaring the Pareto set based on empirical means and using posterior samples to validate or challenge the incumbent.

Asymptotic optimality is maintained: posterior-based stopping rules and sample-complexity guarantees ensure that only the non-dominated (with respect to the posterior mean) candidates are eventually selected, and computational costs remain tractable.

---

These facets collectively establish BSPMI as a robust and flexible strategy for incumbent selection, estimation, and decision-making in Bayesian optimization, bandits, information economics, experimental design, and infinite-dimensional inverse problems. The concept is underpinned by rigorous statistical theory, spectral analysis, and information-theoretic loss measures, and is widely applicable in both established and emerging domains of statistical learning.