--- title: Bayesian Pooled Posterior Maximisation url: https://www.emergentmind.com/topics/bayesian-pooled-posterior-maximisation type: topic --- # Bayesian Pooled Posterior Maximisation Bayesian pooled posterior maximisation denotes a family of inferential constructions in which posterior information is generated separately and then recombined into a pooled posterior object, pooled decision rule, or pooled approximate posterior. In the cited literature, the expression is used in at least four technically distinct settings: a two-stage approximation for hierarchical Bayes based on combining independent source-specific posteriors; a linear-opinion-pool approach for likelihood-free inference (LFI); a Blackwell-theoretic upper-bound rule for multi-agent decision making based on the posterior under pooled private information; and a moment-matching procedure for approximate posteriors driven by massively parallel importance weighting [1603.09272][2212.02658][2605.06028][2503.08264]. A common feature is that pooling occurs at the posterior level rather than through a single monolithic likelihood evaluation, but the resulting object is not uniformly an exact Bayesian posterior in the original model. ## 1. Conceptual scope and unifying structure In hierarchical modeling, pooling is performed by first sampling each source-specific posterior independently and then treating those posterior draws as observed data in a substitute hierarchical model with a scaled likelihood [1603.09272]. In LFI, pooling is performed by combining multiple approximate posteriors through a linear opinion pool, with the weight chosen to optimize the asymptotic frequentist risk of the pooled posterior mean rather than by equal weighting [2212.02658]. In multi-agent decision making, pooled posterior maximisation refers to the decision rule that predicts with the posterior $\Pr(y \mid x, d_{1:M})$ based on the agents’ full pooled private information, which is presented as the information-theoretic upper bound under Blackwell’s ordering [2605.06028]. In approximate-posterior learning, QEM pools information from a reweighted set of latent samples into posterior moment estimates and then updates the approximate posterior by moment matching rather than gradient ascent [2503.08264]. These uses are related but not identical. In some cases the pooled object is a posterior distribution over latent parameters; in others it is a posterior-derived point estimator or action rule. This suggests that “Bayesian pooled posterior maximisation” functions less as the name of one standardized algorithm than as a broader design pattern: independent or modular posterior analyses are retained, and a second operation aggregates them to recover a joint decision, a shared hyperparameter structure, or a refined approximation. ## 2. Two-stage hierarchical Bayes via independent posterior combination The hierarchical-model formulation begins from \[ X_{ji} \sim F_{X|\theta}(\cdot \mid \theta_j), \qquad \theta_j \sim F_{\theta|\phi}(\cdot \mid \phi), \qquad \phi \sim F_\phi, \] with full posterior \[ \pi(\phi,\theta_1,\ldots,\theta_J \mid \mathbf{x}_1,\ldots,\mathbf{x}_J). \] Direct inference requires joint sampling of all $\theta_j$ and $\phi$, typically by Gibbs or other MCMC, and becomes expensive when $J$ is large, each source model is complex, or the posterior is high-dimensional or mixes poorly [1603.09272]. The proposed two-stage procedure, called **meta-analysis of Bayesian analyses (MBA)**, first fits each source independently and obtains posterior samples \[ \boldsymbol{\theta}_j^*=(\theta_{j1}^*,\ldots,\theta_{jL}^*) \] from $\pi(\theta_j \mid \mathbf{x}_j)$. It then defines \[ \psi_j \triangleq \mathbb{E}(\theta_j \mid \mathbf{x}_j), \] models $\psi_j$ hierarchically through $\psi_j \sim F_{\psi|\phi}(\cdot \mid \phi)$, and treats the posterior samples $\boldsymbol{\theta}_j^*$ as if they were data generated from a distribution centered at $\psi_j$. The substitute likelihood is \[ l^*(\psi_j;\boldsymbol{\theta}_j^*)= \prod_{l=1}^L \left[f^*(\theta_{jl}^* \mid \psi_j)\right]^{1/L} = \exp\left[\frac{1}{L}\sum_{l=1}^L \log f^*(\theta_{jl}^* \mid \psi_j)\right]. \] The exponent $1/L$ is critical: it prevents the substitute likelihood from becoming arbitrarily sharp as the number of posterior draws increases, and turns the product into a geometric average likelihood. The resulting substitute posterior is \[ \pi^*(\phi,\psi_1,\ldots,\psi_J \mid \boldsymbol{\theta}_1^*,\ldots,\boldsymbol{\theta}_J^*) \propto \pi(\phi)\prod_{j=1}^J \pi(\psi_j\mid \phi)\prod_{l=1}^L \left[f^*(\theta_{jl}^* \mid \psi_j)\right]^{1/L}. \] The update equations mirror the Gibbs structure of the full hierarchical model: \[ \phi \sim \pi(\phi \mid \psi_1,\ldots,\psi_J) \propto \pi(\phi)\prod_{j=1}^J \pi(\psi_j\mid \phi), \] \[ \psi_j \sim \pi(\psi_j \mid \phi,\boldsymbol{\theta}_j^*) \propto \pi(\psi_j\mid \phi)\,l^*(\psi_j;\boldsymbol{\theta}_j^*). \] The paper further justifies $l^*$ heuristically by showing that it approximates the original source-posterior structure, with the central approximation that the substitute likelihood behaves like the original data likelihood times the prior on source-specific parameters. Closed forms are available in conjugate settings. For multivariate normal source parameters with $\theta_j \sim \mathcal{N}_p(\mu,\Sigma)$ and $\theta_{jl}^* \sim \mathcal{N}_p(\psi_j,S_j)$, the conditional posterior is $\psi_j \sim \mathcal{N}_p(\mu',\Sigma')$ with \[ \Sigma' = (\Sigma^{-1}+S_j^{-1})^{-1}, \qquad \mu' = \Sigma' \left(\Sigma^{-1}\mu + S_j^{-1}\frac{1}{L}\sum_{l=1}^L \theta_{jl}^*\right). \] An inverse Wishart analogue is also given, and these closed forms make the second-stage pooling very fast. The computational rationale is explicit. Source-wise inference is embarrassingly parallel; the source-specific subproblems are lower-dimensional and often converge faster; the second-stage reconstruction is usually simple and conjugate; and direct full-hierarchy bottlenecks are avoided. In simulation, for a normal-normal hierarchy with $J=10$ and $n_j=5$, MBA and full hierarchical Bayes had very similar MSEs and credible interval coverage, while MBA was about twice as fast on average, with FHM at $0.49\,$s and MBA at $0.29\,$s. In an inverse Wishart hierarchy with $J=20$, FHM was around $0.42\,$s and MBA around $0.17\,$s. In the retail cheese sales example with weekly sliced cheese sales in $88$ retail stores, full hierarchical inference in Stan/HMC took about **1 hour**, whereas the combined MBA analysis took about **5 seconds** for $1000$ posterior draws after burn-in, roughly **300-fold** faster, while producing posterior distributions for $(\mu,\Sigma)$ close to those from full hierarchical Bayes [1603.09272]. ## 3. Risk-optimised posterior pooling in likelihood-free inference In LFI, pooled posterior maximisation is introduced to avoid committing to a single summary statistic vector, discrepancy, or even a single LFI algorithm [2212.02658]. Standard LFI approximates a partial posterior $\pi(\theta\mid S)$ using summaries $S$, but different summaries can yield different posterior centers, variances, and misspecification behavior, while high-dimensional summary vectors are costly and statistically unstable because of the curse of dimensionality. For two summary sets $S_1$ and $S_2$, the pooled posterior is defined by the linear opinion pool \[ \widetilde\pi_\omega(\theta\mid S) := (1-\omega)\widetilde\pi(\theta\mid S_1)+\omega\widetilde\pi(\theta\mid S_2), \qquad \omega\in[0,1]. \] Its posterior mean is \[ \bar\theta(\omega) = E_{\widetilde\pi_\omega}[\theta] = (1-\omega)\mu_1+\omega\mu_2, \qquad \mu_j := E(\theta\mid S_j). \] The method is not standard Bayesian updating on a joint likelihood from $(S_1,S_2)$; it is a mixture of posterior approximations. It is also not ordinary equal-weight posterior averaging, because the weight is selected to optimize asymptotic risk. Because the plain linear pool can inflate posterior variance when component posteriors have different locations, a recentered pool is also defined. The recentered construction preserves the pooled posterior mean but yields \[ \mathrm{Var}_c(\theta\mid S) = (1-\omega)\mathrm{Var}(\theta\mid S_1)+\omega\mathrm{Var}(\theta\mid S_2), \] with $\mathrm{Var}_c(\theta\mid S)\leq \mathrm{Var}(\theta\mid S)$ in the positive-semidefinite sense. For uncertainty quantification, the paper recommends the recentered pool. The optimization criterion is the asymptotic expected loss of the pooled posterior mean, \[ \mathcal{R}_{0}(\omega) := \lim_{\nu\rightarrow\infty}\liminf_{n\rightarrow\infty} E\left[\min\{nL\{\theta_0,\overline\theta(\omega)\},\nu\}\right], \] which reduces to asymptotic mean squared error under quadratic loss. Under asymptotic normality of the summaries, injectivity and regularity of the summary mean map, a BvM-type result for each component posterior, and smooth loss, the paper derives an exact optimal weight $\omega^\star_+$ and proves \[ \mathcal{R}_0(\omega^\star_+)\le \min\{\mathcal{R}_0(S_1),\mathcal{R}_0(S_2)\}. \] A particularly important result concerns incompatibility: if one summary set is compatible and another is incompatible, then asymptotically the pooled posterior discards the incompatible one, with \[ \omega^\star=0 \quad\text{and}\quad \widetilde\omega=o_p(1). \] Because estimating the covariance term $\Omega_\Sigma$ is hard and noisy, two practical alternatives are recommended: \[ \widehat\omega :=\frac{tr \overline{H}\,\overline\Sigma_1}{tr \overline{H}\,\overline\Sigma_1+tr \overline{H}\,\overline\Sigma_2}, \] and \[ \widetilde\omega :=\frac{tr\overline{H}\,\overline\Sigma_1}{(\bar\theta_1-\bar\theta_2)^\top (\bar\theta_1-\bar\theta_2)+tr\overline{H}\,\overline\Sigma_1+tr\overline{H}\,\overline\Sigma_2}. \] These require only posterior draws from the component LFI posteriors. The paper remarks that estimating $\Omega_\Sigma$ can hurt finite-sample performance, a phenomenon described as akin to the **forecast combination puzzle**. The method is explicitly designed as a wrapper around existing algorithms, including **ABC**, **Bayesian synthetic likelihood (BSL)**, and discrepancy-based ABC methods such as Wasserstein ABC. Empirically, in the g-and-k model, the pooled posterior using $\widetilde\omega$ achieved total MSE $0.0655$, compared with $0.1856$ for $S_1$, $0.0690$ for $S_2$, and $0.0617$ for the expensive joint analysis. In the stochastic volatility model, pooled methods substantially beat both individual posteriors and the joint posterior, with total MSE $0.0301$ for $\widehat\omega$ and $0.0302$ for $\widetilde\omega$, versus $0.0639$ for $S_1$, $0.0574$ for $S_2$, and $0.0549$ for the joint summary analysis. In M/G/1, pooling also improved over either BSL or Wasserstein ABC alone, with total MSE $0.4393$ for $\widetilde\omega$ versus $0.5603$ for $S$ and $0.5498$ for $\mathcal{D}$ [2212.02658]. ## 4. Blackwell-optimal aggregation in multi-agent decision making In the multi-agent setting, pooled posterior maximisation is formulated through Blackwell’s informativeness framework [2605.06028]. A decision problem is defined as $(\mathcal{S}, \mathcal{A}, \varphi, \rho)$, where $\mathcal{S}$ is the state space, $\mathcal{A}$ the action space, $\varphi(a,s)$ the utility of action $a$ in state $s$, and $\rho(s)$ the prior on states. An information structure is $(\mathcal{D}, \sigma)$ with signal space $\mathcal{D}$ and conditional signal distribution \[ \sigma(d \mid s)=\Pr(d \mid s). \] Given the prior, the posterior is \[ \Pr(s \mid d)=\frac{\sigma(d \mid s)\rho(s)}{\Pr(d)}. \] The expected value of an information structure under optimal decision making is \[ \omega(\sigma)= \sum_{d \in \mathcal{D}} \Pr(d)\, \max_{a \in \mathcal{A}} \sum_{s \in \mathcal{S}} \varphi(a,s)\Pr(s \mid d). \] If $(\mathcal{D}',\sigma')$ is a garbling of $(\mathcal{D},\sigma)$, written \[ (\mathcal{D}',\sigma') \unlhd (\mathcal{D},\sigma), \] then every Bayesian decision maker weakly prefers $(\mathcal{D},\sigma)$ to $(\mathcal{D}',\sigma')$, so $\omega(\sigma') \le \omega(\sigma)$. The paper maps multi-agent QA and classification into this framework by identifying the state with the true label $y$, the action with the prediction $\hat y$, and each agent’s private information with a signal $d_m$. The joint private information is $d_{1:M}$, with information structure $(\mathcal{D}^M,\sigma_0(d_{1:M}\mid y,x))$. Any aggregation \[ g=\gamma(d_1,\dots,d_M) \] induces a garbling of the pooled private information, because $g$ is a deterministic function of $d_{1:M}$. Hence ensemble voting rules and multi-round debate are both less informative than access to the full pooled private information. The resulting optimal decision rule is \[ \hat{y}^{*} = \operatorname*{argmax}_{y \in \mathcal{Y}} \Pr(y \mid x, d_{1:M}), \] which the paper explicitly identifies as the **maximum Bayesian pooled posterior**. The upper-bound claim is decision-theoretic rather than purely empirical: because any aggregation of private information is a garbling, no voting or debate protocol can dominate access to the pooled source under Blackwell’s ordering. For practical approximation, the paper assumes conditional independence of agents’ private information given label and input, \[ d_m \perp d_{m'} \mid y,x \qquad (m \neq m'), \] and a uniform prior $\Pr(y \mid x)$. Then \[ \Pr(y \mid x, d_{1:M}) \propto \prod_{m=1}^{M}\Pr(y \mid x, d_m). \] This yields the **product-of-posteriors estimator** \[ \Pr(y \mid x, d_{1:M}) \approx \prod_{m=1}^{M}\widetilde{\Pr}(y \mid x, d_m). \] The practical algorithm, **MA-PoP** (Multi-Agent Pooled Posterior), estimates each agent’s posterior by Monte Carlo response sampling. For each agent $m$, it samples $N$ responses $y_n^{(m)} \sim \Pr(y \mid x, d_m)$ and estimates answer-option probabilities through semantic similarity: \[ \widetilde{\Pr}(y = y_{o_j} \mid x, d_m) = \frac{1}{N}\sum_{n=1}^{N} \frac{ \phi\!\left(\tau(y_n^{(m)}, y_{o_j})\right) } { \sum_{j'=1}^{J}\phi\!\left(\tau(y_n^{(m)}, y_{o_{j'})\right) }. \] The method uses an **NLI cross-encoder**—specifically `nli-deberta-v3-large`—for semantic compatibility scoring, and a **Deep Sets** calibration head to ensure permutation equivariance with respect to option ordering. On six QA benchmarks—**MMLU Professional Medicine**, **MMLU Formal Logic**, **HellaSwag**, **CommonsenseQA**, **HH-RLHF**, and **MedMCQA**—MA-PoP outperformed single-agent baselines, majority voting, self-consistency, log-linear opinion pooling, inverse surprising popularity, and several MAD variants. In the 5-agent heterogeneous benchmark table, MA-PoP achieved accuracy **0.8787** on MMLU Professional Medicine, **0.7367** on MMLU Formal Logic, **0.8433** on HellaSwag, **0.8800** on CSQA, **0.5900** on HH-RLHF, and **0.6467** on MedMCQA. Calibration improved markedly with the Deep Sets head; on MedMCQA, Falcon-7B ECE dropped from **0.2740** to **0.0871**, and Falcon-34B ECE dropped from **0.1973** to **0.0305**. The paper also reports that increasing Monte Carlo samples from $N=1$ to $N=5$ improves performance, after which gains saturate, and uses $N=5$ as a practical balance [2605.06028]. ## 5. Moment-pooled approximate posterior fitting with QEM QEM reframes approximate-posterior learning as an EM-like procedure in which the E-step computes posterior moments using massively parallel importance weighting (MPIW), and the M-step fits the approximate posterior by setting its moments equal to those estimates [2503.08264]. The key object is the posterior \[ p(z' \mid x)=\frac{p(x \mid z')p(z')}{p(x)}, \qquad p(x)=\int p(x \mid z')p(z')\,dz', \] and the target of the E-step is $\mathbb{E}_{p(z' \mid x)}[m(z')]$ for suitable moment functions. In the massively parallel setting, each latent variable $z_i$ has $K$ samples, and the method considers all $K^n$ combinations. The MPIW estimator produces pooled posterior moment estimates from this reweighted sample set. QEM then replaces gradient ascent with direct moment matching. For an exponential-family approximate posterior \[ \log Q(z';\eta)=\eta^\top T(z') - A(\eta), \] the mean parameters are \[ \mu = \mathbb{E}_{Q(z';\eta)}[T(z')]. \] Instead of updating natural parameters by a gradient step, QEM updates mean parameters through \[ \Delta \mu = \lambda\left(\text{one iter}(z)-\mu\right), \] equivalently \[ m_t = (1-\lambda)m_{t-1}+\lambda\,\text{one iter}_t. \] The paper emphasizes that QEM is **not standard EM**: standard EM updates model parameters in the prior or likelihood, whereas QEM updates the parameters of the approximate posterior $Q$. The supported approximate posterior families explicitly include Gaussian, Gamma, Beta, Dirichlet, Binomial, Multinomial, Categorical, and combinations thereof. For a scalar Gaussian, the update is given through the first two moments: \[ \widehat{\mu} = \widehat{\mathbb{E}[Z]}, \qquad \widehat{\sigma}^2 = \widehat{\mathbb{E}[Z^2]-\widehat{\mu}^2. \] For Beta and Gamma families, the fitting uses moment or log-moment statistics; for Dirichlet, expected log-components; and for discrete exponential families, posterior category or success probabilities. QEM is presented as a form of Bayesian pooled posterior maximisation because it pools information from reweighted latent samples into posterior moment estimates and updates the approximate posterior by moment matching rather than ELBO gradient optimization. The method is designed to exploit conditional-independence structure, with appendix-level memory reduction from \[ O(K^n) \quad\text{to}\quad O(K^{\max_i |\mathcal P_i|}). \] An asymptotic theorem states that if $\widehat m_t$ is unbiased with finite variance and \[ \lambda(t)=1-t^{-p}, \qquad 0