--- title: Expert-Tilted Bayes Posterior url: https://www.emergentmind.com/topics/expert-tilted-bayes-posterior type: topic --- # Expert-Tilted Bayes Posterior In the cited literature, the phrase *Expert-Tilted Bayes Posterior* is used for several related but non-identical constructions in which a baseline Bayesian update is reweighted, transported, or adjusted by additional expert structure. The added structure may be a denoising oracle and a quadratic likelihood tilt in a linear inverse problem, a Bayes-linear adjustment across alternative prior and likelihood judgements, a moment-based surrogate likelihood, a test-derived loss, a new set of class priors, or a tilt on a random probability measure [2407.00745] [1512.00969] [2109.07792]. What unifies these constructions is not a single canonical density, but the use of an explicit tilting mechanism to move posterior inference toward a target regarded as more faithful, more robust, more computationally tractable, or more decision-relevant than a single unmodified Bayes posterior. ## 1. Conceptual scope and unifying structure Several mathematically distinct objects appear under this label or a closely allied interpretation. | Construction | Update form | Salient property | |---|---|---| | Tilted transport | $\nu_t(x) \propto \pi_t(x)\exp\!\left(-\frac12 x^\top Q_t x + x^\top b_t\right)$ | Exact reverse diffusion maps $\nu_t$ to $\nu$ | | Posterior belief assessment | $\mathbb{E}_G[Y] = \mathbb{E}[Y] + \mathrm{Cov}(Y,G)\mathrm{Var}(G)^{-1}(G-\mathbb{E}[G])$ | Closer in mean-square to $P_t(Y)$ | | PETEL / BETEL | $\pi(\theta)\,L(\theta)\,\exp\{-\alpha_n R_n(\theta)\}$ or $\pi(\theta)\prod_i p_i(\theta,\lambda(\theta))$ | Moment-based Bayes or generalized Bayes update | | e-posterior / robust-test posterior | $\bar P(\theta\mid y)=1/S_\theta(y)$ or $d\Pi_X(P)\propto \exp\{-\beta \bar T(X,P)\}d\Pi(P)$ | Frequentist-valid risk bounds or non-asymptotic concentration | In the transport formulation, the expert contribution is explicit in the score decomposition: the denoising oracle provides an “expert” score for the prior, and the likelihood contributes a quadratic tilt that is itself an “expert” favoring consistency with the data [2407.00745]. In posterior belief assessment, the tilt is not a literal density transformation but an expectation-level correction obtained from a finite collection of alternative Bayesian analyses judged “not ruled out”; it does not assign model probabilities and does not assume one alternative is true [1512.00969]. In PETEL, BETEL, and RBETEL, the tilt replaces or augments the likelihood by exponentially tilted empirical likelihood terms, often with an additional penalty or contamination mechanism [2109.07792] [1801.00243]. In the e-posterior literature, the object is explicitly *not* a probability distribution: it is a reciprocal e-density-like quantity used to construct minimax-safe decision rules and risk certificates [2301.01335]. This diversity suggests that the phrase denotes a class of posterior modifications rather than a single standardized posterior. Some variants preserve Bayesian semantics exactly, some produce a single adjusted posterior judgement, and some are Bayes-like devices whose primary output is calibrated uncertainty or worst-case risk control. ## 2. Tilted transport in Bayesian inverse problems For the Bayesian linear inverse problem $$ y = A x + \varepsilon, \qquad \varepsilon \sim \mathcal N(0,\sigma^2 I_{d'}), $$ with prior density $\pi$ on $x\in\mathbb R^d$, the Bayes posterior is $$ p(x\mid y)\propto \pi(x)\exp\!\left(-\frac{1}{2\sigma^2}\|Ax-y\|^2\right). $$ Writing $Q:=\sigma^{-2}A^\top A$ and $b:=-\sigma^{-2}A^\top y$, this becomes the quadratic tilt $$ \nu := \mathbb T_{Q,b}\pi,\qquad d\nu \propto \exp\!\left(-\frac12 x^\top Qx + x^\top b\right)\pi(x)\,dx. $$ The score-based denoising oracle supplies an approximation $s(x)\approx \nabla\log\pi(x)$, and in the variance-preserving Ornstein–Uhlenbeck setting the time-dependent prior score satisfies Tweedie’s formula. Tilted transport then defines a boosted family $$ \nu_t := \mathbb T_{Q_t,b_t}\pi_t, $$ where $(Q_t,b_t)$ evolve according to $$ \dot Q_t = 2(I+Q_t)Q_t,\qquad Q_0=Q, $$ $$ \dot b_t = (I+2Q_t)b_t,\qquad b_0=b. $$ If $X_t\sim \nu_t$ and one runs the OU reverse SDE from $t$ to $0$, then $X_0$ has law $\nu$; sampling $\nu$ is therefore equivalent to sampling the boosted posterior and reversing the diffusion [2407.00745]. The boosted posterior has log-density $$ \log \nu_t(x)=\log \pi_t(x)-\frac12 x^\top Q_t x + x^\top b_t + C_t, $$ with gradient $$ \nabla\log \nu_t(x)=\nabla\log \pi_t(x)-Q_t x+b_t. $$ The paper interprets this as a combination of two experts: $$ \nabla\log p(x\mid y)=\nabla\log\pi(x)-\frac{1}{\sigma^2}A^\top(Ax-y), $$ and at transport time $t$, $$ \nabla\log \nu_t(x)=s_t(x)-Q_t x+b_t. $$ The likelihood-derived quadratic tilt boosts curvature by $-Q_t$, while $\pi_t$ becomes smoother and closer to unimodal as $t$ increases. The method quantifies the role of the signal-to-noise ratio $$ \mathrm{SNR}:=\lambda_{\min}(Q)=\lambda_{\min}(A)^2/\sigma^2 $$ and the condition number $\kappa(A):=\lambda_{\max}(A)/\lambda_{\min}(A)$ in determining posterior difficulty [2407.00745]. The principal theoretical guarantee is a strong log-concavity condition for the boosted posterior at the blowup time $T$. With susceptibility $$ \chi_t(\pi):=\sup_{x\in\mathbb R^d}\left\| \mathrm{Cov}\!\left[\mathbb T_{tI,tx}\pi\right]\right\|, $$ and $\kappa(Q)=\kappa(A)^2$, strong log-concavity holds if $$ \chi_{\|Q\|}(\pi) < \|Q\|^{-1}\cdot \frac{\kappa}{\kappa-1}. $$ Under this condition, the strong convexity parameter satisfies $m>0$, Langevin dynamics converges in KL at rate $e^{-2\rho t}$ with $\rho\ge m$, and ULA, MALA, or HMC can be used with standard controls. For Ising models, $\chi_t(\pi)=1$, so the criterion reduces to $$ \lambda_{\max}(Q)-\lambda_{\min}(Q)<1, $$ which matches the Kunisky low-degree computational threshold and the rapid mixing regime of Glauber dynamics. The paper further reports Gaussian-mixture experiments with mixtures of 25 Gaussians in dimensions $d=20,40,80$, $d'=d$, $\kappa(A)=20$, and $\mathrm{SNR}\in[10^{-5},10^{-1}]$, as well as scalar field $\varphi^4$ experiments on a $128\times 128$ lattice, where tilted transport accelerates relaxation and yields a direct Bakry–Émery guarantee for $\beta<0.48$ [2407.00745]. ## 3. Posterior belief assessment as expectation-level expert tilting Posterior belief assessment addresses complex problems in which no single prior and likelihood pair can be claimed to represent the analyst’s actual probabilistic judgements. The construction begins with a quantity of interest $Y$, observed data $D$, and several alternative modelling judgements $J_0,J_1,\dots,J_K$ that are “not ruled out” and regarded as representative of prior knowledge and problem structure. One computes the posterior expectations under these alternatives and forms $$ G=\big(E[Y\mid D,J_0],E[Y\mid D,J_1],\dots,E[Y\mid D,J_K]\big). $$ The posterior belief assessment is then the Bayes-linear projection $$ \mathbb E_G[Y] = \mathbb E[Y] + \mathrm{Cov}(Y,G)\,\mathrm{Var}(G)^{-1}\big(G-\mathbb E[G]\big), $$ with adjusted variance $$ \mathrm{Var}(Y\mid G) = \mathrm{Var}(Y)-\mathrm{Cov}(Y,G)\,\mathrm{Var}(G)^{-1}\,\mathrm{Cov}(G,Y). $$ This construction is explicitly expectation-based rather than model-probability-based, and the paper states that it is not Bayesian model averaging: no model probabilities are assigned, no assumption is made that one $J_k$ is true, and the weights are determined by second-order relationships to optimize mean-square closeness to the future posterior prevision $P_t(Y)$ [1512.00969]. The foundational device is the Temporal Sure Preference principle. If $P_t(Y)$ denotes the future posterior prevision after seeing $D$, then one obtains the orthogonal decomposition $$ X = (X-P_t(X)) \oplus (P_t(X)-E[X\mid D]) \oplus (E[X\mid D]-E[X]) \oplus E[X], $$ and the corresponding variance partition $$ \mathrm{Var}(X) = \mathrm{Var}(X-P_t(X)) + \mathrm{Var}(P_t(X)-E[X\mid D]) + \mathrm{Var}(E[X\mid D]). $$ From this, the paper proves that for each component $Y_i$, $$ \mathbb E\big[(Y_i-\mathbb E_G[Y_i])^2\big] \le \mathbb E\big[(Y_i-\mathbb E[Y_i\mid D,J_0])^2\big], $$ and similarly $$ \mathbb E\big[(P_t(Y_i)-\mathbb E_G[Y_i])^2\big] \le \mathbb E\big[(P_t(Y_i)-\mathbb E[Y_i\mid D,J_0])^2\big]. $$ The adjusted judgement is therefore at least as close in mean square to the analyst’s future posterior prevision as the original single Bayesian analysis [1512.00969]. When infinitely many alternatives are contemplated, the framework uses second-order exchangeability and co-exchangeability. Within each class $J(i)$, $$ E[Y\mid D,J_{i_j}] = M(J_i)+R_j(J_i), $$ and Bayes-linear sufficiency implies $$ \mathbb E[E[Y\mid D,J_{i_j}] \mid D] = \mathbb E[M(J_i)\mid D]. $$ This reduces an infinite family of alternatives to a finite set of class means. The methodology was illustrated on calibration of the NEMO ocean model, where the target quantity was global mean temperature at approximately $1405$ m depth. The reported result was $$ E_G[y(d5)] = 2.921^\circ\mathrm C, $$ compared with the original $$ E[y(d5)\mid z(d1{:}4),J_0] = 2.951^\circ\mathrm C. $$ The adjusted variances were reported as $0.0262$ for the posterior belief assessment and $0.0226$ under the single analysis, with an uncertainty reduction lower bound of approximately $14\%$; the weights indicated that high-discrepancy classes contributed most to the tilt [1512.00969]. ## 4. Exponentially tilted empirical likelihood and robust moment restrictions A second major usage arises in generalized Bayes procedures that replace a misspecified or unavailable likelihood by exponentially tilted empirical likelihood. For i.i.d. observations $D=\{(X_i,Y_i)\}_{i=1}^n$, parameter $\theta\in\Theta\subset\mathbb R^d$, and loss $\ell(Y,X;\theta)$, the empirical risk minimizer is $$ \hat\theta = \arg\min_{\theta\in\Theta} R_n(\theta), \qquad R_n(\theta)=\frac1n\sum_{i=1}^n \ell(Y_i,X_i;\theta). $$ Using moment conditions $$ g_i(\theta):=\nabla_\theta \ell(Y_i,X_i;\theta), $$ the exponentially tilted empirical likelihood weights are $$ p_i(\theta,\lambda) = \frac{\exp\{\lambda^\top g_i(\theta)\}} {\sum_{j=1}^n \exp\{\lambda^\top g_j(\theta)\}}, $$ where $\lambda(\theta)$ solves $$ \lambda(\theta) = \arg\min_{\xi\in\mathbb R^d} \left\{\sum_{i=1}^n \exp\big(\xi^\top g_i(\theta)\big)\right\}. $$ The Penalized ETEL posterior is $$ \pi_{\mathrm{PE}}(\theta\mid X_{1:n}) \propto \pi(\theta)\,L(X_{1:n};\theta)\,\exp\{-\alpha_n R_n(\theta)\}. $$ Under smoothness and uniqueness assumptions, if $$ C_1\log n \le \alpha_n \le C_2\sqrt{n\log n}, $$ then the PETEL posterior satisfies a Bernstein–von Mises theorem: $$ d_{\mathrm{TV}} \!\left( \pi_{\mathrm{PE}}(\cdot\mid X_{1:n}), \, \mathcal N\!\big(\hat\theta,n^{-1}V_{\theta^*}\big) \right) \le C\sqrt{\frac{\log n}{n}}, $$ with sandwich covariance $$ V_{\theta^*}=H_{\theta^*}^{-1}\Delta_{\theta^*}H_{\theta^*}^{-1}. $$ The credible ellipsoid based on the posterior mean and covariance has frequentist coverage error bounded by $$ C_3\frac{(\log n)^{3/2}}{\sqrt n}. $$ The paper extends this to non-smooth losses and to a sparse high-dimensional model-averaged PETEL posterior with variable-selection consistency [2109.07792]. BETEL and RBETEL place the same exponential tilting within moment-condition models. For data $x_{1:n}$, parameter $\theta\in\Theta\subseteq\mathbb R^p$, and moment restrictions $$ \mathbb E^F[g(X,\theta)] = 0, $$ ETEL arises from maximizing entropy subject to the moment constraints. The implied probabilities are $$ p_i(\theta,\lambda) = \frac{\exp\{\lambda^\top g(x_i,\theta)\}} {\sum_{j=1}^n \exp\{\lambda^\top g(x_j,\theta)\}}, $$ and the BETEL posterior is $$ \pi(\theta\mid x_{1:n}) \propto \pi(\theta)\prod_{i=1}^n p_i(\theta,\lambda(\theta)). $$ RBETEL adds latent indicators $s_i\in\{0,1\}$ for “good” observations, imposes the moment conditions only on the active subset, and uses the joint posterior $$ \pi_{\mathrm{RBETEL}}(\theta,s,v\mid x) \propto \pi(\theta)\,\pi(v)\,\pi(s\mid v)\, \prod_{i=1}^n \left(\widetilde w_i(\theta,s)\sum_{j=1}^n s_j\right)^{s_i} \mathbf 1\!\left(\sum_{i=1}^n s_i>\frac n2\right). $$ The majority constraint $\sum_i s_i>n/2$ enforces domination by the “good” subset, and the robust loss is constructed so that contaminated observations can be assigned $s_i=0$ and receive zero implied probability. In the empirical illustration relating brain weight to body weight, RBETEL assigned low posterior probabilities of being “good” to three leverage outliers, approximately $0.053$, $0.032$, and $0.001$, while producing posterior means close to robust M-estimation and sharper posteriors than a parametric Student-$t$ error model [1801.00243]. ## 5. Decision-theoretic and test-based Bayes-like tilts Another line of work constructs Bayes-like posteriors directly from robust tests. Let $(M,\mathcal A)$ be a model of candidate probability measures, endowed with prior $\Pi$, and let $l$ be a loss such as total variation, squared Hellinger, or an $\mathbb L_j$ loss. For each pair $(P,Q)\in M\times M$, one defines a bounded antisymmetric per-observation statistic $t(P,Q)(x)$ and aggregates it over the sample: $$ T(X,P,Q)=\sum_{i=1}^n t(P,Q)(X_i). $$ With tuning parameters $\beta>0$ and $\lambda>0$, the construction first defines $$ d\Pi_X(Q\mid P)\propto \exp\{\lambda T(X,P,Q)\}\,d\Pi(Q), $$ then the averaged score $$ \bar T(X,P)=\int T(X,P,Q)\,d\Pi_X(Q\mid P), $$ and finally the posterior $$ d\Pi_X(P)\propto \exp\{-\beta \bar T(X,P)\}\,d\Pi(P). $$ The paper proves non-asymptotic concentration under local prior growth conditions. Under Assumption 3 alone, $$ \mathbb E\,\Pi_X\big(B(P^*,K_0 r)\big)\le 2e^{-\xi}, $$ and with the variance assumption one obtains the sharper radius involving $r_n(\beta,P)$. The theorems require only independence, not identical distribution, so the posterior remains stable around the average law when the data are independent but non-i.i.d. The paper gives explicit constants, including $K_0=220$ for total variation with $c=1/3$ and $y=1/100$, and $K_0=1694$ for squared Hellinger with $c=1/125$ and $\beta=1/500$ [2107.12011]. The e-posterior is more radical. For an e-collection $\mathcal S=\{S_\theta:\theta\in\Theta\}$ satisfying $$ \mathbf E_{Y\sim P_\theta}[S_\theta]\le 1, $$ the e-posterior is defined by $$ \bar P(\theta\mid y):=\frac{1}{S_\theta(y)}, $$ with the convention $1/0=\infty$. The paper states explicitly that this is not a probability distribution. Its basic decision object is $$ \bar R(y,a)=\sup_{\theta\in\Theta}\bar P(\theta\mid y)\,L(\theta,a), $$ and Proposition 1 gives the validity bound $$ \mathbf E_{Y\sim P_\theta} \left[ \frac{L(\theta,\delta(Y))}{\bar R(\delta)} \right] \le 1. $$ Under the Savage–Dickey construction, $$ S_\theta^{\mathrm{sd}}(y)=\frac{p_W(y)}{p_\theta(y)}, \qquad \bar P^{\mathrm{sd}}(\theta\mid y) = \frac{p_\theta(y)}{p_W(y)} = \frac{w(\theta\mid y)}{w(\theta)}, $$ so the e-posterior becomes the posterior-to-prior ratio. The paper emphasizes that the same e-posterior can support arbitrary loss functions chosen post hoc, and that generalized Savage–Dickey e-processes preserve validity under optional stopping [2301.01335]. These two constructions differ sharply from density-based Bayes updates. The robust-test posterior is still a posterior measure on the model, but its energy is test-derived rather than likelihood-derived. The e-posterior is an e-density-like object designed for worst-case decision-making. Both place robustness and finite-sample validity ahead of standard posterior semantics. ## 6. Prior adaptation, cold posteriors, nonparametric tilts, and posterior mean bounds In multiclass classification under prior probability shift, expert tilting can be completely explicit. If the original class priors are $\pi_k=p(y=k)$, the original posteriors are $q_k(x)=p(y=k\mid x)$, and the expert supplies new priors $\pi_k^{(E)}$, then, under unchanged class-conditionals, $$ p^{(E)}(y=k\mid x) = \frac{(\pi_k^{(E)}/\pi_k)\,q_k(x)} {\sum_j (\pi_j^{(E)}/\pi_j)\,q_j(x)}. $$ The paper proves that the per-example likelihood vector $p(x\mid y=\cdot)$ is uniquely determined up to a positive scale by the original posteriors and priors, using a Perron–Frobenius argument for a positive matrix $A=M+I$ whose dominant eigenvector spans the nullspace of the associated homogeneous system. Computationally, the update is $O(K)$ per example, vectorizable, and equivalent at the logit level to adding the bias $\log(\pi_k^{(E)}/\pi_k)$ before softmax [2007.01386]. In Bayesian deep learning, tempering itself is interpreted as an expert tilt. With posterior energy $$ U(\theta):=-\sum_{i=1}^n \log p(y_i\mid x_i,\theta)-\log p(\theta), $$ the fully tempered posterior is $$ p_T(\theta\mid D)\propto p(D\mid\theta)^{1/T}p(\theta) \propto \exp(-U(\theta)/T). $$ The paper defines a cold posterior as the case in which the best posterior predictive performance is achieved at $T<1$. On ResNet-20 for CIFAR-10, both accuracy and test cross-entropy improved markedly for $T<0.1$, continuing down to temperatures as small as $10^{-4}$; on a CNN-LSTM for IMDB, the best test accuracy and cross-entropy were attained for $0.01n/2$ [2109.07792] [1801.00243]. The robust-test posterior assumes a loss with triangle-type behavior, bounded antisymmetric tests, and local prior growth conditions; the e-posterior assumes a suitably chosen e-collection and a loss satisfying a no-sure-gain condition, but its bounds can become loose if the e-collection is badly chosen even though they remain valid [2107.12011] [2301.01335]. Other variants impose narrower structural assumptions. Prior-shift posterior adaptation requires unchanged class-conditionals between training and deployment, strictly positive priors, calibrated original posteriors, and the same label set [2007.01386]. Cold posteriors improve predictive performance in the reported deep-learning experiments, but the paper emphasizes that they are not coherent posteriors under the original model, and that $T=1$ is restored as optimal in well-specified synthetic settings [2002.02405]. CRM tilting requires the integrability conditions ensuring finite total mass and well-defined normalization [1312.5137]. Taken together, these results suggest a precise but plural meaning of *Expert-Tilted Bayes Posterior*. In one group of papers, the tilt is an exact posterior transformation that preserves Bayesian semantics, as in tilted transport or conjugate tilting of CRMs. In a second group, the tilt is an expectation-level or empirical-likelihood correction designed to better align inference with expert judgement, moment restrictions, or contamination structure. In a third group, the tilt produces Bayes-like or nonprobabilistic objects whose central output is calibrated concentration or risk control rather than a conventional posterior density. Across all of these uses, “expert tilt” denotes the deliberate insertion of additional structure into posterior updating, with the aim of improving tractability, robustness, calibration, or fidelity to the decision maker’s actual judgements. Source: https://www.emergentmind.com/topics/expert-tilted-bayes-posterior