---
title: Integral Updating of Posteriors
url: https://www.emergentmind.com/topics/integral-updating-of-posteriors
type: topic
---

# Integral Updating of Posteriors

Searching arXiv for the cited papers and closely related work to ground the article.
Integral updating of posteriors denotes a family of Bayesian and Bayesian-adjacent update procedures in which the posterior is obtained only after an explicit integration step. Depending on context, that integral may normalize an exponentially tilted density, produce the predictive density in sequential Bayes, marginalize latent parameters or hyperparameters in hierarchical models, or aggregate experiment-specific information before a single update. Across these uses, the common structure is that posterior formation is not merely multiplicative; it requires an integral that either enforces normalization or averages over uncertainty carried by prior, latent, or auxiliary quantities [2602.01573; 1308.2791; 2508.01572; 2412.20416].

## 1. Conceptual scope

The expression has no single universal meaning. In generalized Bayes, it refers to the normalization integral that turns an exponential tilt of a baseline distribution into a probability measure. In recursive Bayes, it refers to the predictive normalizer
\[
Z_t=\int p(\theta\mid y_{1:t-1})\,p(y_t\mid \theta)\,d\theta,
\]
which updates the previous posterior into the next one. In hierarchical models, it denotes marginalization over latent parameters or hyperparameters. In objective Bayes with heterogeneous experiments, it can denote a revised update in which information measures are aggregated first and Bayes’ theorem is applied once with a joint noninformative prior [2602.01573; 2508.01572; 2412.20416; 1308.2791].

| Usage | Representative form | Role of the integral |
|---|---|---|
| Loss-based/generalized Bayes | \(\pi(\theta\mid x)\propto \pi(\theta)e^{-\eta L(\theta;x)}\) | Partition function \(Z(x)\) normalizes the tilt |
| Sequential Bayes | \(p(\theta\mid y_{1:t})\propto p(\theta\mid y_{1:t-1})p(y_t\mid\theta)\) | \(Z_t\) is the predictive density |
| Hierarchical Bayes | \(p(\eta\mid D)\propto p(\eta)\prod_i \int p(\theta_i\mid\eta)p(D_i\mid\theta_i)\,d\theta_i\) | Marginalizes dataset-level parameters |
| Objective-prior aggregation | \(p(\theta\mid D_{1:J})\propto \sqrt{\det I_{\text{total}}(\theta)}\prod_j L_j(D_j\mid\theta)\) | Builds a joint noninformative prior from aggregated information |

This breadth has interpretive consequences. In some settings the integral has evidential status analogous to marginal likelihood; in others it is only a normalization device or the value of a variational problem. A central theme of recent work is that identical algebraic forms can encode different objects: conditional beliefs, randomized decision rules, predictive updates, or latent-variable marginalizations [2602.01573].

## 2. Exponential tilting, generalized Bayes, and the status of the normalizing integral

A canonical loss-based update takes the form
\[
\pi(\theta\mid x)=\frac{\pi(\theta)\exp\{-\eta L(\theta;x)\}}{Z(x)},\qquad
Z(x)=\int_\Theta \pi(\theta)\exp\{-\eta L(\theta;x)\}\,d\theta.
\]
For cumulative data \(x_{1:n}\), the loss becomes additive,
\[
L_n(\theta)=\sum_{i=1}^n \ell(\theta;x_i),
\]
and the same Gibbs form yields sequential coherence: tilting by \(L_m\) and then \(L_{m+1:n}\) produces the same distribution as tilting once by \(L_n\) [2602.01573].

The decision-theoretic characterization in "When Is Generalized Bayes Bayesian? A Decision-Theoretic Characterization of Loss-Based Updating" distinguishes **belief posteriors** from **decision posteriors** [2602.01573]. Belief posteriors are conditional beliefs justified by Savage and Anscombe–Aumann foundations and arise from a prior \(\pi(\theta)\) together with a likelihood \(p(x\mid \theta)\). Decision posteriors are randomized decision rules \(q(\cdot\mid x)\in \Delta(\Theta)\), justified by preferences over decision rules rather than by conditional-belief coherence. In that regime, the baseline \(\pi\) is a regularization or default distribution over actions, and coherence is sequential or batching coherence of the update operator.

The main equivalence theorem is sharp. A generalized Bayes update coincides with ordinary Bayes if and only if the loss is, up to scale and a data-only term, negative log-likelihood:
\[
L(\theta;x)=-c\,\log p(x\mid \theta)+g(x),\qquad c>0,
\]
with \(g(x)\) independent of \(\theta\) [2602.01573]. Setting \(c=1\) recovers ordinary Bayes exactly. A related diagnostic is
\[
A(\theta)=\int_{\mathcal X}\exp\{-\eta \ell(\theta;x)\}\,\mu(dx).
\]
If generalized Bayes equals a belief posterior for all \(x\), then \(A(\theta)\) must be finite and independent of \(\theta\); conversely, if \(A(\theta)\equiv A\in(0,\infty)\), then \(\exp\{-\eta \ell(\theta;x)\}/A\) defines a valid likelihood [2602.01573].

Outside the log-loss regime, the normalizing integral loses its canonical evidential interpretation. The same paper shows that for any data-only shift \(c(x)\),
\[
\ell_c(\theta;x)=\ell(\theta;x)+c(x)
\quad\Rightarrow\quad
q_c(\theta\mid x)=q(\theta\mid x),\qquad
Z_c(x)=e^{-\eta c(x)}Z(x).
\]
Therefore the posterior mapping \(x\mapsto q(\cdot\mid x)\) does not determine \(x\mapsto Z(x)\), and Bayes-factor-like ratios built from such \(Z(x)\) are not well-defined evidence unless the loss recovers a genuine likelihood [2602.01573].

The same work also establishes why non-degenerate loss-based posteriors are not compatible with linear von Neumann–Morgenstern utility over \(\Delta(\Theta)\). Under vNM axioms, any optimizer is supported on \(\arg\max_\theta u_x(\theta)\), so non-degenerate posteriors require nonlinear preferences over decision rules. Under weak order, loss monotonicity, sequential coherence, separability, and an \(f\)-divergence deviation cost, product additivity uniquely selects relative entropy up to scale, yielding the entropy-regularized variational problem
\[
q^*=\arg\min_q\left\{\mathbb E_q[\ell(\theta;x)]+\frac{1}{\eta}D_{\mathrm{KL}}(q\|\pi)\right\},
\]
whose unique minimizer is exactly the exponential tilt. The dual value is
\[
-\frac{1}{\eta}\log Z(x),
\]
so the normalization integral is simultaneously a partition function and the value of an entropy-penalized risk minimization problem [2602.01573].

## 3. Sequential recursion, streaming data, and posterior reuse

In standard recursive Bayes with streaming or batched observations,
\[
p(\theta\mid y_{1:t})
=
\frac{p(\theta\mid y_{1:t-1})\,p(y_t\mid \theta)}{Z_t},
\qquad
Z_t=\int p(\theta\mid y_{1:t-1})p(y_t\mid \theta)\,d\theta.
\]
Here \(Z_t=p(y_t\mid y_{1:t-1})\) is the previous posterior’s predictive density for the new data. This is the classical integral update for static parameters in online inference [2508.01572; 1908.00225].

"A strategy to avoid particle depletion in recursive Bayesian inference" studies this recursion in Monte Carlo form [2508.01572]. In prior-proposal recursive Bayes, proposals are drawn from the previous posterior, and under conditional independence the Metropolis–Hastings acceptance ratio simplifies to a pure likelihood ratio:
\[
\alpha(\theta'\mid \theta)=\min\!\left\{1,\frac{p(y_t\mid \theta')}{p(y_t\mid \theta)}\right\}.
\]
The predictive normalizer \(Z_t\) therefore need not be evaluated explicitly. The difficulty is particle depletion: empirical-resampling proposals reuse a discrete set of past particles, acceptance rates decline as the transient posterior contracts or shifts, duplicates accumulate, and the sample can collapse to a single value. The paper’s remedy, Smoothed Prior-Proposal Recursive Bayes, replaces discrete reuse by a continuous regularized mixture proposal indexed by \(\lambda\in[0,1]\); \(\lambda=1\) recovers multinomial resampling, whereas \(\lambda=0\) gives a joint Gaussian proposal. In simulations, \(\lambda\in[0,0.9]\) yielded accurate logistic-regression updates, and in a hierarchical species-distribution model with 100 parameters, the smoothed scheme recovered all-at-once posteriors with manageable memory [2508.01572].

"Updating Variational Bayes: Fast sequential posterior inference" develops a deterministic approximation to the same sequential recursion by replacing the unavailable exact prior \(p(\theta\mid D_{1:t-1})\) with the previous variational approximation \(q_{t-1}(\theta)\) [1908.00225]. The resulting pseudo-posterior is
\[
\tilde p(\theta\mid D_{1:t})\propto p(D_t\mid \theta)\,q_{t-1}(\theta),
\]
and the new approximation \(q_t\) is obtained by minimizing \(\mathrm{KL}(q_t\|\tilde p)\). UVB-IS further accelerates the update by reusing samples from \(q_{t-1}\) through importance weights \(q_\lambda/q_{t-1}\). In the reported simulations, UVB and UVB-IS preserved predictive accuracy while reducing computation; for one mixture-model experiment at \(T_{10}=100\) and \(K=1\), UVB and UVB-IS used \(14.7\%\) and \(4.6\%\) of the SVB computational time, respectively [1908.00225].

A distinct but related setting is posterior adaptation under label shift. "Posterior Adaptation With New Priors" assumes \(p(x\mid y)\) is unchanged while class priors vary, and derives
\[
p_{\mathrm{new}}(y\mid x)
=
\frac{p_{\mathrm{old}}(y\mid x)\,\frac{p_{\mathrm{new}}(y)}{p_{\mathrm{old}}(y)}}
{\sum_k p_{\mathrm{old}}(k\mid x)\,\frac{p_{\mathrm{new}}(k)}{p_{\mathrm{old}}(k)}}.
\]
The update is a posterior reweighting by prior ratios followed by renormalization; the unknown scale in the recovered likelihood cancels in the denominator. The method is valid under label shift and calibration of the original posteriors, but not under covariate shift [2007.01386].

Recursive updating also appears in a non-Bayesian but structurally analogous form in PAC-Bayes. "Recursive PAC-Bayes: A Frequentist Approach to Sequential Prior Updates with No Information Loss" replaces evidence-based posterior updating with a recursive bound on randomized classifiers, decomposing posterior risk into an excess term plus a downscaled prior risk that is bounded recursively [2405.14681]. The contribution is not a Bayesian posterior in the probabilistic sense, but it addresses the same operational question: how to turn each posterior-like distribution into the next prior without discarding information from earlier batches.

## 4. Aggregating information before updating: objective priors and moment constraints

A different meaning of integral updating arises when the prior itself must evolve with the information structure of the combined data. "Modification of Bayesian Updating where Continuous Parameters have Differing Relationships with New and Existing Data" considers independent experiments \(D_1,\ldots,D_J\) with continuous parameters but experiment-specific noninformative priors [1308.2791]. If Jeffreys’ prior for experiment \(j\) is
\[
\pi_{J,j}(\theta)\propto \sqrt{\det I_j(\theta)},
\]
then standard sequential updating is order-dependent whenever \(I_1(\theta)\) and \(I_2(\theta)\) differ in functional form:
\[
p(\theta\mid D_1,D_2)\propto L_1L_2\sqrt{\det I_1}
\qquad\text{versus}\qquad
p(\theta\mid D_2,D_1)\propto L_1L_2\sqrt{\det I_2}.
\]
The proposed remedy is to aggregate expected Fisher information first,
\[
I_{\text{total}}(\theta)=\sum_{j=1}^J I_j(\theta),
\]
form the joint noninformative prior
\[
\pi_{\text{joint}}(\theta)\propto \sqrt{\det I_{\text{total}}(\theta)},
\]
and apply Bayes’ theorem once:
\[
p(\theta\mid D_{1:J})\propto \sqrt{\det I_{\text{total}}(\theta)}\prod_{j=1}^J L_j(D_j\mid \theta).
\]
Because information addition and likelihood multiplication are commutative, the posterior is order-invariant [1308.2791].

The paper reports two numerical demonstrations. In a normal-plus-cubic-normal example, with 20,000 simulations and one-sided \(5\%\) and \(95\%\) bounds, the uniform-prior path had empirical tail frequencies \(2.0\%\) and \(8.7\%\), the \(\theta^2\)-prior path had \(7.0\%\) and \(2.4\%\), and the joint prior from integral updating had \(5.7\%\) and \(5.6\%\), materially improving probability matching [1308.2791]. In a binomial/negative-binomial example with \(n=40\), \(r=2\), and \(\theta\in[0.01,0.11]\), the paper again reports improved coverage for the joint prior over either ordering of standard updating [1308.2791]. The same framework extends to other Fisher-information-based priors of the form \(f(I(\theta))\).

Another integral extension of posterior updating appears in maximum relative entropy. "Updating Probabilities with Data and Moments" maximizes relative entropy over the joint \(P(x,\theta)\) subject to normalization, a data constraint \(P(x)=\delta(x-x_{\text{obs}})\), and posterior-level moment constraints \(\langle f_i(\theta)\rangle = F_i\) [0708.1593]. The resulting canonical posterior is
\[
P_{\mathrm{new}}(\theta\mid x_{\mathrm{obs}})
=
\frac{1}{Z(\lambda;x_{\mathrm{obs}})}
P_{\mathrm{old}}(\theta)L(x_{\mathrm{obs}}\mid \theta)
\exp\!\Big(\sum_i \lambda_i f_i(\theta)\Big),
\]
with multipliers determined by
\[
F_i=\frac{\partial \log Z(\lambda;x_{\mathrm{obs}})}{\partial \lambda_i}.
\]
When all \(\lambda_i=0\), the update reduces to Bayes’ rule. The moment information therefore acts as an exponential tilt of the Bayes posterior, much as in generalized Bayes, but the tilt is calibrated by explicit posterior constraints rather than by a user-chosen loss [0708.1593].

The same paper emphasizes that constraints need not commute. Distinct data constraints commute, but moment constraints and data constraints need not. Sequential updating is appropriate when later information supersedes earlier information; simultaneous updating is appropriate when all constraints are intended to hold in the final posterior. The difference is substantive rather than merely algorithmic, because the two update orders encode different informational states [0708.1593].

## 5. Marginalization over latent parameters, model error, and dynamical structure

In hierarchical Bayesian models, integral updating is often literal marginalization. "Hierarchical Bayesian Modeling for Uncertainty Quantification and Reliability Updating using Data" specifies
\[
p(\Theta,\eta\mid D)\propto p(\eta)\prod_{i=1}^{N_p} p(\theta_i\mid \eta)\,p(D_i\mid \theta_i),
\]
with hyperparameters \(\eta\) governing between-dataset variability [2412.20416]. The hyperposterior is
\[
p(\eta\mid D)\propto p(\eta)\prod_{i=1}^{N_p}\int p(\theta_i\mid \eta)p(D_i\mid \theta_i)\,d\theta_i.
\]
Reliability is then updated by integrating failure probability over both parameter variability and posterior hyperparameter uncertainty:
\[
P_f(D)=\int\!\!\int P(g(X,\theta)\le 0)\,p(\theta\mid \eta)\,p(\eta\mid D)\,d\theta\,d\eta.
\]
For linear Gaussian models this can be done analytically through Gaussian convolution; for dynamical structural models the paper uses a two-stage TMCMC procedure for \(p(\eta\mid D)\) and subset simulation for rare-event estimation [2412.20416]. The contrast with conventional Bayesian modeling is interpretive: the hierarchical formulation keeps between-dataset variability explicit, whereas a pooled posterior \(p(\theta\mid D)\) can understate uncertainty and yield overly optimistic reliability estimates.

"Iterative Updating of Model Error for Bayesian Inversion" gives a different latent-integral construction [1707.04246]. Let \(G\) be the accurate forward model, \(\tilde G\) a reduced model, and \(M(x)=G(x)-\tilde G(x)\) the modeling-error map. The algorithm estimates the distribution of \(M(x)\) under the current posterior \(\pi_k\) and updates by
\[
\pi_{k+1}(x)\propto \pi_0(x)\int \pi_{\text{noise}}(y-\tilde G(x)-M(z))\,\pi_k(z)\,dz.
\]
Each update integrates the likelihood over an iteratively refined modeling-error distribution. In the linear Gaussian case, the paper proves geometric convergence of the resulting posterior means and covariances; for general models, it introduces particle approximations and shows convergence of each iterand in the large-particle limit [1707.04246].

A related marginalization principle appears in fully Bayesian ensemble filtering. "A generalised and fully Bayesian framework for ensemble updating" writes, at one assimilation time,
\[
f_{x^t\mid y^t}(x^t\mid y^t)
=
\int f_{x^t\mid \theta^t,y^t}(x^t\mid \theta^t,y^t)\,
f_{\theta^t\mid y^t}(\theta^t\mid y^t)\,d\theta^t,
\]
and constructs per-member updates by simulating \(\theta^{t,(i)}\) from \(f_{\theta^t\mid x^{t,-(i)},y^t}\), excluding the ensemble member being updated [2103.14565]. The update distribution \(q\) is constrained so that marginalizing over the prior state and the sampled parameter reproduces the assumed posterior. In the linear-Gaussian case this yields a square-root-filter-type update that is optimal under an expected Mahalanobis criterion; in a finite-state hidden Markov model, the paper enforces posterior pairwise marginals through a dynamic-programming construction [2103.14565]. The exclusion of the target ensemble member from the parameter draw is presented as essential to avoid double use of information and to improve uncertainty representation.

For continuous-time latent Markov processes, integral updating can be lifted to path space. "A path integral approach to Bayesian inference in Markov processes" expresses the posterior over trajectories as
\[
P[x(\cdot)\mid y(\cdot)]
\propto
P_{\text{prior}}[x(\cdot)]
\exp\!\Big\{-\int_0^T V(x(t),t)\,dt\Big\},
\]
where \(V(x,t)=-\log L(y(t)\mid x)\) acts as a potential [1710.07755]. The unnormalized posterior density \(\psi\) then satisfies an imaginary-time Schrödinger, or Feynman–Kac, equation
\[
\partial_t \psi = \mathcal L_{\text{prior}}\psi - V\psi.
\]
In linear-Gaussian settings this recovers classical filtering equations; more generally it recasts Bayesian updating in dynamical systems as evolution under a prior generator plus a likelihood-induced sink term [1710.07755].

## 6. Computational formulations, interpretation, and recurrent misconceptions

One computational line treats the integral update itself as an object for acceleration. "Fast Bayesian Updates via Harmonic Representations" expands prior and likelihood in an orthogonal basis and shows that the product update becomes convolution in coefficient space [2511.06978]. On a periodic domain,
\[
c_k(\pi \ell)=\sum_m a_m b_{k-m},
\]
and the evidence is the zero-frequency coefficient:
\[
Z=L\,c_0(\pi\ell).
\]
After truncation, the update becomes a circular convolution implementable with FFTs, giving \(O(N\log N)\) complexity instead of the \(O(N^2)\) cost of naive convolution [2511.06978]. The method is deterministic and naturally sequential, but its efficiency depends on smoothness and spectral decay; nonsmooth targets, heavy tails, and high-dimensional parameter spaces weaken the spectral advantage.

A recurrent misconception is to treat every normalizing integral as evidence in the strong Bayesian sense. The generalized-Bayes analysis rejects that equivalence except in the log-loss case: outside a genuine likelihood model, the partition function is not canonical evidence, and Bayes factors are not identified without additional structure [2602.01573]. A second misconception is that sequential Bayes is automatically order-invariant even with noninformative priors. That is true with a fixed proper prior, but not when the prior itself depends on the experiment, as in Jeffreys-type objective Bayes; in that setting, order invariance may require aggregation of Fisher information before a single update [1308.2791].

Other limitations are model-specific. Posterior adaptation under changing class priors assumes label shift and calibrated old posteriors; it does not address covariate shift [2007.01386]. Recursive Monte Carlo schemes can exchange explicit evaluation of \(Z_t\) for proposal-quality problems such as particle depletion or low effective sample size [2508.01572]. Sequential variational methods avoid revisiting all past data, but they accumulate approximation error when \(q_{t-1}\) departs materially from the exact \(p(\theta\mid D_{1:t-1})\) [1908.00225]. Hierarchical reliability updating requires identifiable hyperparameters, adequate forward models, and numerically stable rare-event integration [2412.20416]. Iterative model-error correction depends on the accuracy of the induced error distribution and on an independence approximation between state and model discrepancy that is exact only in special cases [1707.04246].

Taken together, these literatures show that integral updating of posteriors is not a single algorithm but a structural pattern. The integral may normalize, marginalize, aggregate information, or define a variational dual quantity. Its meaning depends on the epistemic role of the updated distribution. When the update is generated by a true likelihood, the integral has the status of predictive or marginal likelihood. When the update is loss-based, constraint-based, or auxiliary-variable-based, the same algebra can instead encode a coherent decision rule, an entropy projection, an order-invariant objective update, or a latent-variable marginalization [2602.01573; 0708.1593; 1308.2791].

Source: https://www.emergentmind.com/topics/integral-updating-of-posteriors