Marginal Posterior-Based Criterion in Bayesian Inference

• Marginal Posterior-Based Criterion is a Bayesian approach that focuses on directly estimating the marginal posterior by integrating out nuisance parameters.
• It employs techniques such as mixed Laplace approximations and pseudo-marginal methods to efficiently approximate high-dimensional likelihoods.
• The method has practical applications in model selection, likelihood-free inference, and communications by reducing computational burdens and mitigating the curse of dimensionality.

A marginal posterior-based criterion is a Bayesian inferential principle in which the primary object is a posterior marginal for a quantity of interest rather than the full joint posterior over all unknowns. In the standard partition $\theta=(\gamma,\eta)$, with $\gamma$ the target and $\eta$ nuisance, the target becomes

$$\pi(\gamma\mid Y)\propto \int \exp\{\ell_n(\gamma,\eta)\}\,\pi(\gamma,\eta)\,d\eta,$$

or, in likelihood-free notation,

$$p(\vartheta\mid x)=\int p(\vartheta,\eta\mid x)\,d\eta=\frac{p(x\mid \vartheta)}{p(x)}\,p(\vartheta).$$

Taken together, the literature identifies a common strategy: if scientific conclusions depend mainly on low-dimensional summaries, then inference, approximation, and decision rules can be organized directly around marginal posteriors, often avoiding the computational and statistical burdens of reconstructing the entire joint posterior (Spokoiny, 2023, Miller et al., 2021, Jeffrey et al., 2020).

1. Formal basis and motivation

The central mathematical move is marginalization over nuisance coordinates. In simulation-based inference and related Bayesian settings, one distinguishes a subset of parameters of interest, $\boldsymbol{\theta}_s$ or $\vartheta$, from nuisance coordinates, and defines the inferential target by integrating out the latter. This yields the marginal likelihood-to-evidence ratio

$$r(x,\boldsymbol{\theta}_s)=\frac{p(x\mid \boldsymbol{\theta}_s)}{p(x)}=\frac{p(\boldsymbol{\theta}_s\mid x)}{p(\boldsymbol{\theta}_s)},$$

which directly recovers the marginal posterior once multiplied by the prior. The same structure appears in both classical Bayesian asymptotics and likelihood-free neural ratio estimation, although the computational realizations differ substantially (Miller et al., 2020, Miller et al., 2021).

The motivation is explicitly computational and inferential. High-dimensional posterior density estimation suffers from the “curse of dimensionality,” and in many physical and statistical problems the full posterior is “unwieldy and seldom used in practice.” Several works therefore argue that one should estimate only the 1D or 2D marginals that are scientifically interrogated. In this view, the target is not the joint posterior followed by numerical marginalization, but the marginal posterior itself. An important consequence is that all 2D marginals together can reveal the covariance structure of the full posterior, while avoiding the costs of full density estimation or high-dimensional MCMC (Jeffrey et al., 2020, Miller et al., 2020).

This reorientation also changes what counts as an effective dimension. In mixed Laplace theory, the relevant dimension becomes the dimension of interest $p$, not the dimension of the nuisance component. In direct marginal learning, nuisance parameters are averaged out implicitly by the simulation design. In both cases, marginalization is not merely a summary step after inference; it is the organizing principle of the inference procedure itself (Spokoiny, 2023, Jeffrey et al., 2020).

2. Marginalization as an approximation and sampling strategy

A major theoretical development is the “mixed Laplace approximation.” Instead of approximating the full posterior in all coordinates by a single Gaussian, the nuisance variable is treated by a conditional Laplace expansion for each fixed target value. The resulting marginal posterior is approximated by a Gaussian mixture,

$$\pi(\gamma\mid Y)\approx \int \varphi_p\!\left(\gamma;\,\hat\gamma(\eta),\,\mathcal{I}_\gamma(\eta)^{-1}\right)\,w(d\eta),$$

and the approximation error depends essentially on the target dimension $\gamma$0. Under stronger conditions such as $\gamma$1 or semi-orthogonality, the Gaussian mixture can be replaced by a single Gaussian, recovering a classical Laplace result for the marginal posterior rather than the full joint posterior (Spokoiny, 2023).

For scalar targets with nuisance parameters, higher-order tail area approximations provide a different marginal-posterior construction. With $\gamma$2 and target

$\gamma$3

the HOTA method uses

$\gamma$4

to obtain independent draws from an approximate marginal posterior by solving $\gamma$5 for $\gamma$6. The method is presented as an alternative to MCMC, requiring only standard maximum likelihood routines and yielding posterior means, quantiles, HPD regions, and prior sensitivity analyses from independent samples (Ruli et al., 2012).

In latent-variable problems with intractable likelihoods, pseudo-marginal methods pursue exact marginal-posterior targeting. Pseudo-Marginal Hamiltonian Monte Carlo assumes a non-negative unbiased likelihood estimator

$\gamma$7

constructs an extended Hamiltonian system, and samples a chain whose $\gamma$8-marginal is exactly $\gamma$9. As the precision parameter $\eta$0 increases, the sample trajectories and acceptance probabilities converge to those of ideal HMC on the marginal posterior (Alenlöv et al., 2016).

A concrete applied instance is inference for log Gaussian Cox processes using an approximate marginal posterior for $\eta$1. There the domain is partitioned into blocks, block counts are modeled with a multivariate Poisson lognormal approximation, and pseudo-marginal MCMC is used for the approximate marginal likelihood. The resulting workflow separates hyperparameter inference from latent-field recovery: one first samples $\eta$2 from the approximate marginal posterior and then samples the latent Gaussian field conditionally on $\eta$3, reducing the cost associated with repeated operations on high-dimensional covariance matrices (Shirota et al., 2016).

3. Direct estimation of marginal posteriors

One line of work replaces posterior sampling by direct learning of marginal posteriors. In “Solving high-dimensional parameter inference: marginal posterior densities & Moment Networks,” the proposal is to train density estimators directly on low-dimensional subsets such as $\eta$4, minimizing

$\eta$5

so that the learned conditional density estimates the true marginal posterior. The same paper introduces Moment Networks, in which an $\eta$6-trained regression $\eta$7 returns the posterior mean, and a second network $\eta$8 returns second moments and variances. The hierarchy can be extended to covariances and higher moments of arbitrary lower-dimensional marginals (Jeffrey et al., 2020).

In swyft, the object of inference is again the marginal likelihood-to-evidence ratio for a selected subset of parameters. The method learns

$\eta$9

by binary classification between samples from the joint $$\pi(\gamma\mid Y)\propto \int \exp\{\ell_n(\gamma,\eta)\}\,\pi(\gamma,\eta)\,d\eta,$$0 and from the product $$\pi(\gamma\mid Y)\propto \int \exp\{\ell_n(\gamma,\eta)\}\,\pi(\gamma,\eta)\,d\eta,$$1. Its Nested Ratio Estimation procedure iteratively constrains the prior to regions of parameter space likely to explain the observation, while iP3 sample caching reuses expensive simulations through a Poisson point process construction. The design is explicitly “marginal-posterior-first”: each 1D marginal is estimated directly, and many marginals can be trained in parallel on the same simulations (Miller et al., 2020).

TMNRE adds two further elements. First, it learns marginal likelihood-to-evidence ratios for selected marginals rather than the full posterior. Second, it truncates the prior to a region $$\pi(\gamma\mid Y)\propto \int \exp\{\ell_n(\gamma,\eta)\}\,\pi(\gamma,\eta)\,d\eta,$$2 defined by low-density tail cutoffs in the learned one-dimensional marginals, so later rounds focus simulations where posterior mass is non-negligible. Because the posterior estimator is “locally amortized” over the constrained region, the method also supports empirical tests of calibration by comparing nominal and empirical credible intervals under repeated simulation (Miller et al., 2021).

4. Selection and reduction rules built from marginal posteriors

In model selection for regression, the median probability model is a canonical marginal posterior-based rule. It includes variable $$\pi(\gamma\mid Y)\propto \int \exp\{\ell_n(\gamma,\eta)\}\,\pi(\gamma,\eta)\,d\eta,$$3 whenever

$$\pi(\gamma\mid Y)\propto \int \exp\{\ell_n(\gamma,\eta)\}\,\pi(\gamma,\eta)\,d\eta,$$4

The rule is predictive-optimal in orthogonal designs and in the nested correlated designs originally studied by Barbieri and Berger. The same work also shows that in more general correlated designs, especially with duplicate or highly collinear predictors, exact optimality may depend on joint inclusion structure rather than only on marginal posterior inclusion probabilities, although the median probability model can remain “safe” and close to the predictive optimum (Barbieri et al., 2018).

In fully Bayesian inversion, posterior reduction itself becomes the object of analysis. One study distinguishes three categories: use of the full joint posterior, use of the marginal posterior of the model parameters, and use of the marginal posterior of the hyperparameters. For linear inverse problems, the synthetic test reported there finds that the appropriate reduction is category (3), typified by Akaike’s Bayesian information criterion (ABIC), in which one first maximizes the marginal posterior of the hyperparameters and then conditions on the resulting estimate to infer the model parameters. The paper argues that category (2), the marginal posterior of the model parameters, can become inappropriate when the number of model parameters is large (Sato et al., 2022).

Exact posterior marginals also underwrite exact decision rules in discrete Bayesian networks. For a query variable $$\pi(\gamma\mid Y)\propto \int \exp\{\ell_n(\gamma,\eta)\}\,\pi(\gamma,\eta)\,d\eta,$$5 and evidence $$\pi(\gamma\mid Y)\propto \int \exp\{\ell_n(\gamma,\eta)\}\,\pi(\gamma,\eta)\,d\eta,$$6,

$$\pi(\gamma\mid Y)\propto \int \exp\{\ell_n(\gamma,\eta)\}\,\pi(\gamma,\eta)\,d\eta,$$7

The border algorithm, revised polytree algorithm, and parentless polytree method are designed to compute these posterior marginals exactly by local message passing rather than full-joint enumeration. The resulting posterior table over $$\pi(\gamma\mid Y)\propto \int \exp\{\ell_n(\gamma,\eta)\}\,\pi(\gamma,\eta)\,d\eta,$$8 can then support MAP selection, thresholding, or other posterior-probability-based criteria (Minh, 2014).

5. Exactness, invariance, and specialized realizations

A precise invariance theorem appears in contingency-table models. For generalized odds ratios of the form

$$\pi(\gamma\mid Y)\propto \int \exp\{\ell_n(\gamma,\eta)\}\,\pi(\gamma,\eta)\,d\eta,$$9

posterior inference under multinomial sampling and under a fixed-margin model is identical if and only if the coefficients defining the contrast sum to zero within each fixed partition block: $$p(\vartheta\mid x)=\int p(\vartheta,\eta\mid x)\,d\eta=\frac{p(x\mid \vartheta)}{p(x)}\,p(\vartheta).$$0 Under prior independence between the marginal and conditional components, this condition removes all dependence of the contrast on the fixed margins, so the posterior of $$p(\vartheta\mid x)=\int p(\vartheta,\eta\mid x)\,d\eta=\frac{p(x\mid \vartheta)}{p(x)}\,p(\vartheta).$$1 depends only on the conditional parameters, which have the same posterior in both sampling schemes (Stern et al., 27 Apr 2026).

The communications literature contains a different but related realization in soft MIMO detection. There the algorithm based on marginal posterior probability statistics computes per-layer marginal posterior probabilities, transforms the sampled marginal posterior distribution toward a Gaussian reference by solving

$$p(\vartheta\mid x)=\int p(\vartheta,\eta\mid x)\,d\eta=\frac{p(x\mid \vartheta)}{p(x)}\,p(\vartheta).$$2

and then reconstructs log-MAP LLRs from first- and second-order moment statistics of the transformed distribution. The method is explicitly motivated by the difficulty of recovering reliable counter-hypotheses from pruned search trees; layer-wise marginal posteriors supply a lower-complexity surrogate (Zhang et al., 2022).

These examples clarify that marginal posterior-based criteria need not be confined to one computational paradigm. They appear as exact query distributions in graphical models, as invariance objects in contingency tables, and as layer-wise sufficient summaries for approximate decoding in communications. The unifying feature is not the surrounding algorithmic machinery, but the decision to treat a posterior marginal as the decisive inferential object.

6. Neighboring evidence-based criteria and principal limitations

Several neighboring methods use posterior output to estimate marginal likelihoods rather than marginal posteriors, but they are closely related because Bayes factors and posterior model probabilities depend on such integrals. THAMES estimates the reciprocal marginal likelihood $$p(\vartheta\mid x)=\int p(\vartheta,\eta\mid x)\,d\eta=\frac{p(x\mid \vartheta)}{p(x)}\,p(\vartheta).$$3 from posterior draws using a truncated uniform density on an ellipsoid centered at the posterior mean, yielding an estimator that is unbiased for $$p(\vartheta\mid x)=\int p(\vartheta,\eta\mid x)\,d\eta=\frac{p(x\mid \vartheta)}{p(x)}\,p(\vartheta).$$4, consistent, has finite variance, and is asymptotically normal. A separate importance-sampling approach uses the product of marginal posterior distributions of parameter blocks as the proposal density for marginal likelihood estimation, while the truncated posterior mixture estimator uses mixtures of posterior ordinates from lagged MCMC states to stabilize evidence calculations from posterior samples (Metodiev et al., 2023, 1311.0674, Tuomi et al., 2011).

A related but distinct development is Bayesian sparsification via the marginal likelihood. In SpaM, the Laplace-approximated log marginal likelihood is optimized during training, and the resulting posterior precision is reused to define the Optimal Posterior Damage pruning score

$$p(\vartheta\mid x)=\int p(\vartheta,\eta\mid x)\,d\eta=\frac{p(x\mid \vartheta)}{p(x)}\,p(\vartheta).$$5

This is marginal-likelihood-based rather than marginal-posterior-based in the narrow sense, but it exemplifies a broader family of criteria that reuse posterior geometry or posterior simulation to define tractable decision scores (Dhahri et al., 2024).

The main limitations are likewise recurrent. Direct marginal learning still depends on adequate simulation coverage and sufficient model capacity; disagreement between marginal flows and Moment Networks is treated as a diagnostic of insufficient simulations or insufficient network complexity. Moment-based summaries do not replace a posterior when one needs detailed non-Gaussian structure, tails, or multimodal features. Truncation strategies can wrongly exclude relevant parameter space if the estimated posterior is overconfident. Higher-order tail area approximations are derived for regular models and can degrade when the prior is very informative and the posterior mode is far from the MLE. Product-of-marginals importance sampling can deteriorate when the product of marginal posteriors is a poor approximation to the joint posterior or when blocks are chosen badly (Jeffrey et al., 2020, Miller et al., 2021, Ruli et al., 2012, 1311.0674).

Taken together, these limitations indicate that a marginal posterior-based criterion is not a universal replacement for full posterior inference. It is instead a targeted strategy whose validity depends on the scientific goal, the degree to which nuisance integration preserves the quantities of interest, and the fidelity with which the marginal posterior itself can be approximated or learned.