Generalized Bayesian M-Posterior

• The paper introduces a robust framework by replacing the log-likelihood with an empirical risk loss, yielding a Bayesian M-posterior analogous to frequentist M-estimation.
• It establishes asymptotic normality via a Bernstein–von Mises theorem, ensuring calibrated credible intervals and frequentist consistency under regularity conditions.
• The approach incorporates learning rate calibration and algorithmic strategies like median-of-means to mitigate outlier effects and enhance robustness in posterior inference.

A generalized Bayesian M-posterior is the probability distribution on parameter space Θ induced by substituting a loss function—typically the empirical risk of an M-estimator—in place of the classical log-likelihood within the Bayesian updating framework. This construction formalizes the Bayesian posterior analog of M-estimators from frequentist statistics, leading to posterior distributions that encapsulate robust point estimation and uncertainty quantification. The generalized M-posterior class includes the Gibbs posterior, disparity posteriors, and robust posteriors calibrated for nonstandard loss functions or composite models. Recent theoretical developments rigorously characterize the frequentist asymptotics and robustness properties of M-posteriors, including their Bernstein–von Mises limits, influence functions, breakdown points, and adjustments for learning-rate sensitivity in generalized Bayes inference (Marusic et al., 1 Oct 2025, Tamano et al., 19 Nov 2025, Minsker et al., 2022).

1. Formal Definition and General Construction

Given i.i.d. data $X^{1:n} = \{X_1, \ldots, X_n\}$ and a prior $\pi(\theta)$ on $\Theta \subset \mathbb{R}^p$, a generalized (Bayesian) M-posterior is defined by replacing the log-likelihood in Bayes’ formula with a general loss $\rho(x, \theta)$ and possible weighting:

$$L_n^{\alpha}(\theta) := \frac{1}{n} \sum_{i=1}^n \alpha_i \rho(X_i, \theta), \quad \pi_n(\theta | X^{1:n}) \propto \exp\big\{ -w n L_n^{\alpha}(\theta) \big\} \pi(\theta)$$

Here, $\alpha_i \ge 0$ are observation weights, and $w > 0$ is a tempering/scaling parameter (learning rate). For $\alpha_i \equiv 1$, $w=1$, this reduces to the “disparity” or Gibbs posterior. By choosing loss functions that coincide with the negative log-likelihood, standard Bayesian inference is recovered.

This construction encompasses a wide array of robust and nonstandard posteriors, including settings where loss functions correspond to robust M-estimation (e.g., Huber, quantile, or Tukey’s bisquare losses), or where the likelihood is reweighted—such as via Gamma random weights for heavy-tailed or contaminated models (Marusic et al., 1 Oct 2025).

2. Asymptotic Normality and the Bernstein–von Mises Theorem

Under two principal regularity conditions, a Bernstein–von Mises (BvM) theorem holds for M-posteriors:

A. Weighted M-LAN (Assumption 3.1):

Establishes local asymptotic normality (LAN) for the pseudo-likelihood constructed from $\rho(x, \theta)$, requiring sufficient differentiability, a local Lipschitz property on $\rho$, and a quadratic expansion of the expected loss about its risk minimizer $\theta^*$.

B. Concentration (Assumption 3.2):

Ensures that the posterior mass concentrates near the minimizer $\theta^*$ as sample size increases.

Given these, Theorem 3.1 states that the M-posterior is asymptotically normal in total variation distance:

$$d_{TV}\big[\,\pi_n(\cdot | X^{1:n}),\,N(\hat{\theta}_n, V_{\theta^*}^{-1}/(w n))\,\big] \to 0$$

where $\hat{\theta}_n$ is the empirical (weighted) M-estimator solving $\sum_i \alpha_i \psi(X_i, \hat{\theta}_n) = 0$ with $\psi(x, \theta) = \partial_\theta \rho(x, \theta)$, and $$V_{\theta^*} = -\partial^2_\theta E\, \rho(X, \theta)\vert_{\theta^*}$$ (Marusic et al., 1 Oct 2025).

3. Robustness Properties: Influence Function and Breakdown Point

The generalized M-posterior framework admits a rigorous analysis of robustness via two main concepts:

A. Posterior Influence Function (PIF):

Quantifies the infinitesimal sensitivity of the entire posterior density to contamination at $x_0$:

$$\operatorname{PIF}(x_0; \theta) := \partial_\epsilon\, \pi_{n,\epsilon}(\theta) \vert_{\epsilon=0} = n\,\pi_n(\theta) \big[ g(x_0, \theta) - \int \pi_n(\theta') g(x_0, \theta') d\theta'\big]$$

with $$g(x, \theta) = E_{F_n}[\rho(X, \theta)] - \rho(x, \theta)$$. Uniform B-robustness corresponds to the uniform boundedness of the PIF over both $\theta$ and $x_0$.

B. Posterior Breakdown Point:

Quantifies maximal resistance to arbitrary contamination by measuring the smallest fraction $m/n$ of contaminated points that can cause the posterior (in, e.g., 2-Wasserstein distance) to shift by an unbounded amount:

$$\epsilon_{W_2}^*(\pi_n, X^{1:n}) := \min\left\{\,\frac{m}{n}: \sup_{F_n'\in\mathcal{F}_{n,m}} W_2(\pi_n(\cdot\,|\,F_n'), \pi_n(\cdot\,|\,F_n)) = \infty\,\right\}$$

The precise breakdown point depends on properties of $\rho$ and $\pi$; for convex losses with bounded score function and flat priors, $\epsilon^* = 1/2$ (maximal possible), while redescending or truncated losses can exhibit even higher robustness (Marusic et al., 1 Oct 2025).

Sufficient and Necessary Conditions

• Boundedness of the score $\psi = \partial_\theta \rho$ and convexity/coercivity of the loss ensure uniform B-robustness.
• When $\psi$ diverges with $x$, boundedness fails, preventing robust control.
• The robustness properties are directly inherited by posterior means and quantiles.

4. Learning Rate Sensitivity and Calibration

Generalized Bayes posteriors include an explicit learning rate parameter $\eta$:

$$\pi_n^\eta(\theta | x) \propto \exp\bigl\{ -\eta\,M_n(\theta) \bigr\} \pi_n(\theta)$$

where $M_n(\theta)$ is the empirical risk and $\pi_n(\theta)$ may include regularization. Asymptotic results show that credible intervals naively scale as $\eta^{-1/2}$. Thus, for data and loss functions not corresponding to an exact likelihood, posterior uncertainty depends acutely on $\eta$, possibly resulting in severe miscalibration if $\eta$ is chosen heuristically.

The open-faced sandwich adjustment rescales posterior covariance to match the robust “sandwich” estimator, but only addresses variance and not centering effects. The location–scale calibration approach proposes a postprocessing mapping:

$$\theta_\text{calib}^{(d)} = \tilde{\theta}_n + \Omega (\theta^{(d)} - \theta_\mathrm{GB})$$

with

$\Omega = (V_{\rm target}^*)^{1/2} (H_0)^{1/2}$

where $V_{\rm target}^*$ is the sandwich covariance, $H_0 = \eta J_\lambda^*$ the working curvature, and $\tilde{\theta}_n$ is a suitable data-dependent center (e.g., MAP or mean). Theorem 3.2 shows that the calibrated samples recover the correct uncertainty (asymptotically invariant to $\eta$), guaranteeing frequentist coverage in large samples (Tamano et al., 19 Nov 2025).

5. Algorithmic Instantiations: Median-of-Means and Empirical Implementation

Robust M-Posterior via Median-of-Means (MOM):

Partition the data into $k$ equal-sized blocks, compute blockwise losses, and replace the empirical average in the risk by a median-of-means surrogate. The M-posterior density is then:

$$\Pi_N^{(M)}(d\theta|X_N) = \frac{ \pi(\theta) \exp\bigl(-N \hat{L}(\theta)\bigr) }{ \int_\Theta \pi(\theta') \exp\bigl(-N \hat{L}(\theta')\bigr) d\theta' }$$

where

$$\hat{L}(\theta) = \argmin_{z \in \mathbb{R}} \sum_{j=1}^k \rho\left( \frac{ \sqrt{n} [\bar{L}_j(\theta) - z] }{ \Delta_n } \right)$$

Algorithmically, the block risk averages $\bar{L}_j(\theta)$, a robust $\rho$-function, and a tuning scale $\Delta_n$ are used to mitigate the impact of outliers, with all updates amenable to standard MCMC posterior computation (Minsker et al., 2022).

Plug-in Calibrated Posterior:

In empirical settings, plug-in estimation of $J^*_\lambda$ and $K^*$ (from posterior draws and the observed score function) enables direct application of location–scale transformations to posterior samples, achieving calibrated uncertainty bands regardless of learning rate (Tamano et al., 19 Nov 2025).

6. Frequentist Consistency, Robustness, and Applications

Canonical losses such as Huber, quantile, and reweighted log-likelihoods yield M-posteriors with the following traits:

• Asymptotic normality: Posterior contracts at the M-estimator, with variance determined by the curvature of the loss and the chosen learning rate or sandwich scaling.
• Consistency: M-posteriors provide point estimates and credible regions that converge to their frequentist M-estimator analogs.
• Robustness: Bounded posterior influence function, maximal breakdown point ($\epsilon^* = 1/2$) under suitable loss-prior choices.
• Empirical validation: In real and simulated data (e.g., contaminated mean estimation or robust regression), M-posteriors retain concentration near “true” parameter values under heavy-tailed contamination, while classical posteriors are susceptible to outlier-induced bias (Marusic et al., 1 Oct 2025, Minsker et al., 2022).

A summary of representative applications is presented in the following table:

Application	Loss Function	Robustness Metric
Huber Location Estimation	$ρ_c(x-θ)$ (Huber)	$ε^* = 1/2$; bounded PIF
Bayesian Quantile Regression	$ρ_τ(r) = r(τ - I\{r<0\})$	$ε^* \geq \min(τ,1-τ)$
Gaussian with Gamma Reweighting	$ρ(x,θ) = -\log\int π(α) f(x|θ)^{α} dα$	$ε^* \geq 1/2$
Poisson Factorization	$−κ \log(…)$ (Gamma reweighted Poisson)	robust fit

M-posteriors thus function as robust, Bayesian analogs to a wide family of frequentist estimators, equipped with rigorous frequentist and robustness guarantees (Marusic et al., 1 Oct 2025).

7. Comparison to Classical and Generalized Bayesian Procedures

In standard models under correct specification, the M-posterior coincides with the classical Bayesian posterior, and credible sets realize frequentist coverage (“matching”). In misspecified or pseudo-likelihood settings, classical likelihood-based posteriors lose calibration; generalized M-posteriors require adjustment (e.g., open-faced sandwich or location–scale calibration) to recover valid uncertainty quantification (Tamano et al., 19 Nov 2025).

Empirical results confirm that properly calibrated M-posterior intervals maintain near-nominal coverage and stable interval widths across learning rates, while uncalibrated generalized posteriors exhibit severe under- or over-coverage. In robust estimation tasks, M-posteriors guarantee stability that the classical Bayesian posterior lacks under gross contamination (Minsker et al., 2022, Tamano et al., 19 Nov 2025).

• Marušić, M. et al., "A theoretical framework for M-posteriors: frequentist guarantees and robustness properties" (Marusic et al., 1 Oct 2025)
• Syring, N. and Martin, R., "Location--Scale Calibration for Generalized Posterior" (Tamano et al., 19 Nov 2025)
• Minsker, S. and Yao, Y., "Generalized Median of Means Principle for Bayesian Inference" (Minsker et al., 2022)