Bayesian Sample Influence Measures

• Sample influence measures are quantitative diagnostics in Bayesian analysis that assess the sensitivity of posterior results to changes in individual observations via likelihood and prior perturbations.
• They leverage MCMC simulation to compute variance of log-likelihoods and replicate predictive distributions, enabling effective detection of influential and outlying data points.
• The approach supports model validation and refinement by quantifying prior-data conflict and facilitating robust outlier identification in both classical and high-dimensional models.

Sample influence measures are quantitative diagnostics designed to assess the sensitivity of Bayesian inferential results to individual observations or subsets of the data. In the Bayesian context, influence and leverage measure how perturbations to likelihood or prior components associated with observations affect the posterior or derived predictive quantities. By formalizing and computationally realizing these ideas for general Bayesian models—including those fitted with Markov chain Monte Carlo (MCMC)—researchers are equipped to diagnose influential or outlying data points, assess model robustness, and interpret predictive information criteria. These diagnostics also support detection of prior-data conflict, supplementing both model validation and selection procedures.

1. Bayesian Leverage and Influence: Core Definitions

The Bayesian local influence for an observation is defined through the effect of a small change in the data-point’s weight in the likelihood. Letting the observation weights be $w = (w_1, \dots, w_n)$ in a pseudo-likelihood

$$L_w(\theta) = \prod_{i=1}^n p(Y_i \mid x_i, \theta)^{w_i},$$

the Kullback–Leibler divergence between the posterior $p(\theta|Y)$ ($w_i=1$) and its perturbed version $p_w(\theta|Y)$ quantifies observation-level sensitivity. Taylor expansion yields a quadratic approximation: $$\Delta(w) \approx \frac{\phi''(1)}{2} (w-1)^{\mathrm{T}} V (w-1),$$ where $$V_{ij} = \operatorname{Cov}_\theta [\log p(Y_i \mid \theta), \log p(Y_j \mid \theta) \mid Y]$$ under the posterior. The Bayesian local influence of observation $i$ is thus

$$\mathrm{LINF}_i = V_{ii} = \operatorname{Var}_\theta \left[\log p(Y_i \mid \theta) \mid Y\right],$$

which expresses posterior uncertainty in the individual log-likelihood.

Bayesian leverage is defined in relation to replication at the design point $x_i$. The leverage (also called the Bayesian hat-value) is

$$h_i = \mathbb{E}_{Y_{r i}}\left\{ \mathbb{E}[ \log p(Y_{r i} \mid \theta) \mid \theta, Y_{r i} ] - \mathbb{E}[ \log p(Y_{r i} \mid \theta) \mid Y ] \right\},$$

quantifying how much observing an additional $Y_i$ at $x_i$ would alter predictive expectations. For the linear model, $h_i$ reduces to the frequentist hat value.

Practical deletion or upweighting influence diagnostics are constructed by finite perturbations: the “doubling influence” $\mathrm{DINF}_i$ (weight set to 2), and the “zeroing influence” $\mathrm{ZINF}_i$ (weight set to zero).

2. MCMC Computation of Diagnostics

The computational implementation leverages standard MCMC simulation. Given draws $\{\theta^{(m)}\}$ from $p(\theta \mid Y)$:

• The variance–covariance matrix $V$ is estimated as the sample covariance of $\log p(Y_i \mid \theta^{(m)})$ across $m$.
• Leverage $h_i$ is estimated via a two-sample Monte Carlo, by drawing pairs $(\theta^{(1)}, \theta^{(2)})$ from independent MCMC runs and computing the mean KL divergence between the predictive distributions for the replicate $Y_{r i}$.

For multivariate perturbations (subsets), the same machinery applies, allowing construction of matrices for joint or conditional influence and subsequent eigenanalysis to detect clusters of jointly influential or maskingly anomalous observations.

These diagnostics are fast, requiring only the MCMC samples obtained in the main analysis, and the necessary computations are simple statistics over the log-likelihood trace.

3. Outlier Detection: Leverage, Influence, and Conformal Measures

A key insight is that a distinction between leverage and influence facilitates robust outlier identification. High leverage ($h_i$) indicates that $x_i$ is unusual in predictor space, irrespective of $Y_i$, while high influence ($V_{ii}$) corresponds to an outcome $Y_i$ exerting abnormal effect on the posterior.

The conformal local outlier statistic (Editor’s term) is introduced: $$\mathrm{CLOUT}_i = \frac{\mathrm{CLINF}_i}{\mathrm{CLLEV}_i}, \quad\text{with}\qquad \mathrm{CLINF}_i = \frac{V_{ii}}{\sum_j V_{jj}}, \quad \mathrm{CLLEV}_i = \frac{h_i}{\sum_j h_j}$$ A high $\mathrm{CLOUT}_i$ identifies points that are more influential than their leverage would suggest and is thus suited to general-purpose outlier detection regardless of whether they substantially affect the overall fit.

4. Relationship to Predictive Information Criteria

Local influence and leverage diagnostics are shown to be directly related to model complexity penalties in standard Bayesian predictive criteria.

• The WAIC optimism penalty is

$$p_W = \sum_{i=1}^n \operatorname{Var}_\theta[\log p(Y_i \mid \theta)] = \sum_{i=1}^n \mathrm{LINF}_i,$$

making local influence the additive component of model “optimism.”

• The DIC effective number of parameters can be represented by the sum of leverages:

$$p_D^* = \sum_{i=1}^n h_i = \sum_{i=1}^n \mathrm{LLEV}_i.$$

• An alternative penalty, $$p_V = 2\mathrm{Var}[\sum_{i=1}^n \log p(Y_i \mid \theta)]$$, provides further information about the “global” variability inherent in the data-likelihood.

5. Prior–Data Conflict Diagnostic and Cross-Conflict Assessment

The ratio $p_V / p_W$ is introduced as a diagnostic for systematic conflict between the prior and the observed data. In the presence of a diffuse prior, $p_V \approx p_W$. However, when the prior pulls the posterior mean away from the MLE, $p_V$ increases. Substantially $p_V / p_W \gg 1$ flags prior-data tension or cross-conflict between data subsets in hierarchical models, motivating further posterior examination or modularization.

This diagnostic supplements traditional model checks and is applicable to complex Bayesian models including those with hierarchical or structured priors.

6. Practical Implications and Applications

Bayesian influence and leverage diagnostics support data-quality assessment, model checking, and robust summary inference. Leveraging the diagnostic matrices $V$ (influence) and $H$ (leverage), one can:

• Identify and summarize individual and subgroup anomalies (“masking” and “swamping” are resolved via joint and conditional perturbation matrices).
• Guide model refinement and data cleaning, detecting problematic cases that may not dramatically alter posterior means but signal local fit issues.
• Interpret complexity penalties in predictive criteria in terms of sample-wise and population-level contributions.

Eigenanalysis of these matrices extends outlier detection to multivariate and high-dimensional settings, enabling detection of “collective” aberrance.

7. Illustrative Examples and Computational Considerations

Empirical demonstrations on both classical benchmark datasets (Abalone, Bike Sharing) and substantive applications (e.g., survival analysis with UNOS data) confirm the diagnostics' utility in practice. Calculations require only standard posterior draws and basic vector operations, ensuring scalability to moderately high-dimensional Bayesian models commonly encountered in applied research.

In summary, Bayesian sample influence measures constitute a rigorous and computationally tractable framework for model sensitivity analysis, integrating seamlessly with posterior simulation and predictive modeling workflows (Plummer, 25 Mar 2025).