Bayes Factor Sensitivity Analysis

• Bayes factor sensitivity analysis is a framework that quantifies how evidence for competing models changes when prior distributions, model structures, or computational approximations are varied.
• It incorporates methods like simulation-based calibration, reweighting approaches, and functional estimation to assess the robustness of Bayesian model comparisons.
• The analysis informs study design and decision rules by revealing the impact of prior choices and model misspecification on hypothesis testing outcomes.

Searching arXiv for recent and foundational papers on Bayes factor sensitivity analysis. Bayes factor sensitivity analysis examines how the evidential comparison between hypotheses changes when the analyst varies prior distributions, model structure, computational approximations, or the data-generating assumptions under which the Bayes factor is interpreted. In its most basic form, the Bayes factor comparing hypotheses or models $H_1$ and $H_0$ is $BF_{10} = p(y\mid H_1)/p(y\mid H_0)$, where each marginal likelihood is a prior predictive integral. Because the marginal likelihood averages the likelihood over the full prior rather than only over posterior mass, Bayes factors are especially sensitive to prior specification, model misspecification, and numerical approximation. Contemporary work therefore treats sensitivity analysis not as an optional supplement but as a core part of Bayes-factor-based inference, ranging from workflow-based robustness checks and simulation-based calibration to functional estimation of Bayes factor surfaces over hyperparameter spaces (Schad et al., 2021).

1. Definition and inferential role

Bayes factors arise in Bayesian model comparison through marginal likelihoods. For competing models $\mathcal{M}_1$ and $\mathcal{M}_0$, with parameters $\Theta_1,\Theta_0$, likelihoods $p(y\mid \Theta_i,\mathcal{M}_i)$, priors $p(\Theta_i\mid \mathcal{M}_i)$, and model priors $p(\mathcal{M}_i)$, the marginal likelihood is

$$p(y\mid \mathcal{M}_i)=\int p(y\mid \Theta_i,\mathcal{M}_i)\,p(\Theta_i\mid \mathcal{M}_i)\,d\Theta_i,$$

and the Bayes factor is

$H_0$0

It updates prior odds to posterior odds through

$H_0$1

This makes the Bayes factor the core evidential quantity, while posterior model probabilities require the additional specification of prior model odds (Schad et al., 2021).

Sensitivity analysis enters because $H_0$2 is a prior predictive quantity. It depends on the entire prior, not merely on posterior concentration near well-fitting parameter values. A wide or heavy-tailed prior distributes mass over many low-likelihood regions and can lower the marginal likelihood; a narrow prior can raise it if the data are consistent with its support. This dependence is more acute for model comparison than for posterior parameter estimation, where informative data often attenuate prior influence (Schad et al., 2021).

A second, more general formulation treats Bayes-factor sensitivity as variation of $H_0$3 over a hyperparameterized prior family $H_0$4. In that setting,

$H_0$5

so sensitivity analysis becomes the study of the function $H_0$6, or equivalently the relative surface $H_0$7 for a fixed reference $H_0$8 (Doss et al., 2018).

2. Sources of sensitivity

The primary source of Bayes factor sensitivity is prior choice. Because marginal likelihoods integrate the likelihood over the prior, priors that appear innocuous for posterior estimation can induce substantial shifts in model evidence. In cognitive-science examples, varying only the prior on a key effect parameter can move $H_0$9 from favoring the alternative to favoring the null. Wide priors may be strongly penalized by Occam’s razor when the observed effect is small, whereas narrower priors centered near realistic values may yield moderate support for the alternative (Schad et al., 2021).

Model specification is a second major source. Bayes factors assume the model is correctly specified; misspecification can bias posterior model probabilities even when MCMC converges well. In hierarchical mixed models, omission of random slopes can systematically favor the alternative model. Simulation-based calibration detects this by showing that average posterior model probabilities no longer match prior model probabilities under repeated simulation (Schad et al., 2021).

Computational approximation is a third source. In realistic models, marginal likelihoods are rarely analytic. Bridge sampling can be stable when the posterior sample size is sufficiently large, but stability may deteriorate with too few effective draws. Savage–Dickey density ratios can become unstable when the posterior density near the null value is estimated from few samples. Even apparently stable estimates can remain biased if MCMC underexplores regions that dominate the marginal likelihood integral (Schad et al., 2021).

Repeated-sampling variability constitutes a fourth source. Bayes factors fluctuate across replicated data from the same generating process. Prior predictive and posterior predictive simulations show that, in small or noisy designs, Bayes factors can range from favoring $BF_{10} = p(y\mid H_1)/p(y\mid H_0)$0 to favoring $BF_{10} = p(y\mid H_1)/p(y\mid H_0)$1 across replications, whereas larger studies yield substantially tighter distributions. This suggests that Bayes factors are not intrinsically immune to replication variability; rather, their stability depends on design informativeness (Schad et al., 2021).

Large-sample asymptotics provide a complementary perspective. A general decomposition based on Chib’s identity writes $BF_{10} = p(y\mid H_1)/p(y\mid H_0)$2 as a sum of a log-likelihood ratio, a log prior ratio, and a log posterior ratio. In this formulation, the posterior ratio acts as a penalty term, and Bayes factor consistency depends on how priors affect posterior contraction rates in the competing models. This suggests that sensitivity is driven not only by prior mass at specific parameter values but also by how priors determine asymptotic concentration and complexity penalties (Chib et al., 2016).

3. Methodological frameworks

A workflow-oriented framework treats sensitivity analysis as a sequence of robustness checks. One begins by specifying the observational models and priors, then performing prior predictive checks to ensure that generated data are scientifically plausible. The models are fit repeatedly on the same dataset to assess stability to MCMC randomness, and Bayes factors are recomputed across runs. This is followed by simulation-based calibration, in which data are generated from the model prior, both models are fit, and the resulting posterior model probabilities are averaged. Under unbiased computation and correct specification, average posterior model probabilities should recover the prior model probabilities. The same workflow then investigates prior sensitivity, model-structure sensitivity, and data-variability sensitivity via prior predictive and posterior predictive simulation (Schad et al., 2021).

A functional MCMC framework studies entire families $BF_{10} = p(y\mid H_1)/p(y\mid H_0)$3 and $BF_{10} = p(y\mid H_1)/p(y\mid H_0)$4, where $BF_{10} = p(y\mid H_1)/p(y\mid H_0)$5 denotes posterior expectations under prior $BF_{10} = p(y\mid H_1)/p(y\mid H_0)$6. With posterior samples from a reference prior $BF_{10} = p(y\mid H_1)/p(y\mid H_0)$7, one can estimate

$BF_{10} = p(y\mid H_1)/p(y\mid H_0)$8

by Monte Carlo averages of prior ratios. This yields Bayes factor estimates for all $BF_{10} = p(y\mid H_1)/p(y\mid H_0)$9 from a single chain, together with strong consistency, functional central limit theorems, and asymptotic normality for the empirical maximizer of $\mathcal{M}_1$0 under suitable conditions (Doss et al., 2018).

A reweighting approach focused on posterior robustness dispenses with model refit. For a base prior $\mathcal{M}_1$1, alternative prior $\mathcal{M}_1$2, and prior ratio $\mathcal{M}_1$3, the marginal-likelihood ratio satisfies

$\mathcal{M}_1$4

This permits direct approximation of Bayes factor changes under alternative priors from base-posterior draws alone. The same framework yields Hellinger and Kullback–Leibler distances between base and alternative posteriors, allowing posterior robustness and Bayes-factor robustness to be assessed jointly without repeated fitting (Sugasawa, 2024).

A more recent efficiency-oriented framework introduces an extended model with a hyper-prior $\mathcal{M}_1$5 on the sensitivity parameter $\mathcal{M}_1$6 governing the prior under $\mathcal{M}_1$7. With an anchor value $\mathcal{M}_1$8,

$\mathcal{M}_1$9

For a uniform hyper-prior, the hyper-prior ratio disappears. Posterior density ratios $\mathcal{M}_0$0 are estimated by the importance-weighted marginal density estimator, and because $\mathcal{M}_0$1 enters only through the prior, the likelihood cancels from the estimator. This recovers the full sensitivity curve from one additional fit rather than one fit per grid point (Bartoš et al., 23 Apr 2026).

4. Computational and geometric approaches

Computational sensitivity analysis over continuous prior families predates the recent density-ratio approach. One line of work estimates $\mathcal{M}_0$2 and $\mathcal{M}_0$3 over $\mathcal{M}_0$4 from a small number of MCMC runs at skeleton hyperparameters. Reverse logistic estimation is used to recover normalizing-constant ratios, and importance sampling with control variates stabilizes reweighting across the prior family. In Bayesian variable selection, this permits construction of Bayes factor surfaces over hyperparameters such as $\mathcal{M}_0$5 and $\mathcal{M}_0$6, making empirical Bayes selection and robustness assessment computationally feasible without rerunning the full analysis for every hyperparameter setting (Buta et al., 2012).

A related empirical-process treatment formalizes this strategy by viewing $\mathcal{M}_0$7 and $\mathcal{M}_0$8 as stochastic processes indexed by $\mathcal{M}_0$9. Uniform laws of large numbers and functional central limit theorems then justify simultaneous confidence bands and uncertainty quantification for Bayes factor surfaces. This suggests a form of global Bayes factor sensitivity analysis in which not just pointwise Bayes factors but whole regions of hyperparameter space can be declared stable or unstable under Monte Carlo uncertainty (Doss et al., 2018).

Geometric robustness analysis provides a different perspective. Using the square-root density representation of priors, likelihoods, and posteriors, the nonparametric Fisher–Rao metric supplies geodesic perturbation paths and bounded discrepancy measures. Within this framework, geometric $\Theta_1,\Theta_0$0-contamination classes replace linear contamination classes, and one can derive local sensitivity indices for Bayes factors as directional derivatives of marginal likelihoods along geodesic perturbation directions. This yields an intrinsic robustness analysis of Bayes factors to prior and likelihood perturbations, together with global posterior sensitivity summaries based on Fisher–Rao distances (Kurtek et al., 2014).

In simple-vs-composite testing, another computational perspective exploits the identity that the Bayes factor is the posterior mean of the likelihood ratio under the alternative posterior. This makes the posterior distribution of the likelihood ratio informative for Bayes factor sensitivity: a narrow posterior distribution indicates that the Bayes factor is a stable summary, while a highly dispersed distribution indicates that the Bayes factor averages over highly variable evidential contributions. Fractional Bayes factors appear as posterior moments of the likelihood ratio, and the posterior cumulative distribution of the likelihood ratio can be used to assess the robustness of threshold-based decisions derived from a Bayes factor (Smith et al., 2010).

5. Reporting devices and domain-specific variants

One reporting-oriented approach replaces a single Bayes factor with a Bayes factor function. Here, the Bayes factor is expressed directly as a function of a standardized effect size $\Theta_1,\Theta_0$1, using closed-form formulas derived from common test statistics such as $\Theta_1,\Theta_0$2, $\Theta_1,\Theta_0$3, $\Theta_1,\Theta_0$4, and $\Theta_1,\Theta_0$5. The prior on the non-centrality parameter is chosen so that its mode corresponds to the target effect size, producing a curve $\Theta_1,\Theta_0$6. This function displays how evidence changes as assumptions about plausible effect sizes vary, thereby converting prior sensitivity into an effect-size-based sensitivity curve (Johnson et al., 2022).

This framework has been extended to partial correlation coefficients. In the multivariate-normal setting with sample partial correlation $\Theta_1,\Theta_0$7 and test statistic

$\Theta_1,\Theta_0$8

the null distribution is central $\Theta_1,\Theta_0$9, while the alternative is expressed through a non-centrality parameter linked to the population partial correlation. A normal moment prior on the non-centrality parameter is then centered at a target standardized effect size, yielding Bayes factor functions for partial correlation that make sensitivity to alternative-effect assumptions explicit (Datta et al., 13 Mar 2025).

In process tracing, Bayes factor sensitivity has been cast in terms of fully specified generative models for qualitative evidence. Two models are proposed: a binomial model for open-ended evidence pools and a hypergeometric model for bounded archives. Sensitivity is then quantified through tipping points: the amount of observation bias, the number of recoded observations, the smoking-gun weight, or the number of prior pseudo-observations needed to move the Bayes factor below a decision threshold such as 20. This use of Bayes factor sensitivity places robustness margins, rather than raw Bayes factors alone, at the center of substantive interpretation (López et al., 15 Jun 2026).

In meta-analysis, Bayes factor sensitivity focuses mainly on priors for the global effect and, to a lesser extent, heterogeneity. The main tested quantity is often a common or average effect $p(y\mid \Theta_i,\mathcal{M}_i)$0 or $p(y\mid \Theta_i,\mathcal{M}_i)$1, while sensitivity is explored by varying the prior scale under the alternative. The paper’s examples show that Bayes factor conclusions can be stable over realistic prior scales but drift toward the null under implausibly wide priors, illustrating Bartlett’s paradox in the meta-analytic context (Mulder et al., 27 Nov 2025).

6. Applications, design implications, and interpretation

Sensitivity analysis has direct implications for study design and decision rules. In cognitive-science applications, simulation-based calibration can be combined with a utility function to evaluate decision thresholds such as “claim discovery if $p(y\mid \Theta_i,\mathcal{M}_i)$2”. The resulting average expected utility depends on the probabilities of true discovery, false discovery, true rejection, and false rejection under simulated data, and the optimal threshold need not coincide with conventional Bayes factor benchmarks such as 10 (Schad et al., 2021).

Clinical trial design provides a different design-level use. In two-arm, two-stage phase II trials with binary endpoints, Bayes factors can serve as primary evidence measures at interim and final analyses. Exact operating characteristics such as Bayesian power, Bayesian type-I error, and the probability of compelling evidence for the null can be computed without Monte Carlo simulation by summing over prior-predictive Beta–Binomial distributions. Varying design priors, futility and efficacy thresholds, and interim timings then yields a genuine Bayes factor sensitivity analysis at the design stage, showing how strongly required sample size and expected sample size under $p(y\mid \Theta_i,\mathcal{M}_i)$3 depend on prior-predictive separation between null and alternative models (Kelter, 1 Jun 2026).

Meta-analysis supplies a cumulative setting in which Bayes factor sensitivity is especially consequential. Because Bayes factors update coherently as studies accumulate, they are natural tools for evidence monitoring. But this same cumulative use requires explicit attention to prior robustness, particularly for priors on the overall effect size. The paper emphasizes that sensitivity analysis should routinely accompany Bayes factor meta-analysis by examining prior scales that are scientifically plausible, while avoiding diffuse specifications that induce spurious support for the null via Bartlett’s paradox (Mulder et al., 27 Nov 2025).

Across these applications, a recurrent misconception is that a single Bayes factor is a complete evidential report. The recent literature suggests instead that Bayes-factor-based inference is best understood as a sensitivity problem: one must ask how evidence changes with prior width, hyperparameter choice, computational method, design informativeness, and structural assumptions. Another misconception is that stable numerical output guarantees correctness; simulation-based calibration and posterior or prior predictive replication show that numerical stability, estimator unbiasedness, and inferential robustness are distinct properties (Schad et al., 2021).

7. Current themes and open directions

Current work converges on the view that Bayes factor sensitivity analysis is fundamentally functional rather than pointwise. Instead of reporting one Bayes factor, analysts increasingly report curves, surfaces, confidence bands, or calibrated robustness workflows that display how $p(y\mid \Theta_i,\mathcal{M}_i)$4 varies over a class of priors or models. This is evident in empirical-process methods for $p(y\mid \Theta_i,\mathcal{M}_i)$5, posterior-density-ratio methods for full sensitivity curves, and Bayes factor functions based on test statistics (Doss et al., 2018).

A second current theme is computational reuse. Reweighting identities based on prior ratios, reverse logistic estimation, importance sampling with control variates, and posterior-density-ratio estimators all aim to avoid refitting the model for each sensitivity setting. This suggests that practical Bayes factor sensitivity analysis is shifting from brute-force grid refitting toward single-fit or few-fit functional estimation strategies (Bartoš et al., 23 Apr 2026).

A third theme is calibration. Simulation-based calibration for Bayes factor estimation, geometric perturbation methods, and asymptotic decompositions of $p(y\mid \Theta_i,\mathcal{M}_i)$6 all attempt to distinguish evidential sensitivity caused by legitimate scientific assumptions from sensitivity caused by estimator bias, misspecification, or poor posterior contraction behavior. This suggests that future work will likely continue to integrate workflow diagnostics, asymptotic theory, and computational surrogates rather than treating Bayes factor computation as a standalone numerical task (Chib et al., 2016).

Taken together, the literature portrays Bayes factor sensitivity analysis as the systematic study of how prior predictive evidence changes under perturbations of priors, models, computations, and data regimes. Its mature forms include prior predictive checking, repeated computation under MCMC variation, simulation-based calibration, functional Bayes factor surfaces, posterior-distance diagnostics, and domain-specific reporting devices such as Bayes factor functions. In this broader sense, Bayes factor sensitivity analysis is not merely an adjunct to hypothesis testing but a general framework for making Bayes-factor-based evidence scientifically interpretable and methodologically auditable (Sugasawa, 2024).