Bayesian Hypothesis Testing: Bayes Factors

• Bayesian hypothesis testing via Bayes factors is a method for model selection that compares marginal likelihoods, integrating over parameter uncertainty to quantify evidence.
• Closed-form solutions like the Pearson Bayes factor for t-tests and ANOVA offer efficient evidence quantification without requiring raw data integration.
• Robust computational strategies, including Laplace approximations and non-local priors, ensure error control and unite Bayesian and frequentist inference principles.

Bayesian hypothesis testing via Bayes factors is a foundational methodology for model selection and hypothesis evaluation, quantifying the relative evidential support for competing scientific statements on the basis of observed data. The approach is characterized by direct comparison of the marginal likelihoods (model evidences) of each hypothesis, integrating over parameter uncertainty according to specified prior distributions. Recent developments have produced both closed-form solutions for common test settings and robust computational approaches, advancing the practical application of Bayes factors in both classical and modern inference workflows.

1. Foundations of Bayes Factor Hypothesis Testing

The Bayes factor for hypotheses $\mathcal{H}_0$ and $\mathcal{H}_1$ is defined as the ratio of their marginal likelihoods: $$\mathrm{BF}_{10} = \frac{p(\mathcal{D} \mid \mathcal{H}_1)}{p(\mathcal{D} \mid \mathcal{H}_0)} = \frac{\int p(\mathcal{D} \mid \theta_1) \, p(\theta_1 \mid \mathcal{H}_1) \, d\theta_1}{ \int p(\mathcal{D} \mid \theta_0) \, p(\theta_0 \mid \mathcal{H}_0) \, d\theta_0 }$$ where $p(\theta_j \mid \mathcal{H}_j)$ are the respective priors and $p(\mathcal{D} \mid \theta_j)$ the likelihoods. For point nulls, $p(\mathcal{D} \mid \mathcal{H}_0)$ reduces to the likelihood at $\theta_0$.

Bayes factors quantify the degree to which the data favor one hypothesis over another, providing a graded, symmetric scale of evidence. Posterior odds are given via

$$\text{posterior odds} = \text{prior odds} \times \text{Bayes factor.}$$

This forms the core of Bayesian evidence quantification, enabling both detection and discrimination between null and alternative, in contrast to $p$-value–based methodologies, which lack symmetric treatment of $\mathcal{H}_0$ and $\mathcal{H}_1$ (Mulder et al., 27 Nov 2025).

2. Exact Methods and Closed-Form Bayes Factors

A substantial contribution is the derivation of analytic, closed-form Bayes factors for widely-used test settings—specifically, the Pearson Bayes factor (PBF) for the two-sample $t$-test and one-way ANOVA (Faulkenberry, 2020, Wang et al., 2015).

Two-Sample $t$-Test (PBF)

Under the random-effects model

$$Y_{ij} = \mu + a_i + \epsilon_{ij}, \quad a_i \sim N(0, \sigma_a^2),\ \epsilon_{ij} \sim N(0, \sigma^2)$$

the null $\mathcal{H}_0$ corresponds to $\sigma_a^2=0$; the alternative, $\sigma_a^2 > 0$. Employing a Pearson Type VI prior on the variance ratio $\tau = \sigma_a^2 / \sigma^2$: $$\pi(\tau) = \frac{\kappa ( \kappa \tau )^\beta (1 + \kappa \tau)^{-\alpha-\beta-2}}{B(\alpha + 1, \beta + 1)}$$ and with specific choices $\kappa = r$, $\beta = (n-p)/2 - \alpha - 2$, $\alpha \in [-1/2, 0]$, the Bayes factor reduces to

$$\mathrm{PBF}_{10} = \frac{\Gamma(\nu/2) \, \Gamma(\alpha+3/2)}{\Gamma((\nu+1)/2) \, \Gamma(\alpha+1)} \left(1+\frac{t^2}{\nu}\right)^{(\nu-2\alpha-2)/2}$$

where $t$ is the observed $t$-statistic and $\nu$ the degrees of freedom. For ANOVA $F$-tests, a direct generalization holds: $$\mathrm{PBF}_{10} = \frac{ \Gamma( x/2 + \alpha + 1 ) \Gamma( y/2 ) } { \Gamma( (x + y)/2 ) \Gamma( \alpha + 1 ) } \left( \frac{y}{y + x F} \right)^{ \alpha - y/2 + 1 }$$ with between-group $df = x$, within-group $df = y$.

This closed-form eliminates the need for integrating over the prior, requiring only minimal summary statistics (e.g., $t$, $F$, $df$s), and facilitates retrospective evidence quantification when raw data are unavailable. A simulation paper demonstrates that, for the balanced one-way ANOVA design, the PBF is more conservative under the null relative to JZS default Bayes factors, yet similar to BIC-based approximations (Faulkenberry, 2020).

Avoidance of Paradoxes

The model circumvents both Bartlett’s paradox (improper priors leading to universal support for the null) and the information paradox (Bayes factor failing to increase with $|t|$) by introducing a heavy-tailed prior and integrating out the variance ratio, ensuring consistency and robustness (Wang et al., 2015).

3. Approximations, Computational Strategies, and Robust Default Methods

For larger and more complex models or when only summary or maximum likelihood estimates are available, a variety of fast and accurate approximations have been developed:

• Laplace/Gaussian Approximation: Under regularity, the marginal likelihood can be approximated by Laplace’s method, underpinning both the BIC-based Bayes factor and the Savage–Dickey density ratio approximation (Bartoš et al., 2022, Martin et al., 2021, Faulkenberry, 2018). The general BIC formula for nested models is

$$\log \mathrm{BF}_{10} \approx -\frac{1}{2} ( \mathrm{BIC}_1 - \mathrm{BIC}_0 )$$

where $\mathrm{BIC}_k = -2 \log L_k + d_k \log n$ for model $k$, with $L_k$ the maximum likelihood, $d_k$ the number of free parameters, and $n$ the sample size (Martin et al., 2021).

• Savage–Dickey Normal Approximation: For $\mathcal{H}_0: \theta = \theta_0$ vs.\ $\mathcal{H}_1: \theta \sim g(\theta)$ with $g$ normal, the Bayes factor can be approximated as:

$$\mathrm{BF}_{01} \approx \sqrt{ \frac{ \sigma_0^2 + \mathrm{SE}_{\hat\theta}^2 }{ \mathrm{SE}_{\hat\theta}^2 } } \times \exp \left(-\frac{1}{2} \left[ \frac{(\hat\theta - \theta_0)^2}{\mathrm{SE}_{\hat\theta}^2} - \frac{(\hat\theta - \mu_0)^2}{\sigma_0^2 + \mathrm{SE}_{\hat\theta}^2} \right] \right)$$

with $\hat\theta$ the MLE and $\mathrm{SE}_{\hat\theta}$ its standard error (Bartoš et al., 2022).

• Empirical Bayes Factors: The empirical Bayes factor employs a posterior "prior" from the observed sample, with analytic bias correction (e.g., $$\mathrm{EBF}_{01}(x) = \sqrt{2}\,\mathrm{e}^{-(z^2-1)/2}$$ for the normal location model) and close relationship to widely applicable information criteria (Dudbridge, 2023).
• Computation from Fitted Models: For standard least squares or maximum likelihood fits, the model evidence can be estimated with a Laplace–Gaussian formula using fitted covariance and prior parameter ranges, allowing Occam's razor to be fully quantified and applied (Dunstan et al., 2020).

4. Prior Specification and the Role of Non-Local Priors

Bayesian hypothesis testing via Bayes factors is sensitive to the specification of the prior under the alternative hypothesis. Several considerations emerge:

• Symmetry and Placement: Priors for standardized effect sizes are typically centered at zero under the alternative to ensure symmetry. Common selections are $\mathcal{N}(0,1)$ or Cauchy, the latter mitigating the Bartlett–Lindley paradox in two-sample inference (Mulder et al., 27 Nov 2025, Wang et al., 2015).
• Non-Local Priors: Non-local priors, such as normal-moment densities $j(\lambda \mid \tau^2, r)$ with $$j(\lambda) = |\lambda|^{2r}(2\tau^2)^{-r-1/2} \Gamma(r+1/2)^{-1} \exp(-\lambda^2/(2\tau^2))$$, enforce $\pi(\lambda)=0$ at the null and accelerate evidence accumulation for the null hypothesis. Hyperparameters can be chosen to center modal prior mass at a scientifically meaningful effect size (Datta et al., 2023, Johnson et al., 2022).
• Calibration: Vague or excessively broad priors can cause Bayes factors to spuriously favor the null. Sensitivity to prior width is well-documented and motivates default or empirically justified selections (Mulder et al., 27 Nov 2025). Non-local priors improve polynomial rates of evidence accumulation for the true null, as shown analytically for $t$ and $F$ tests (Datta et al., 2023).
• Interval Null Hypotheses: In clinical contexts, interval nulls ($|\theta - \theta_0| \leq \Delta$) are often more meaningful. Bayes factors for interval nulls can be computed directly from test statistics using nonlocal priors on the corresponding noncentrality parameter, with demonstration of frequentist type I error calibration (Chakraborty et al., 21 Feb 2024).

5. Methodological Properties: Error Control, Frequentist Optimality, and Practical Implications

Bayesian tests via Bayes factors possess both Bayesian and frequentist optimality properties.

• Error Rate Control: With appropriate choice of prior and threshold, Bayesian tests can be calibrated to classical type I error rates. For monotone likelihood ratio families, the test statistic $B(X)$ (the Bayes factor) is a monotone function of the classical statistic, and rejecting when $B(X) \leq k(\alpha)$ matches the power and size of the classical uniformly most powerful (UMP) test (Shively et al., 2013, Fowlie, 2021). In settings with nuisance parameters, using Jeffreys' priors preserves this property.
• Neyman–Pearson Optimality: The Bayes factor, thresholded to control (Bayesian average) type I error at a fixed level, is Neyman–Pearson optimal in maximizing power (or minimizing type II error) among all tests of fixed size (Fowlie, 2021).
• Evidence Accumulation and Sequential Inference: Bayes factors are coherently updated with new data (via multiplication), and retain validity under optional stopping (Mulder et al., 27 Nov 2025). This coherence extends to meta-analyses, where evidence is accumulated over studies without error inflation, in contrast to repeated $p$-value testing.

6. Bayes Factor Functions and Unified Bayesian Inference

The Bayes factor function (BFF) generalizes the traditional Bayes factor by expressing it as a function of a hypothesized effect size or parameter value, mapping standardized effect sizes to evidence for or against the null (Johnson et al., 2022, Pawel, 14 Mar 2024). For standard tests (e.g., $z$, $t$, $F$, $\chi^2$), closed-form BFFs parameterized by dispersion effectively link observed statistics to interpretable effect-size scales. Plots of the BFF versus effect size eliminate the need for arbitrary significance thresholds and allow aggregation across independent studies by direct multiplication of BFFs (Johnson et al., 2022).

This functional approach also yields "support curves" that enable:

• Inversion for point estimates—maximum evidence estimates (MEE),
• Construction of support intervals (intervals of parameter values with at least $k$-to-1 evidence in their favor),
• Transparent presentation of evidence across the parameter space,
• Direct exploitation for meta-analysis, replication studies, and general hypothesis calibration (Pawel, 14 Mar 2024).

7. Robustness, Sensitivity Analysis, and Workflow Best Practices

Bayes factor analyses require scrutiny of robustness and workflow engineering, including:

• Estimation Robustness: Several computational strategies (bridge sampling, Laplace approximations, MCMC-based methods, INLA) are used to estimate marginal likelihoods, with reproducibility assessed via repeated chain fits and simulation-based calibration (SBC). SBC compares average posterior probabilities against true generating model frequencies and monitors BF estimator bias (Schad et al., 2021, Mulder et al., 27 Nov 2025).
• Prior and Data Sensitivity: Prior predictive and posterior predictive checks diagnose the plausibility of modeling assumptions and explore variability in Bayes factor behavior across datasets or prior choices, as wide variation in computed Bayes factors may signal data or prior incompatibility (Schad et al., 2021).
• Utility-Based Decisions: While Bayes factors provide continuous gradations of evidence, decision rules (e.g., threshold-based declaration) require explicit specification of utility or loss functions, with Bayes-optimal actions derived by maximizing expected utility under the posterior (Schad et al., 2021).
• Best-Practice Workflow:
  • Specify model and priors; validate them by prior predictive simulation.
  • Fit models, check MCMC diagnostics, compute Bayes factors.
  • Calibrate and check stability (multiple chains, restarts).
  • Employ SBC to assess estimator bias.
  • Conduct sensitivity analysis (prior width, data resampling).
  • If making discrete decisions, specify and optimize utility.

A carefully documented, simulation-calibrated workflow is essential, especially for robust application to complex models or if results will guide policy or clinical action (Schad et al., 2021).

---

Bayesian hypothesis testing via Bayes factors provides a mathematically principled and practically viable approach to model selection and hypothesis assessment. Recent analytic developments (e.g., the Pearson Bayes factor) and robust computational procedures enable its application from classical statistical problems to large-scale, high-dimensional evidence synthesis. The framework is notable for unifying Bayesian and frequentist perspectives via optimal error control, prior-anchored grading of evidence, and direct quantification of support for the null, thus offering a coherent alternative to $p$-value–based inference in both foundational and applied statistical research (Faulkenberry, 2020, Mulder et al., 27 Nov 2025, Fowlie, 2021).