Test-based Bayes Factors

• Test-based Bayes factors (TBFs) are Bayesian methods that derive evidence from the sampling distribution of test statistics using priors on effect parameters.
• They provide closed-form or numerically tractable solutions, making them practical for meta-analytic settings and studies with only summary statistics.
• TBFs accommodate both local and non-local priors, with non-local choices enhancing evidence calibration and better controlling Type I error rates.

Test-based Bayes factors (TBFs) constitute a class of Bayesian hypothesis testing methodologies in which the Bayes factor is computed directly from the sampling distribution of a test statistic (such as a $z$, $t$, $F$, $\chi^2$ value, or a suitably standardized nonparametric statistic) and a prior placed on its noncentrality parameter or other effect-indexing quantity. This paradigm provides closed-form or numerically tractable expressions for Bayes factors in a broad range of testing problems, bypassing the need for full-likelihood models or specification of high-dimensional priors on nuisance parameters. TBFs have become a prominent approach in meta-analytic settings, scientific replication studies, robust evidence calibration, and situations where only summary statistics are available rather than raw data. Their foundational structure allows for alternative prior specifications—such as local (e.g., Gaussian) or non-local (e.g., inverse-moment or moment) priors—whose choice has substantial impact on the evidential properties and frequentist error control of the resulting Bayes factor.

1. Theoretical Foundations of TBFs

Let $T$ be a scalar or low-dimensional test statistic with known sampling distribution $f_0(t)$ under the null hypothesis $H_0$ and alternative distribution $f_1(t)$ under $H_1$ (possibly parameterized by a noncentrality parameter $\lambda$ or standardized effect size $t$0). The generic TBF is defined as

$t$1

where $t$2 is the marginal predictive distribution of the test statistic under $t$3 with prior $t$4 on the effect size parameter. When priors are chosen as normals, Cauchy, moment, or inverse-moment types centered at scientifically meaningful effect sizes, one obtains a "Bayes factor function" (BFF) mapping each candidate $t$5 or $t$6 to the value of the Bayes factor (Datta et al., 20 Jun 2025, Datta et al., 2023, Johnson et al., 2022).

Formally, when $t$7 corresponds to $t$8 and the alternative is indexed by $t$9 (e.g., $F$0 for one-sample $F$1), typical BFF constructions take the form

$F$2

where $F$3 is a function of the prior order $F$4 and statistic, and $F$5 is a (confluent or hypergeometric) function characterizing the marginal (Datta et al., 2023).

Alternative approaches use a likelihood-based predictive distribution for $F$6 under a mixing prior on $F$7 (e.g., normal, Cauchy, gamma), leading to test-calibrated closed formulas for many classical test situations (Lopes et al., 2018, Johnson et al., 2022). For nonparametric statistics, such as Kendall's $F$8 or partial correlation coefficients, recent work leverages the asymptotic distribution of standardized statistics and places priors on the noncentrality parameter, again yielding TBFs in closed form (Zhang et al., 2021, Datta et al., 13 Mar 2025).

2. Comparative Approaches: Local and Non-Local Priors

A salient feature distinguishing TBF methodologies is the nature of the prior assigned to the effect parameter under $F$9. Local priors—such as the normal or Cauchy—assign nonzero mass in any neighborhood of the null. These yield Bayes factors that are less aggressive in accumulating evidence for $\chi^2$0, particularly for small effect sizes or moderate $\chi^2$1 values. Non-local priors—especially the inverse-moment priors (iMOM) and normal moment priors—enforce $\chi^2$2 and accumulate evidence for the null much more rapidly when the observed statistic is small. The iMOM prior of order $\chi^2$3 is

$\chi^2$4

which, when centered on $\chi^2$5, can be tailored so that the mode is at the hypothesized effect size (Datta et al., 20 Jun 2025, Datta et al., 2023).

This distinction has substantial operating characteristics. Non-local TBFs provide polynomial rates of evidence accumulation toward $\chi^2$6 under the null, contrasting with the slower rates of local prior-based Bayes factors, thereby offering substantially better control of Type I error while retaining exponential evidence under the alternative (Datta et al., 2023).

3. Practical Computation and Closed-form Results

For the most frequently encountered test statistics—$\chi^2$7, $\chi^2$8, $\chi^2$9, $T$0—TBFs admit closed-form or one-dimensional integral expressions for the Bayes factor. For example, with a normal moment prior (order $T$1) on the noncentrality parameter in the $T$2-test, one obtains (Datta et al., 2023, Johnson et al., 2022): $T$3 and, for a $T$4-test,

$T$5

with $T$6 and $T$7 as defined above, and $T$8 the degrees of freedom.

For the Bayesian $T$9-test with an arbitrary $f_0(t)$0-prior $f_0(t)$1 on the standardized effect, the Bayes factor reduces to a single one-dimensional integral: $f_0(t)$2 facilitating numerical evaluation even in subjective or "informed" prior settings (Gronau et al., 2017).

Test-based BIC approximations are further available when only summary statistics (e.g., $f_0(t)$3, $f_0(t)$4, $f_0(t)$5) are reported, with the standard formula for ANOVA: $f_0(t)$6 which is widely adopted for meta-analysis or reanalysis of published results (Faulkenberry, 2018).

4. Interpretation, Calibration, and Evidence Scales

TBFs yield a direct, ratio-scale measure of evidential support for $f_0(t)$7 versus $f_0(t)$8, rendering them fundamentally different from $f_0(t)$9-values which do not quantify support for $H_0$0. The interpretive framework for TBFs is based on established Bayes factor scales (e.g., $H_0$1–$H_0$2: weak, $H_0$3–$H_0$4: positive, $H_0$5: very strong) or, alternatively, via log-scales such as the base-$H_0$6 units advocated in empirical Bayes settings (Dudbridge, 2023). The continuous mapping $H_0$7 provided by BFFs supplies a full weight-of-evidence curve, permitting scientific interpretation of what effect sizes are supported or disfavored by the data (Datta et al., 20 Jun 2025, Johnson et al., 2022). This approach also rigorously exposes Lindley’s paradox, in which Bayes factors may support $H_0$8 even when the corresponding test is “significant” at a given $H_0$9.

TBFs indexed by prior scale (e.g., $f_1(t)$0 matching a standardized effect size) allow practitioners to plot $f_1(t)$1BF as a function of hypothesized alternative magnitude, replacing dichotomous hypothesis testing with an evidential continuum.

5. Applications to Meta-analysis, Replication, and Nonparametric Tests

Because TBFs can be computed solely from summary statistics, they enable evidence synthesis and meta-analytic aggregation across studies. The BFF formalism is especially conducive to replication science, where independent studies’ BFFs are multiplicatively combined: $f_1(t)$2 This property holds for $f_1(t)$3, $f_1(t)$4, $f_1(t)$5, $f_1(t)$6 statistics, as well as for generalized statistics such as partial correlation $f_1(t)$7-values (Datta et al., 13 Mar 2025), and for Fisher- or Wilcoxon-type statistics under rank tests (Zhang et al., 2021).

The extension to nonparametric settings (e.g., Kendall’s $f_1(t)$8), leverages asymptotic normality of standardized test statistics, specifying working likelihoods and priors for the noncentrality parameter, yielding closed-form Bayes factors interpretable under large-sample limits (Zhang et al., 2021). Consistency and frequentist error control are preserved under mild regularity and appropriate prior scaling.

6. Simulation Performance and Frequentist Properties

Large-scale simulation studies of TBFs with both local and non-local priors demonstrate that non-local moment priors (e.g., iMOM, normal-moment priors) yield Type I error rates near or below nominal $f_1(t)$9 thresholds and display superior power curves in moderate to large effect size regimes, especially compared to default $H_1$0-prior or JZS Bayes factors (Datta et al., 20 Jun 2025, Datta et al., 2023). Under the null, TBF/BFF log-evidence is strongly negative, favoring $H_1$1 even at moderate samples, and rapidly achieves “strong evidence” thresholds. Under the alternative, the log-BF growth is exponential in $H_1$2.

TBF approximations closely track exact Bayes factors produced by software such as the R BayesFactor package, achieving >98% agreement in model decision across a range of designs and sample sizes (Faulkenberry, 2018).

7. Limitations, Extensions, and Practical Considerations

TBFs, while robust and computationally efficient, depend on correct specification of the null and alternative distributions of the test statistic. When the test statistic’s distribution under the alternative is uncertain or involves substantial nuisance structure, care must be taken with prior selection and interpretability. For small sample sizes or in highly unbalanced designs, approximate closed-form or BIC-based TBFs can slightly underestimate evidence for the null or diverge from full Bayesian computations (Faulkenberry, 2018).

Alternative priors, including data-driven or unit-information forms, may be substituted for the standard normal, Cauchy, or moment priors, although closed-form curves may be lost. Empirical Bayes and multiple testing extensions further leverage ensembles of test statistics to calibrate and regularize TBFs across large-scale studies (Dudbridge, 2023).

Reporting conventions developed in this literature emphasize clear specification of (i) the formula used, (ii) the prior and its calibration, (iii) the precise test statistic, degrees of freedom, and sample size, and (iv) interpretive context using an absolute evidence scale (Faulkenberry, 2018, Johnson et al., 2022).

---

• "Computing Bayes factors to measure evidence from experiments: An extension of the BIC approximation" (Faulkenberry, 2018)
• "Empirical Bayes factors for common hypothesis tests" (Dudbridge, 2023)
• "Bayesian Hypothesis Testing: Redux" (Lopes et al., 2018)
• "Bayes factor functions for reporting outcomes of hypothesis tests" (Johnson et al., 2022)
• "Informed Bayesian T-Tests" (Gronau et al., 2017)
• "On Bayes factor functions" (Datta et al., 20 Jun 2025)
• "A simple consistent Bayes factor for testing the Kendall rank correlation coefficient" (Zhang et al., 2021)
• "Bayes Factors Based on Test Statistics and Non-Local Moment Prior Densities" (Datta et al., 2023)
• "Bayes factor functions for testing partial correlation coefficients" (Datta et al., 13 Mar 2025)