AdaFilter-Bon: Bonferroni Adaptive Filtering

• AdaFilter-Bon is a partial conjunction testing method that utilizes a Bonferroni-based p-value combination to detect replicability across multiple studies.
• It adaptively filters out unlikely candidates before applying the rejection rule, thereby reducing the conservativeness of direct multiple-testing correction.
• Empirical evaluations demonstrate that AdaFilter-Bon improves power while controlling familywise error rate and per-family error rate under independence assumptions.

Searching arXiv for recent and foundational papers on AdaFilter-Bon and related terminology. arxiv_search(query="AdaFilter-Bon partial conjunction adaptive filtering best-of-n", max_results=10) AdaFilter-Bon is the Bonferroni-based AdaFilter procedure for familywise error rate control in partial conjunction testing, designed to detect signals that replicate across multiple studies while avoiding the severe conservativeness of direct multiple-testing correction on partial conjunction $p$-values. In the formulation given in “Detecting Multiple Replicating Signals using Adaptive Filtering Procedures” (Wang et al., 2016), the method increases power by adaptively filtering out unlikely candidates of partial conjunction nulls, then applying a Bonferroni-style rejection rule on the filtered set. Later work generalized the same filtering logic to $k$-family-wise error rate control and analyzed how post-filter null proportion estimation can reduce conservativeness (Tran, 21 Aug 2025).

1. Problem setting and statistical objective

AdaFilter-Bon is defined in the setting of replicability analysis across $K$ studies and $M$ hypotheses or features. For hypothesis $j=1,\dots,M$ and study $i=1,\dots,K$, let $p_{ij}$ be the valid per-study $p$-value, and let the ordered values be $P_{(1)j} \le \cdots \le P_{(K)j}$ (Wang et al., 2016).

The target inferential object is the partial conjunction null

$$H_{0j}^{r/K}:\quad \text{fewer than } r \text{ of the } K \text{ base hypotheses are non-null}.$$

Rejecting $k$0 asserts replicability in at least $k$1 studies. This distinguishes replicability from ordinary meta-analysis: the goal is not merely to aggregate evidence across studies, but specifically to identify signals that are discovered in multiple studies (Wang et al., 2016).

For simultaneous testing, let $k$2 if $k$3 is rejected and $k$4 otherwise, with total discoveries $k$5 and false discoveries

$k$6

The principal error criteria are

$k$7

AdaFilter-Bon targets FWER and, more strongly, PFER (Wang et al., 2016).

2. Partial conjunction $k$8-values and the role of Bonferroni combination

For a single hypothesis with per-study $k$9-values $K$0, the framework admits several valid partial conjunction $K$1-values under independence. The Bonferroni-based partial conjunction $K$2-value is

$K$3

The Simes-type partial conjunction $K$4-value is

$K$5

The Fisher-type partial conjunction $K$6-value is

$K$7

These constructions apply only to the largest $K$8 $K$9-values because the least favorable null configurations are those with exactly $M$0 non-nulls. For $M$1, all three reduce to $M$2 (Wang et al., 2016).

AdaFilter-Bon uses the Bonferroni partial conjunction statistic internally. This is not an incidental design choice: the filtering argument and the resulting conditional validity inequality are stated in terms of the Bonferroni-based order-statistic construction (Wang et al., 2016). A later $M$3-FWER treatment preserved the same choice, writing for feature $M$4 and replication level $M$5,

$M$6

with $M$7 by construction (Tran, 21 Aug 2025).

3. Core construction of AdaFilter-Bon

AdaFilter-Bon attaches two statistics to each hypothesis $M$8. The filter statistic is

$M$9

and the selection statistic is

$j=1,\dots,M$0

The method’s key intuition is that hypotheses with large $j=1,\dots,M$1 are unlikely to be least favorable partial conjunction nulls, so they can be filtered before multiplicity correction (Wang et al., 2016).

The central validity statement is the conditional validity lemma. Under independence across studies, if $j=1,\dots,M$2 is true then for any fixed $j=1,\dots,M$3,

$j=1,\dots,M$4

Equivalently,

$j=1,\dots,M$5

because $j=1,\dots,M$6 always holds. This inequality is the technical device that makes adaptive thresholding valid (Wang et al., 2016).

The Bonferroni AdaFilter threshold is

$j=1,\dots,M$7

The rejection rule is

$j=1,\dots,M$8

An equivalent adjusted-$j=1,\dots,M$9-value formulation ranks the selection statistics $i=1,\dots,K$0 and defines

$i=1,\dots,K$1

$i=1,\dots,K$2

Reject $i=1,\dots,K$3 if

$i=1,\dots,K$4

This produces the same rejections as the $i=1,\dots,K$5-based rule and is the computationally convenient formulation emphasized in the procedure description (Wang et al., 2016).

4. Error control, monotonicity, and why power increases

The assumptions for finite-$i=1,\dots,K$6 FWER and PFER control are explicit. The base $i=1,\dots,K$7-values must be valid, independence across studies is required for the conditional validity lemma, and the proof of finite-$i=1,\dots,K$8 FWER/PFER control uses independence of all $i=1,\dots,K$9 base $p_{ij}$0-values (Wang et al., 2016).

Under those assumptions, AdaFilter-Bon satisfies

$p_{ij}$1

The proof strategy introduces a data-dependent auxiliary threshold $p_{ij}$2 independent of $p_{ij}$3, shows $p_{ij}$4 and $p_{ij}$5 if $p_{ij}$6, and then applies the conditional validity inequality to bound the expected number of false rejections (Wang et al., 2016).

The power advantage over the direct Bonferroni approach comes from replacing the global multiplicity burden $p_{ij}$7 by the filtered set size. Direct Bonferroni for partial conjunction tests rejects $p_{ij}$8 if $p_{ij}$9, which can be extremely conservative because the partial conjunction null is composite. In the $p$0 case, if

$p$1

with $p$2, then applying Bonferroni to $p$3 gives

$p$4

For large $p$5 and sparse signals, this bound is often far below $p$6, indicating extreme conservativeness. By contrast, the AdaFilter heuristic for $p$7 yields

$p$8

which can be much larger than $p$9 when $P_{(1)j} \le \cdots \le P_{(K)j}$0 (Wang et al., 2016).

AdaFilter-Bon does not satisfy complete monotonicity: reducing some base $P_{(1)j} \le \cdots \le P_{(K)j}$1-values can in rare cases reduce the overall threshold and hurt other hypotheses. It does satisfy partial monotonicity: decreasing any of the $P_{(1)j} \le \cdots \le P_{(K)j}$2 $P_{(1)j} \le \cdots \le P_{(K)j}$3-values for a given hypothesis $P_{(1)j} \le \cdots \le P_{(K)j}$4 cannot turn a rejection of $P_{(1)j} \le \cdots \le P_{(K)j}$5 into acceptance. The latter is the monotonicity property highlighted as the natural interpretability requirement per hypothesis (Wang et al., 2016).

A sequential extension is also given. After an initial run, remove rejected hypotheses and re-apply AdaFilter-Bon to the remaining ones, iterating until no further rejections. This “Sequential AdaFilter Bonferroni” controls FWER at $P_{(1)j} \le \cdots \le P_{(K)j}$6 and increases power (Wang et al., 2016).

5. Practical variants, computation, and empirical behavior

The method accommodates missing $P_{(1)j} \le \cdots \le P_{(K)j}$7-values or varying numbers of studies per hypothesis. If hypothesis $P_{(1)j} \le \cdots \le P_{(K)j}$8 is measured in only $P_{(1)j} \le \cdots \le P_{(K)j}$9 studies and the target replicability level is $$H_{0j}^{r/K}:\quad \text{fewer than } r \text{ of the } K \text{ base hypotheses are non-null}.$$0, the definitions become

$$H_{0j}^{r/K}:\quad \text{fewer than } r \text{ of the } K \text{ base hypotheses are non-null}.$$1

The same paper also describes sign-replicability handling by testing one-sided partial conjunction nulls for “$$H_{0j}^{r/K}:\quad \text{fewer than } r \text{ of the } K \text{ base hypotheses are non-null}.$$2 positives” and “$$H_{0j}^{r/K}:\quad \text{fewer than } r \text{ of the } K \text{ base hypotheses are non-null}.$$3 negatives” separately at $$H_{0j}^{r/K}:\quad \text{fewer than } r \text{ of the } K \text{ base hypotheses are non-null}.$$4 each, then taking the union of rejections (Wang et al., 2016).

Computationally, sorting $$H_{0j}^{r/K}:\quad \text{fewer than } r \text{ of the } K \text{ base hypotheses are non-null}.$$5 $$H_{0j}^{r/K}:\quad \text{fewer than } r \text{ of the } K \text{ base hypotheses are non-null}.$$6-values for each of $$H_{0j}^{r/K}:\quad \text{fewer than } r \text{ of the } K \text{ base hypotheses are non-null}.$$7 hypotheses costs $$H_{0j}^{r/K}:\quad \text{fewer than } r \text{ of the } K \text{ base hypotheses are non-null}.$$8, and the adjusted-$$H_{0j}^{r/K}:\quad \text{fewer than } r \text{ of the } K \text{ base hypotheses are non-null}.$$9-value implementation requires an additional pass over $k$00 and $k$01. An R package implementing AdaFilter is available at https://github.com/jingshuw/adaFilter (Wang et al., 2016).

The simulation regime reported for AdaFilter-Bon used $k$02, $k$03, and sparse signals. At PFER target $k$04, direct Bonferroni on partial conjunction $k$05-values had PFER $k$06–$k$07 with recall $k$08–$k$09 for $k$10, and PFER $k$11–$k$12 with recall $k$13–$k$14 for $k$15. AdaFilter-Bon yielded $k$16: PFER $k$17–$k$18, recall $k$19–$k$20; and $k$21: PFER $k$22–$k$23, recall $k$24–$k$25 (Wang et al., 2016).

At FWER target $k$26, direct Bonferroni had FWER $k$27 with recall $k$28–$k$29, original AdaFilter-Bon had FWER $k$30–$k$31 with recall $k$32–$k$33, and Sequential AdaFilter-Bon had FWER $k$34–$k$35 with recall $k$36–$k$37 (Wang et al., 2016).

Although the real applications in the original paper primarily emphasized AdaFilter-BH rather than AdaFilter-Bon, they were presented as corroborating the core AdaFilter advantage across microarray, single-cell RNA sequencing, and metabolomics GWAS settings (Wang et al., 2016).

6. Conservative filtering, $k$38-FWER extensions, and terminological ambiguities

A later development analyzed AdaFilter-Bon through the lens of $k$39-family-wise error rate. In that formulation, for replication level $k$40 across $k$41 studies and $k$42 features, the generalized threshold is

$k$43

and the decision rule rejects $k$44 if $k$45 (Tran, 21 Aug 2025).

That work also formalized a source of conservativeness induced by filtering. If

$k$46

then AdaFilter-Bon satisfies the finite-sample upper bound

$k$47

This shows that effective filtering can make the realized error level substantially smaller than $k$48, with an accompanying power loss (Tran, 21 Aug 2025).

To mitigate this effect, AdaFilter-AdaBon introduces the post-filter null proportion estimator

$k$49

and replaces the rejection threshold by

$k$50

Under weak dependence and the stated limit assumptions, the paper proves asymptotic $k$51-FWER control and reports higher power than the original AdaFilter-Bon in simulations, with $k$52 described as a good default in experiments (Tran, 21 Aug 2025).

A separate source of ambiguity is terminological rather than statistical. In recent language-model work, a similarly spelled “AdaFilter-BoN” refers to adaptive filtering around Best-of-$k$53 inference or to consensus filtering for Process Reward Model training, not to partial conjunction multiple testing. “ROC-n-reroll: How verifier imperfection affects test-time scaling” uses “AdaFilter-BoN” for adaptive filtering Best-of-$k$54 under an imperfect verifier, with ROC-aware selection of $k$55 and thresholds $k$56 and switching or combining BoN with rejection sampling (Dorner et al., 16 Jul 2025). “The Lessons of Developing Process Reward Models in Mathematical Reasoning” describes a consensus filtering mechanism that integrates MC estimation with LLM-as-a-judge and states that this is exactly the goal of an “AdaFilter-BoN”-style approach (Zhang et al., 13 Jan 2025). These usages share the idea of adaptive filtering, but they are distinct from AdaFilter-Bon as a Bonferroni-based procedure for partial conjunction testing.

In its original and most specific sense, AdaFilter-Bon denotes a replicability-testing method that combines order-statistic partial conjunction $k$57-values with adaptive filtering to control FWER or PFER while substantially reducing the conservativeness of direct Bonferroni correction. Its later refinements chiefly address the residual conservativeness introduced by the filtering step itself (Wang et al., 2016).