---
title: 'AdaFilter-Bon: Bonferroni Adaptive Filtering'
url: https://www.emergentmind.com/topics/adafilter-bon
type: topic
---

# AdaFilter-Bon: Bonferroni Adaptive Filtering

Searching arXiv for recent and foundational papers on AdaFilter-Bon and related terminology.
arxiv_search(query="AdaFilter-Bon partial conjunction adaptive filtering best-of-n", max_results=10)
AdaFilter-Bon is the Bonferroni-based AdaFilter procedure for familywise error rate control in partial conjunction testing, designed to detect signals that replicate across multiple studies while avoiding the severe conservativeness of direct multiple-testing correction on partial conjunction \(p\)-values. In the formulation given in “Detecting Multiple Replicating Signals using Adaptive Filtering Procedures” [1610.03330], the method increases power by adaptively filtering out unlikely candidates of partial conjunction nulls, then applying a Bonferroni-style rejection rule on the filtered set. Later work generalized the same filtering logic to \(k\)-family-wise error rate control and analyzed how post-filter null proportion estimation can reduce conservativeness [2508.15363].

## 1. Problem setting and statistical objective

AdaFilter-Bon is defined in the setting of replicability analysis across \(K\) studies and \(M\) hypotheses or features. For hypothesis \(j=1,\dots,M\) and study \(i=1,\dots,K\), let \(p_{ij}\) be the valid per-study \(p\)-value, and let the ordered values be \(P_{(1)j} \le \cdots \le P_{(K)j}\) [1610.03330].

The target inferential object is the partial conjunction null
\[
H_{0j}^{r/K}:\quad \text{fewer than } r \text{ of the } K \text{ base hypotheses are non-null}.
\]
Rejecting \(H_{0j}^{r/K}\) asserts replicability in at least \(r\) studies. This distinguishes replicability from ordinary meta-analysis: the goal is not merely to aggregate evidence across studies, but specifically to identify signals that are discovered in multiple studies [1610.03330].

For simultaneous testing, let \(\varphi_j = 1\) if \(H_{0j}^{r/K}\) is rejected and \(0\) otherwise, with total discoveries \(R=\sum_{j=1}^M \varphi_j\) and false discoveries
\[
V = \sum_{j=1}^M \varphi_j \cdot 1_{\{H_{0j}^{r/K}\text{ is true}\}}.
\]
The principal error criteria are
\[
\text{FWER} = P(V \ge 1),\qquad \text{PFER} = E(V),\qquad \text{FDR} = E(\mathrm{FDP}),\ \mathrm{FDP}=\frac{V}{R\vee 1}.
\]
AdaFilter-Bon targets FWER and, more strongly, PFER [1610.03330].

## 2. Partial conjunction \(p\)-values and the role of Bonferroni combination

For a single hypothesis with per-study \(p\)-values \(P_1,\ldots,P_K\), the framework admits several valid partial conjunction \(p\)-values under independence. The Bonferroni-based partial conjunction \(p\)-value is
\[
P_{r/K}^{B} = (K-r+1)\,P_{(r)}.
\]
The Simes-type partial conjunction \(p\)-value is
\[
P_{r/K}^{S} = \min_{r \le i \le K} \left\{ \frac{K-r+1}{i-r+1}\,P_{(i)} \right\}.
\]
The Fisher-type partial conjunction \(p\)-value is
\[
P_{r/K}^{F} = P\!\left( \chi^2_{\,2(K-r+1)} \ge -2\sum_{i=r}^K \log P_{(i)} \right).
\]
These constructions apply only to the largest \(K-r+1\) \(p\)-values because the least favorable null configurations are those with exactly \(r-1\) non-nulls. For \(K=r=2\), all three reduce to \(\max(P_1,P_2)\) [1610.03330].

AdaFilter-Bon uses the Bonferroni partial conjunction statistic internally. This is not an incidental design choice: the filtering argument and the resulting conditional validity inequality are stated in terms of the Bonferroni-based order-statistic construction [1610.03330]. A later \(k\)-FWER treatment preserved the same choice, writing for feature \(i\) and replication level \(u\),
\[
S_i = (n-u+1)\,P_{i(u)},\qquad F_i = (n-u+1)\,P_{i(u-1)},
\]
with \(F_i \le S_i\) by construction [2508.15363].

## 3. Core construction of AdaFilter-Bon

AdaFilter-Bon attaches two statistics to each hypothesis \(j\). The filter statistic is
\[
F_j := (K-r+1)\,P_{(r-1)j},
\]
and the selection statistic is
\[
S_j := P_{r/K,j}^{B} = (K-r+1)\,P_{(r)j}.
\]
The method’s key intuition is that hypotheses with large \(F_j\) are unlikely to be least favorable partial conjunction nulls, so they can be filtered before multiplicity correction [1610.03330].

The central validity statement is the conditional validity lemma. Under independence across studies, if \(H_{0j}^{r/K}\) is true then for any fixed \(\gamma>0\),
\[
P(S_j < \gamma \mid F_j < \gamma)\ \le\ \gamma.
\]
Equivalently,
\[
P(S_j < \gamma) \le \gamma\cdot P(F_j < \gamma),
\]
because \(S_j \ge F_j\) always holds. This inequality is the technical device that makes adaptive thresholding valid [1610.03330].

The Bonferroni AdaFilter threshold is
\[
\gamma_0^{\mathrm{Bon}} = \sup\Big\{\gamma \in [0,\alpha]\ \Big|\ \gamma \cdot \sum_{j=1}^M 1_{\{F_j < \gamma\}} \le \alpha\Big\}.
\]
The rejection rule is
\[
\text{Reject } H_{0j}^{r/K}\quad \text{if}\quad S_j < \gamma_0^{\mathrm{Bon}}.
\]

An equivalent adjusted-\(p\)-value formulation ranks the selection statistics \(S_{(1)} \le \cdots \le S_{(M)}\) and defines
\[
m_{(j)}^{\mathrm{AF}} = \sum_{h=1}^M 1_{\{F_h \le S_{(j)}\}},
\]
\[
P_{(j)}^{\mathrm{Bon}} = S_{(j)}\cdot m_{(j)}^{\mathrm{AF}}.
\]
Reject \(H_{0(j)}^{r/K}\) if
\[
P_{(j)}^{\mathrm{Bon}} < \alpha.
\]
This produces the same rejections as the \(\gamma_0^{\mathrm{Bon}}\)-based rule and is the computationally convenient formulation emphasized in the procedure description [1610.03330].

## 4. Error control, monotonicity, and why power increases

The assumptions for finite-\(M\) FWER and PFER control are explicit. The base \(p\)-values must be valid, independence across studies is required for the conditional validity lemma, and the proof of finite-\(M\) FWER/PFER control uses independence of all \(K\times M\) base \(p\)-values [1610.03330].

Under those assumptions, AdaFilter-Bon satisfies
\[
E(V)\ \le\ \alpha,\qquad P(V\ge 1)\ \le\ \alpha.
\]
The proof strategy introduces a data-dependent auxiliary threshold \(\gamma_j\) independent of \((F_j,S_j)\), shows \(\gamma_j \le \gamma_0^{\mathrm{Bon}}\) and \(\gamma_j=\gamma_0^{\mathrm{Bon}}\) if \(F_j<\gamma_0^{\mathrm{Bon}}\), and then applies the conditional validity inequality to bound the expected number of false rejections [1610.03330].

The power advantage over the direct Bonferroni approach comes from replacing the global multiplicity burden \(M\) by the filtered set size. Direct Bonferroni for partial conjunction tests rejects \(j\) if \(P_{r/K,j}\le \alpha/M\), which can be extremely conservative because the partial conjunction null is composite. In the \(r=K\) case, if
\[
\mathcal{I}_k=\Big\{ j\in\{1,\ldots,M\}\,\big|\, \text{exactly }k\text{ of }(H_{01j},\ldots,H_{0Kj})\text{ are false}\Big\},
\]
with \(\delta_k=|\mathcal{I}_k|/M\), then applying Bonferroni to \(P_{(K)j}\) gives
\[
E(V)\ \le\ \alpha\Bigl\{\delta_{K-1} + \delta_{K-2}\frac{\alpha}{M} + \cdots + \delta_0 \Bigl(\frac{\alpha}{M}\Bigr)^{K-1} \Bigr\}.
\]
For large \(M\) and sparse signals, this bound is often far below \(\alpha\), indicating extreme conservativeness. By contrast, the AdaFilter heuristic for \(r=K\) yields
\[
\gamma^{\mathrm{Bon}} \approx \frac{\alpha}{M(\delta_K+\delta_{K-1})} + o\Big(\frac{1}{M}\Big),
\]
which can be much larger than \(\alpha/M\) when \(\delta_K+\delta_{K-1}\ll 1\) [1610.03330].

AdaFilter-Bon does not satisfy complete monotonicity: reducing some base \(p\)-values can in rare cases reduce the overall threshold and hurt other hypotheses. It does satisfy partial monotonicity: decreasing any of the \(K\) \(p\)-values for a given hypothesis \(j\) cannot turn a rejection of \(H_{0j}^{r/K}\) into acceptance. The latter is the monotonicity property highlighted as the natural interpretability requirement per hypothesis [1610.03330].

A sequential extension is also given. After an initial run, remove rejected hypotheses and re-apply AdaFilter-Bon to the remaining ones, iterating until no further rejections. This “Sequential AdaFilter Bonferroni” controls FWER at \(\alpha\) and increases power [1610.03330].

## 5. Practical variants, computation, and empirical behavior

The method accommodates missing \(p\)-values or varying numbers of studies per hypothesis. If hypothesis \(j\) is measured in only \(K_j\) studies and the target replicability level is \(r_j \le K_j\), the definitions become
\[
F_j = (K_j-r_j+1)\,P_{(r_j-1)j},\qquad S_j = (K_j-r_j+1)\,P_{(r_j)j}.
\]
The same paper also describes sign-replicability handling by testing one-sided partial conjunction nulls for “\(\ge r\) positives” and “\(\ge r\) negatives” separately at \(\alpha/2\) each, then taking the union of rejections [1610.03330].

Computationally, sorting \(K\) \(p\)-values for each of \(M\) hypotheses costs \(O(MK\log K)\), and the adjusted-\(p\)-value implementation requires an additional pass over \(F_j\) and \(S_j\). An R package implementing AdaFilter is available at `https://github.com/jingshuw/adaFilter` [1610.03330].

The simulation regime reported for AdaFilter-Bon used \(M=10{,}000\), \(K\in\{2,4,8\}\), and sparse signals. At PFER target \(\alpha=1\), direct Bonferroni on partial conjunction \(p\)-values had PFER \(\approx 0.04\)–\(0.06\) with recall \(\approx 14.7\)–\(19.5\%\) for \(\pi_{00}=0.8\), and PFER \(\approx 0.00\)–\(0.01\) with recall \(\approx 14.7\)–\(19.4\%\) for \(\pi_{00}=0.98\). AdaFilter-Bon yielded \(\pi_{00}=0.8\): PFER \(\approx 0.73\)–\(0.76\), recall \(\approx 28.7\)–\(28.9\%\); and \(\pi_{00}=0.98\): PFER \(\approx 0.21\)–\(0.29\), recall \(\approx 38.1\)–\(38.3\%\) [1610.03330].

At FWER target \(\alpha=0.05\), direct Bonferroni had FWER \(\approx 0.00\) with recall \(\approx 15.1\)–\(21.7\%\), original AdaFilter-Bon had FWER \(\approx 0.01\)–\(0.03\) with recall \(\approx 26.8\)–\(35.1\%\), and Sequential AdaFilter-Bon had FWER \(\approx 0.03\)–\(0.04\) with recall \(\approx 27.2\)–\(37.7\%\) [1610.03330].

Although the real applications in the original paper primarily emphasized AdaFilter-BH rather than AdaFilter-Bon, they were presented as corroborating the core AdaFilter advantage across microarray, single-cell RNA sequencing, and metabolomics GWAS settings [1610.03330].

## 6. Conservative filtering, \(k\)-FWER extensions, and terminological ambiguities

A later development analyzed AdaFilter-Bon through the lens of \(k\)-family-wise error rate. In that formulation, for replication level \(u\) across \(n\) studies and \(m\) features, the generalized threshold is
\[
\tilde{t} = \sup\{ t \in [0,k\alpha] : t \cdot \sum_{i=1}^m I\{F_i < t\} \le k\alpha \},
\]
and the decision rule rejects \(H_i^{u/n}\) if \(S_i < \tilde{t}\) [2508.15363].

That work also formalized a source of conservativeness induced by filtering. If
\[
\pi_0(\tilde{t}) \equiv \frac{\sum_{i=1}^m I\{F_i < \tilde{t}\}\cdot I\{H_i^{u/n}\text{ true}\}}{\sum_{i=1}^m I\{F_i < \tilde{t}\}},
\]
then AdaFilter-Bon satisfies the finite-sample upper bound
\[
k\text{-FWER}(R) \le \alpha\cdot E[\pi_0(\tilde{t})] \in [0,\alpha].
\]
This shows that effective filtering can make the realized error level substantially smaller than \(\alpha\), with an accompanying power loss [2508.15363].

To mitigate this effect, AdaFilter-AdaBon introduces the post-filter null proportion estimator
\[
\hat{\pi}_0(t) \equiv \frac{\sum_{i=1}^m I\{F_i < t\}\cdot I\{S_i \ge \theta t\}}{(1-\theta t)\cdot \sum_{i=1}^m I\{F_i < t\}},
\]
and replaces the rejection threshold by
\[
\hat{t}_\theta \equiv \sup\{ t \in [0,1] : \hat{\pi}_0(t)\cdot t\cdot \sum_{i=1}^m I\{F_i < t\} \le k\alpha \}.
\]
Under weak dependence and the stated limit assumptions, the paper proves asymptotic \(k\)-FWER control and reports higher power than the original AdaFilter-Bon in simulations, with \(\theta=0.5\) described as a good default in experiments [2508.15363].

A separate source of ambiguity is terminological rather than statistical. In recent language-model work, a similarly spelled “AdaFilter-BoN” refers to adaptive filtering around Best-of-\(N\) inference or to consensus filtering for Process Reward Model training, not to partial conjunction multiple testing. “ROC-n-reroll: How verifier imperfection affects test-time scaling” uses “AdaFilter-BoN” for adaptive filtering Best-of-\(N\) under an imperfect verifier, with ROC-aware selection of \(N\) and thresholds \(\tau\) and switching or combining BoN with rejection sampling [2507.12399]. “The Lessons of Developing Process Reward Models in Mathematical Reasoning” describes a consensus filtering mechanism that integrates MC estimation with LLM-as-a-judge and states that this is exactly the goal of an “AdaFilter-BoN”-style approach [2501.07301]. These usages share the idea of adaptive filtering, but they are distinct from AdaFilter-Bon as a Bonferroni-based procedure for partial conjunction testing.

In its original and most specific sense, AdaFilter-Bon denotes a replicability-testing method that combines order-statistic partial conjunction \(p\)-values with adaptive filtering to control FWER or PFER while substantially reducing the conservativeness of direct Bonferroni correction. Its later refinements chiefly address the residual conservativeness introduced by the filtering step itself [1610.03330].

Source: https://www.emergentmind.com/topics/adafilter-bon