---
title: 'AdaFilter-AdaBon: Adaptive Multiple Testing'
url: https://www.emergentmind.com/topics/adafilter-adabon
type: topic
---

# AdaFilter-AdaBon: Adaptive Multiple Testing

Searching arXiv for the cited papers and closely related work to ground the article.
AdaFilter-AdaBon is an adaptive multiple-testing procedure for detecting *replicated* signals across several studies while controlling a generalized family-wise error rate, the \(k\)-FWER. It builds directly on the AdaFilter-Bon method of Wang et al. (2022), and improves its power by estimating the proportion of true nulls *after* filtering [2508.15363]. In the literature, the name “AdaFilter” is also used for an unrelated adaptive fine-tuning method in deep transfer learning; that 2019 convolutional-filter method is distinct from the statistical AdaFilter line introduced for partial conjunction testing and should not be conflated with AdaFilter-AdaBon [1911.09659].

## 1. Historical lineage and nomenclature

AdaFilter-AdaBon belongs to a line of methods for *partial conjunction (PC) hypotheses*, where the scientific objective is to identify signals that replicate across multiple studies rather than signals that are merely significant in an aggregate meta-analysis. The earlier AdaFilter framework introduced adaptive filtering procedures for PC hypotheses and developed two principal variants: AdaFilter Bonferroni, often referred to informally as “AdaBon,” for FWER/PFER control, and AdaFilter BH for FDR control [1610.03330]. AdaFilter-AdaBon is a later adaptive refinement of the Bonferroni branch: it retains the same filtering logic as AdaFilter-Bon, but modifies the threshold by incorporating a post-filter estimate of the null proportion [2508.15363].

The statistical use of “AdaFilter” is unrelated to the computer-vision method “AdaFilter: Adaptive Filter Fine-tuning for Deep Transfer Learning,” which is an adaptive **filter-level** fine-tuning framework for deep transfer learning and operates on duplicated convolutional filters and recurrent gating in ResNet-50-style architectures [1911.09659]. That naming overlap is purely terminological. In the multiple-testing literature, “AdaBon” denotes the Bonferroni-style adaptive filtering procedure derived from the original AdaFilter framework [1610.03330], whereas “AdaFilter-AdaBon” denotes the later procedure that estimates the post-filter null proportion to mitigate conservativeness [2508.15363].

A common misconception is therefore to read “AdaFilter-AdaBon” as a hybrid of deep-learning fine-tuning and statistical error control. The record in the cited papers supports the opposite conclusion: AdaFilter-AdaBon is a multiple-testing procedure for replicability analysis, while the 2019 CNN AdaFilter is a transfer-learning algorithm with no direct methodological connection [2508.15363].

## 2. Statistical setting: partial conjunction, replicability, and \(k\)-FWER

The method considers a meta-analysis of \(n \ge 2\) comparable studies, each measuring the same \(m\) features. For feature \(i\) in study \(j\), there is a null hypothesis \(H_{ij}\) and a corresponding valid \(p\)-value \(P_{ij}\), meaning
\[
\Pr(P_{ij} \le t) \le t \quad \text{for all } t \in [0,1] \text{ when } H_{ij} \text{ is true}.
\]
Across studies, the vectors \((P_{i1})_{i=1}^m,\dots,(P_{in})_{i=1}^m\) are assumed independent, while within each study there can be dependence across features [2508.15363].

Replicability is encoded by a number \(u \in \{2,\dots,n\}\). A feature is “replicated at level \(u/n\)” if at least \(u\) of its study-specific nulls are false. The corresponding PC null is
\[
H^{u/n}_i:\quad \text{fewer than \(u\) of } H_{i1},\dots,H_{in} \text{ are false},
\]
with the alternative that at least \(u\) of the \(n\) component nulls are false [2508.15363]. Rejecting \(H^{u/n}_i\) therefore means declaring feature \(i\) replicated in at least \(u\) studies.

A key structural property is *nesting*:
\[
u'' \ge u' \;\Rightarrow\; \text{if } H_i^{u'/n} \text{ is true then } H_i^{u''/n} \text{ is also true}.
\]
This nesting is central to the filtering logic, because failure to replicate at level \(u-1\) precludes replication at level \(u\) [2508.15363].

For testing, the method uses the Bonferroni PC \(p\)-value. If \(P_{i(1)}\le \dots \le P_{i(n)}\) are the order statistics of the study-specific \(p\)-values, then
\[
P^{u/n}_i = f_{\text{Bon}}(P_{i1},\dots,P_{in};u) = (n-u+1)\,P_{i(u)}.
\]
This is the choice used inside AdaFilter-Bon and AdaFilter-AdaBon [2508.15363]. The focus is not the ordinary FWER alone, but the generalized \(k\)-family-wise error rate,
\[
k\text{-FWER}(\mathcal R)
=
\Pr\left(\sum_{i=1}^m I\{i\in\mathcal R\} I\{H_i^{u/n}\text{ true}\} \ge k\right),
\]
which allows up to \(k-1\) false rejections. When \(k=1\), this coincides with FWER [2508.15363].

The motivation is the classical multiplicity burden of high-dimensional replicability analysis. Even when one has valid PC \(p\)-values, testing \(m\) hypotheses with FWER or \(k\)-FWER control leads to very stringent per-hypothesis thresholds. This motivates procedures that reduce the multiplicity burden without sacrificing error control, and filtering is the specific device used here [2508.15363].

## 3. Construction of AdaFilter-AdaBon

AdaFilter-AdaBon keeps the filtering structure of AdaFilter-Bon but adapts the rejection threshold using an estimate of the post-filter null proportion [2508.15363]. For a fixed replicability level \(u\), it defines two quantities for each feature \(i\):
\[
S_i \equiv (n-u+1)\,P_{i(u)},
\]
the testing PC \(p\)-value, and
\[
F_i \equiv (n-u+1)\,P_{i(u-1)},
\]
the filtering \(p\)-value, obtained from the PC null at level \(u-1\) [2508.15363]. Because \(P_{i(u-1)} \le P_{i(u)}\), one has \(F_i \le S_i\).

In AdaFilter-Bon, the rejection threshold is
\[
\hat{t} \equiv \sup\left\{t \in [0, k\alpha] :
t \,\sum_{i=1}^m I\{F_i < t\} \le k \alpha \right\},
\]
so the multiplicity factor is the number retained by the filter rather than the full \(m\) [2508.15363]. The later AdaFilter-AdaBon procedure modifies this by estimating the proportion of true nulls among the retained features. Fix a tuning parameter \(\theta\in(0,1)\). The estimator is
\[
\hat{\pi}_0(t;\theta)
=
\frac{\sum_{i=1}^m I\{F_i < t\}\,I\{S_i \ge \theta t\}}
{(1-\theta t)\sum_{i=1}^m I\{F_i < t\}}.
\]
Its numerator counts retained features whose PC \(p\)-value is at least \(\theta t\), and its denominator normalizes by the retained set size and the factor \(1-\theta t\) [2508.15363].

The conceptual AdaFilter-AdaBon threshold is
\[
\hat{t}_{\theta} \equiv \sup\left\{
t\in[0,1] :
\hat{\pi}_0(t)\, t \sum_{i=1}^m I\{F_i<t\} \le k\alpha
\right\}.
\]
Plugging in the estimator yields the operational definition:
\[
\hat{t}_{\theta} \equiv \sup \left\{ t \in [0,1] :
t \cdot \frac{\sum_{i=1}^m I \{ F_i < t \}\, I\{ S_i \geq \theta t \}}
{1 - \theta t} \le k \alpha
\right\}.
\]
The rejection rule is then simply: reject \(H_i^{u/n}\) if \(S_i < \hat t_\theta\) [2508.15363].

The paper also gives an implementable finite grid
\[
\mathcal G = \left\{G\in\{0,1\}\cup\{F_i, S_i, S_i/\theta\}_{i=1}^m : G\le 1\right\},
\]
and defines
\[
\breve{t}_\theta \equiv \max\left\{
t\in\mathcal G :
t \cdot \frac{\sum_{i=1}^m I\{F_i<t\}\,I\{S_i\ge\theta t\}}
{1-\theta t}
\le k\alpha
\right\}.
\]
A key theorem states that the rejection sets coincide:
\[
\{i: S_i < \hat t_\theta\} = \{i: S_i < \breve t_\theta\}.
\]
The paper states that one can compute \(\breve t_\theta\) in \(O(m)\) time by evaluating the left-hand side at each candidate in \(\mathcal G\) [2508.15363].

This construction preserves the original AdaFilter idea—screen with \(F_i\), test with \(S_i\)—but replaces the fixed post-filter multiplicity correction of AdaFilter-Bon with a multiplicity correction scaled by an estimated post-filter null proportion. This suggests a direct mechanism for reducing conservativeness when filtering preferentially retains alternatives.

## 4. Theoretical guarantees and regularity conditions

The main theoretical result is asymptotic \(k\)-FWER control under weak dependence assumptions [2508.15363]. The paper defines \(m_0\) as the number of true PC nulls and \(m_1\) as the number of alternatives, and assumes almost-sure convergence of the empirical distributions of \(S_i\) and \(F_i\) under both the null and the alternative. It also assumes
\[
\lim_{m\to\infty} \frac{m_0}{m} = \pi_0 \in (0,1).
\]
These are presented as typical empirical-process convergence assumptions encompassing many weak-dependence structures, including finite blocks and mixing processes [2508.15363].

A central inequality is
\[
0 < \tilde S_0(t') \le t'\, \tilde F_0(t'')
\quad \text{for any } 0<t'\le t''\le 1,
\]
which reflects a conditional validity lemma: under the PC null and independence of the study \(p\)-values,
\[
\Pr(S_i < t' \mid F_i < t'') \le t'.
\]
This is the asymptotic analogue of the conditional validity phenomenon that already underpinned the original AdaFilter theory [1610.03330].

Let
\[
\mathcal R = \{i: S_i \le \hat t_\theta\}
\]
be the rejection set and
\[
V_m = \sum_{i=1}^m I\{S_i < \hat t_\theta\} I\{H_i^{u/n} \text{ true}\}
\]
the number of false discoveries. The main theorem states that if Assumption 1 holds, if \(k = k(m) = \omega m\) for some \(\omega\in(0,1)\), and if there exists \(t\in(0,1]\) such that
\[
\lim_{m\to\infty}
t\cdot \frac{
\sum_{i=1}^m I\{F_i < t\} I\{S_i \ge \theta t\}
}{
m(1-\theta t)
}
< \omega \alpha
\]
with probability \(1\), then
\[
\limsup_{m\to\infty} k\text{-FWER}(\mathcal R) \le \alpha
\]
[2508.15363].

The paper’s interpretation is that the estimator \(\hat\pi_0(t)\) is constructed so that, under the null, the proportion of retained features with \(S_i\ge\theta t\) is at least \((1-\theta t)\) in expectation. Intuitively, this makes \(\hat\pi_0(t)\) a conservative estimate of \(\pi_0(t)\) in large samples, so using it in the threshold tightens the constraint when necessary rather than weakening it unsafely [2508.15363].

The guarantees are explicitly asymptotic. The paper states that no explicit finite-sample guarantees are provided, although simulations show good empirical control across a range of settings and correlations [2508.15363]. This is an important qualification: AdaFilter-AdaBon is theoretically rigorous, but its formal guarantee is not a finite-sample exact \(k\)-FWER theorem in the style of the original AdaFilter Bonferroni result under full independence [1610.03330].

## 5. Power improvement, simulations, and computational profile

The motivation for AdaFilter-AdaBon is the conservativeness of AdaFilter-Bon. The paper gives the bound
\[
k\text{-FWER}(\mathcal R) \le \alpha \,\mathbb E[\pi_0(\hat t)] \in [0,\alpha]
\]
for AdaFilter-Bon under independent and valid \(P_{ij}\) [2508.15363]. Because filtering preferentially retains hypotheses with signal, the post-filter null proportion \(\pi_0(\hat t)\) can be much smaller than \(1\), so AdaFilter-Bon may control at a level far below the nominal \(\alpha\). AdaFilter-AdaBon is designed to mitigate exactly this conservativeness.

The paper’s heuristic comparison is explicit. Under AdaFilter-Bon, the effective threshold is roughly
\[
t_{\text{Bon,eff}} \approx \frac{k\alpha}{m_{\text{ret}}},
\]
where \(m_{\text{ret}}\) is the number retained by the filter. Under AdaFilter-AdaBon, the effective threshold is roughly
\[
t_{\text{AdaBon,eff}} \approx \frac{k\alpha}{\hat\pi_0 m_{\text{ret}}}.
\]
If \(\hat\pi_0\) correctly estimates the post-filter null proportion, the expected number of false rejections becomes approximately \(k\alpha\), and the threshold is larger by approximately a factor of \(1/\pi_0\) [2508.15363]. This suggests that the method is specifically advantageous when filtering removes a substantial fraction of true nulls.

The reported simulations use \(m=500\) features and \(n=4\) studies, with blockwise equicorrelated Gaussian noise within each study, \(\rho\in\{-0.8,-0.2,0.2,0.8\}\), signal density \(\pi_1\in\{0.025,0.05,0.075,0.10,0.125,0.15\}\), replicability levels \(u\in\{2,3,4\}\), and target FWER (\(k=1\)) at \(\alpha=0.05\) [2508.15363]. Compared methods include AdaFilter-AdaBon (\(\theta=0.5\)), AdaFilter-Bon, standard Bonferroni and Hochberg applied to Fisher PC \(p\)-values, and adaptive Bonferroni and adaptive Hochberg applied to Fisher PC \(p\)-values.

The stated findings are that AdaFilter-AdaBon maintains FWER below \(0.05\) across all settings; its FWER is systematically higher than AdaFilter-Bon but still comfortably below target; and it yields notably higher TPR than AdaFilter-Bon in most settings, especially for \(u=2\) and moderate-to-high signal density \(\pi_1\ge 0.10\) [2508.15363]. For very stringent replication (\(u=4\)) and extremely sparse signals (\(\pi_1=0.025\)), AdaFilter-Bon matches AdaFilter-AdaBon’s power. Additional simulations for \(k=5\) and \(k=10\) show that all methods are very conservative, while AdaFilter-AdaBon is the least conservative among them and has substantially higher power, especially when \(u=4\) [2508.15363].

On implementation, the paper states that sorting the study \(p\)-values per feature is \(O(n\log n)\) per feature, but because \(n\) is small, the overall complexity is \(O(m)\) in practice; constructing \(\mathcal G\) is \(O(m)\); and evaluating the adaptive statistic can be made linear by cumulative counts [2508.15363]. The method therefore scales linearly in \(m\) up to large numbers of features.

## 6. Relation to AdaFilter, AdaBon, and adjacent procedures

The original AdaFilter formulation introduced the filtering and selection statistics
\[
F_j = (n-r+1)P_{(r-1)j},
\qquad
S_j = (n-r+1)P_{(r)j},
\]
and used them to define the AdaFilter Bonferroni threshold
\[
\gamma_0^{\text{Bon}} =
\sup\Big\{\gamma \in [0,\alpha] \ \Big| \
\gamma \sum_{j=1}^M 1_{\{F_j < \gamma\}} \le \alpha \Big\}
\]
for FWER/PFER control [1610.03330]. In that sense, AdaFilter-AdaBon is not a new filtering architecture but a modification of the calibration step: the term \(\sum 1_{\{F_j<t\}}\) remains, but it is multiplied by an adaptive estimate of the post-filter null proportion [2508.15363].

This relation is best understood in three layers. First, *AdaFilter* is the general framework for adaptive filtering in PC testing [1610.03330]. Second, *AdaBon* or *AdaFilter Bonferroni* is the Bonferroni-style instantiation of that framework [1610.03330]. Third, *AdaFilter-AdaBon* is the later procedure that augments AdaFilter-Bon with post-filter null-proportion estimation to improve power while preserving asymptotic \(k\)-FWER control [2508.15363].

The paper also positions AdaFilter-AdaBon relative to standard Bonferroni, Holm, and Hochberg procedures for PC hypotheses. Those methods treat the PC \(p\)-values like ordinary single-study \(p\)-values and penalize all \(m\) hypotheses equally; they do not use filtering and do not use null-proportion estimation; and they are therefore described as severely conservative in high-dimensional replicability analysis [2508.15363]. In the broader replicability literature, two-stage selection-and-testing procedures and FDR-based partial conjunction methods address related questions, but they do not estimate post-filter null proportions in the same way [2508.15363].

The paper further notes that AdaFilter-AdaBon is defined using Bonferroni PC \(p\)-values. If one prefers more powerful combining functions such as Fisher, one cannot directly plug them into this specific algorithm; adapting the method to other combining functions would require new theory [2508.15363]. This is a substantive methodological boundary rather than a mere implementation detail.

Finally, the 2025 paper outlines extensions beyond \(k\)-FWER. It states that AdaFilter-AdaBon can be augmented to control the false exceedance rate (FDX) and the false discovery rate (FDR) asymptotically through a second threshold \(\hat\tau\) and a Genovese–Wasserman-style bound [2508.15363]. A plausible implication is that the adaptive-filtering-plus-post-filter-estimation principle is broader than the specific \(k\)-FWER instantiation, although the developed theory in the paper is centered on replicability analysis with Bonferroni PC \(p\)-values.

## 7. Interpretation, scope, and limitations

A rejection by AdaFilter-AdaBon means that the corresponding feature is declared replicated in at least \(u\) studies, with the global error metric controlled at the specified \(k\)-FWER level in the asymptotic sense established by the paper [2508.15363]. This is a stronger statement than ordinary meta-analytic significance, because the null explicitly concerns the number of studies in which the effect is non-null.

The method is expected to be most beneficial when there are many features, when a nontrivial fraction of features are truly replicated at level \(u\), when the replication requirement \(u\) is modest, and when dependence across features is moderate rather than pathological [2508.15363]. This follows the paper’s discussion that filtering is especially informative when \(u-1\) replication serves as a meaningful screen for \(u\)-level replication.

Several limitations are explicit. The asymptotic theory allows certain forms of weak dependence across features, but very strong or complex dependence structures may violate the assumptions [2508.15363]. The guarantees are asymptotic as \(m\to\infty\), so for small \(m\) the estimator \(\hat\pi_0(t)\) may be noisy and the adaptive threshold may have less predictable behavior [2508.15363]. The formal theory is developed for Bonferroni PC \(p\)-values, and the asymptotic \(k\)-FWER theorem assumes that \(k\) grows linearly with \(m\), even though simulations examine fixed small \(k\) such as \(1\), \(5\), and \(10\) [2508.15363].

These limitations do not negate the method’s contribution; they define its scope. In the statistical literature, AdaFilter-AdaBon is best understood as an adaptive refinement of AdaFilter-Bon for replicated-signal detection under partial conjunction testing, with asymptotic \(k\)-FWER control and empirically higher power than the original AdaFilter-Bon [2508.15363]. In contrast, the identically named but unrelated deep-learning AdaFilter remains a transfer-learning procedure on convolutional networks and has no role in the statistical construction of AdaFilter-AdaBon [1911.09659].

Source: https://www.emergentmind.com/topics/adafilter-adabon