Simultaneous false discovery rate control in location families

Abstract: When testing a number of statistical hypotheses using data from location families, it is often useful to control the false discovery rate (FDR) not just for hypotheses of the null values but also of other parameter values that are deemed practically insignificant. Here we consider FDR as a curve indexed by the location parameter and suggest a simple generalization of the Benjamini-Hochberg procedure that controls the FDR curve below any user-specified level. As a corollary of our main result, we show that the standard Benjamini-Hochberg procedure -- designed to control the FDR at the null -- also provides simultaneous control of the whole FDR curve for free. We further demonstrate the implications of our results and some practical considerations with a numerical example.

• The paper introduces a generalized BH procedure for simultaneous FDR curve control in location families, ensuring error rates are managed across a continuum of parameter values.
• It leverages the monotone ratio property to derive tight theoretical guarantees and lower bounds, validated in both Gaussian and heterogeneous models.
• Empirical illustrations, including a breast cancer gene expression study, highlight practical trade-offs between power and stringent FDR control in real-world applications.

Simultaneous FDR Control in Location Families: A Technical Analysis

Motivation and Problem Statement

False discovery rate (FDR) control is central in multiple hypothesis testing, particularly in large-scale analyses across scientific domains. Traditionally, FDR control as formulated by Benjamini-Hochberg (BH) targets the proportion of incorrectly rejected null hypotheses. However, standard FDR control only considers the classical null value, ignoring practical insignificance arising from small nonzero parameter values. Many scientific applications require simultaneous control of FDR not just at the null, but also at a spectrum of parameter values that may be deemed negligible or irrelevant for inference.

This paper presents a rigorous framework for simultaneous FDR curve control in location families—collections of distributions parametrized by a real-valued location parameter. The authors offer a generalization of the BH procedure to control the FDR as a function indexed by the location parameter, propose operational methods for such control, and analyze both theoretical guarantees and practical implications for large-scale testing.

Theoretical Foundations and Methodology

FDR Curve Definition

Let $X_i \sim P_{\theta}$ be independent real-valued observations, $P_{\theta}$ belonging to the location family with cumulative distribution $F(x - \theta)$, and $H_{i, \theta}: \theta_i \geq \theta$ be the one-sided null hypothesis. The paper defines the FDR curve for a rejection set $S \subset \{1,\ldots,m\}$ as

$$\text{FDP}_S(\theta) = \frac{\sum_{i \in S} \mathbb{I}\{\theta_i \geq \theta\}}{\max\{1, |S|\}},$$

where $$\text{FDR}(\theta) = \mathbb{E}[\text{FDP}_S(\theta)]$$ formalizes type I error control at every value $\theta$.

Generalized BH Procedure

The core technical idea extends the BHq procedure to a target FDR curve $q(\theta): \mathbb{R} \to [0, 1]$ (not necessarily constant). The generalized method uses "FDR curve-normalized p-values":

$$P_i = \sup_{\theta} \frac{P_{i,\theta}}{q(\theta)},$$

then applies the original BH procedure at nominal level $P_{\theta}$0 to $P_{\theta}$1.

A key theoretical result is the free lunch theorem, which asserts that the standard BH procedure provides simultaneous control of the entire FDR curve at a transformed level $P_{\theta}$2 determined by the monotone ratio property of $P_{\theta}$3—a property satisfied in Gaussian and other location families. The transformation is defined as

$P_{\theta}$4

where $P_{\theta}$5, $P_{\theta}$6 are quantile-based bounds.

Strong Numerical Guarantees

For location families satisfying the monotone ratio property, the authors prove:

• The generalized BH procedure controls the FDR curve everywhere: $P_{\theta}$7 for all $P_{\theta}$8.
• The standard BH procedure is a special case that also provides simultaneous FDR control for free.
• Lower bounds for worst-case FDR curve indicate that the analysis is tight: for null parameters, $P_{\theta}$9, which is close to $F(x - \theta)$0 for small values.
• For Gaussian location models, $F(x - \theta)$1 touches $F(x - \theta)$2 at the null and is strictly lower elsewhere, yielding stronger control at practically insignificant values.

Extensions and Empirical Illustration

Generalization Beyond Single Location Families

The methodology is extended to heterogeneous settings where each hypothesis has a potentially distinct location family $F(x - \theta)$3. The definition of $F(x - \theta)$4 adapts correspondingly, adding a supremum over all $F(x - \theta)$5. The theoretical guarantees are maintained, ensuring robust simultaneous FDR control across varying models.

Breast Cancer Gene Expression Case Study

Using the Storey-Tibshirani breast cancer gene expression dataset (3,170 genes with BRCA1/BRCA2 mutations), the paper demonstrates:

• Effect size hypothesis testing with gene-specific variances (different $F(x - \theta)$6), adopting piecewise constant FDR constraints.
• Signal-to-noise ratio testing with shared Gaussian $F(x - \theta)$7 and tailored FDR thresholds.

Empirical results show that using minimal constraints as the target FDR curve $F(x - \theta)$8 produces stricter $F(x - \theta)$9 and fewer rejections, indicating trade-off between power and simultaneous FDR control. For signal-to-noise settings, the $H_{i, \theta}: \theta_i \geq \theta$0 curve coincides with $H_{i, \theta}: \theta_i \geq \theta$1 at constraint points, with highly stable rejection sets ($H_{i, \theta}: \theta_i \geq \theta$2350 rejections across settings).

Practical and Theoretical Implications

Practical Impact

The simultaneous control framework enables practitioners to:

• Post hoc select rejection thresholds and report FDR control at any location value, accommodating diverse scientific requirements.
• Avoid model-specific tuning of FDR control, as the theory guarantees robustness across location parameter values and model variations.

The methodology is especially relevant in genomics, neuroimaging, and other fields with high-throughput data, where practical insignificance can overshadow strict null hypothesis testing.

Theoretical Implications

• The results reveal fundamental properties of the BH procedure, including its intrinsic power for simultaneous error control without additional computational or statistical burden.
• The monotone ratio property emerges as a critical condition for tight simultaneous FDR guarantees, guiding choice of models and error benchmarks.
• The framework suggests directions for selective compound decision problems, encouraging hybrid criteria and broader risk control beyond classical multiple testing.

Future Directions

• Integration with empirical Bayes approaches and deconvolution methods (Efron 2014) could enable data-driven specification of the FDR curve, tailored to scientific priorities and prior distributions.
• Extensions to dependent data settings, adaptive procedures, and other compound selective error metrics remain open, promising further generality and applicability.

Conclusion

This paper establishes a principled approach for simultaneous FDR control in location families, generalizing the BH procedure for curve-indexed error rates. The theoretical framework, backed by both strong analytical results and empirical demonstration, furnishes practitioners and theorists with tools for nuanced FDR control over a continuum of practically relevant parameter values. The implications span high-throughput scientific applications and drive future research toward broader selective inference and compound risk management in statistical decision theory.

Reference: "Simultaneous false discovery rate control in location families" (2605.09525).