Selection-Adjusted Empirical Bayes Confidence

• The paper introduces selection-adjusted methodologies that correct for double-use bias in empirical Bayes LFDR estimation using MDL, L1O, and L½O techniques.
• It demonstrates through simulations and real protein abundance studies that estimator choice critically impacts bias control and inference accuracy.
• It recommends an optimally weighted hybrid (MDL-BBE) estimator to balance bias reduction and robustness under varying sparsity and signal intensity conditions.

Selection-adjusted empirical Bayes confidence procedures comprise a set of methodologies for constructing confidence intervals or regions in settings where the parameter(s) of interest are selected from many candidates based on observed data, and where information is borrowed across units via empirical Bayes ideas. These procedures explicitly adjust for the bias and over-optimism that typically arise from such selection, especially in high-dimensional or multipopulation studies, aiming to ensure that uncertainty quantification remains valid after the selection step. Methods in this area span from corrected maximum likelihood estimation of local false discovery rates (LFDR) for small and medium-scale data, to interval construction that balances selective coverage, efficiency, and width, often using theoretical frameworks such as the minimum description length (MDL) principle.

1. Fundamental Corrections for LFDR Estimation

A core challenge addressed by selection-adjusted empirical Bayes confidence procedures is the "double-use" bias inherent in conventional empirical Bayes maximum likelihood estimators (MLEs) of the LFDR, particularly when the number of features (e.g., genes, proteins) is small. The unbiased local false discovery rate at feature i, denoted $\psi_i$, is the posterior probability that the null hypothesis is true given the observed data. In the traditional "leave-zero-out" (L0O) MLE approach, all features, including the ith, are used both to estimate the prior parameters (such as $\pi_0$ and $\theta_{\text{alt}}$) and to compute the likelihood at $t_i$:

$$\langle\hat{\theta}^{\text{L0O}}, \hat{\pi}_0^{\text{L0O}}\rangle = \underset{\theta, \pi_0}{\arg\max}\prod_{j=1}^N [\pi_0\,g_{\theta_0}(t_j) + (1-\pi_0)\,g_\theta(t_j)]$$

$$\hat{\psi}_i^{\text{L0O}} = \frac{\hat{\pi}_0^{\text{L0O}}\,g_{\theta_0}(t_i)}{\hat{\pi}_0^{\text{L0O}}\,g_{\theta_0}(t_i) + (1-\hat{\pi}_0^{\text{L0O}})\,g_{\hat{\theta}^{\text{L0O}}}(t_i)}$$

This approach creates a substantial negative bias, especially in small $N$ settings, due to the use of $t_i$ both in parameter estimation and in inference for $t_i$ itself.

To address this, the paper introduces several adjusted estimators:

• MDL (Minimum Description Length) Estimator: Excludes $t_i$ entirely from parameter estimation:

$$\langle\hat{\theta}_i^{\text{MDL}}, \hat{\pi}_{0i}^{\text{MDL}}\rangle = \underset{\theta, \pi_0}{\arg\max}\prod_{j\neq i} [\pi_0\,g_{\theta_0}(t_j) + (1-\pi_0)\,g_\theta(t_j)]$$

then computes

$$\hat{\psi}_i^{\text{MDL}} = \frac{\hat{\pi}_{0i}^{\text{MDL}}\,g_{\theta_0}(t_i)}{\hat{\pi}_{0i}^{\text{MDL}}\,g_{\theta_0}(t_i) + (1-\hat{\pi}_{0i}^{\text{MDL}})\,g_{\hat{\theta}_i^{\text{MDL}}}(t_i)}$$

• Leave-one-out (L1O): Removes $t_i$ only for estimating alternative hypothesis parameters.
• Leave-half-out (L½O): Downweights $t_i$'s contribution in estimation by a factor $\nu=1/2$, allowing partial information retention.

This correction ensures that, for each feature, the LFDR estimate is less biased with respect to selection, i.e., is "selection-adjusted".

2. Bias Reduction and Small-Scale Applicability

By omitting $t_i$ or reducing its influence in parameter estimation, the MDL and related corrected MLE approaches substantially reduce the double-use bias. This improvement is most critical in small and medium-scale studies (e.g., conventional gene expression, proteomics, metabolomics), where conventional empirical Bayes procedures can be badly biased. Simulations in the paper demonstrate:

• Corrected MLEs (especially MDL and L1O) provide low bias when the fraction of affected features is moderate to large.
• When only a solitary feature is affected, completely omitting $t_i$ (as in MDL or L1O) can result in high positive bias, which is mitigated by L½O.
• All corrected MLEs suffer strong negative bias when over 90% of features are unaffected, underscoring the necessity of bias correction in sparse signal settings and the challenge of unknown sparsity in practice.

3. Empirical Comparison in Protein Abundance Studies

The bias reduction properties are directly observed in real data. Analysis of a 20-protein abundance dataset from breast cancer studies reveals that the (unadjusted) L0O, MDL, and L1O yield very different decisions about which proteins are "affected." Notably:

• Uncorrected methods, due to double-use, can indicate spuriously low LFDRs for extreme values (leading to possible false discovery).
• Conservative estimators (such as the binomial-based estimator or its r-value approximation) have positive bias, particularly in nearly-null settings.
• As shown in volcano and LFDR-vs-p-value plots, the inference about which proteins are differentially abundant is highly sensitive to the choice of estimator.

Thus, the estimator used directly impacts downstream scientific decision making, reinforcing the importance of well-calibrated, selection-adjusted estimators.

4. Simulation Outcomes and the MDL-BBE Combination

Through extensive simulations varying feature count, effect detectability, and number of non-null features, the following findings emerge:

• All corrected MLEs (including MDL) show notable negative bias when all features are null; conservative estimators are less biased in this case.
• When several features are affected, conservative estimators (BBE, RV) are positively biased, but MDL and L1O yield low bias.
• No single estimator uniformly dominates all scenarios.

To address these trade-offs, the paper introduces an optimally weighted estimator (MDL-BBE) that linearly combines the MDL estimator (low bias in dense settings) with the binomial-based estimator (robust in extremely sparse settings). This hedging achieves lower worst-case bias across possible realities regarding the proportion of affected features.

5. Practical Recommendations for Real-World Data Analysis

Because the true proportion of affected features is unknown in real data, and since no method is uniformly best across all scenarios, it is recommended to use an optimally weighted combination of the best-corrected MLE (typically MDL) and a conservative estimator (such as BBE or RV). The optimal linear weights are chosen based on minimax considerations described in the text (Section 3.2.2).

This hybrid approach ensures protection against the risks of negative bias (false discoveries) and excessive conservativeness (failure to discover real effects) under uncertainty about actual sparsity/density of signals.

6. Minimum Description Length Principle and Theoretical Foundations

The theoretical motivation for the best-performing correction (MDL estimator) relies on the minimum description length principle. By minimizing the codelength $-\log$ probability of observed data (excluding $t_i$ for feature $i$), the MDL estimator provides a natural information-theoretic adjustment for selection effects. This framework corresponds to using out-of-sample log-likelihood for parameter estimation, thereby avoiding overfitting to selected observations.

Appendices in the paper further connect the MDL estimator to minimax coding theory and establish its asymptotic and universality properties, supporting its use as a rigorously justified selection-adjusted empirical Bayes estimator.

---

In summary, selection-adjusted empirical Bayes confidence procedures grounded in MDL-corrected MLEs and optimal estimator combination provide low-bias, theoretically motivated, and practically robust solutions for inference in small-to-moderate scale studies where selection-induced bias is non-negligible. The key innovation is the principled avoidance of double-counting information, direct control of bias under varying sparsity, and rigorous justification via coding-theoretic arguments, culminating in improved reliability of interval estimation and hypothesis testing in real-world biological applications (Padilla et al., 2010).