Coupled Rejection Metrics Overview

• Coupled rejection metrics are defined as linked statistical criteria that integrate multiple decision thresholds for enhanced discrimination between true signals and noise.
• They are applied in hypothesis testing, p-value pooling, selective classification, unsupervised anomaly detection, and adversarial detection to achieve improved error control and system-level guarantees.
• These metrics leverage explicit coupling to align theoretical calibration with practical performance, enabling adaptive trade-offs between risk and coverage.

A coupled rejection metric is a statistical, probabilistic, or algorithmic criterion in which two or more rejection thresholds, scores, or decision statistics are explicitly linked—either algebraically, via joint functional constraints, or through theoretical calibration—to produce a system-level property that cannot be achieved by applying standard individual rejection rules in parallel. Coupling may enable improved discrimination between classes of interest (such as signal and noise, or correct and adversarial), establish theoretical guarantees (e.g., on cost or coverage), or provide nuanced trade-offs not attainable by uncoupled methods. Coupled rejection metrics find application in hypothesis testing, anomaly and adversarial detection, LLM reliability, composite estimator design, and more, as documented across several formal frameworks in contemporary research.

1. Characterizations and Formal Definitions

A coupled rejection metric typically arises in one of the following settings, each characterized by explicit or implicit synchronicity between rejection statistics:

• Hypothesis Testing: Coupled pre- and post-experimental rejection odds quantify the probability of true and false rejections, incorporating power, type I error, and priors into unified odds or ratios. For example, the "pre-experimental rejection ratio" $R_{\text{pre}} = (1-\beta)/\alpha$ and the "post-experimental rejection ratio" $R_{\text{post}}(x)$ are coupled via their respective conditioning on experimental design and observed data (Bayarri et al., 2015).
• Pooling $p$-values: Central and marginal rejection levels $R_c(\alpha)$, $R_m(\alpha)$ encapsulate rejection regions when all versus one $p$-values are set to the same value, with their quotient $Q_g(\alpha)$ quantifying the degree of coupling (centrality) in a multivariate test statistic (Salahub et al., 2023).
• Learning with Selective Rejection: In classification with a reject option, selective accuracy, risk, and joint acceptance/failure rates can be evaluated simultaneously over threshold sweeps, as in the Area under the Generalized Risk Coverage Curve (AUGRC), which couples classifier accuracy to confidence ranking fidelity over all possible working points (Traub et al., 2024).
• Unsupervised Detection: In the "RejEx" approach, the ExCeeD stability metric produces a global, detector-independent rejection threshold—explicitly coupling the rejection rule to the theoretical stability of bootstrapped decision boundaries—resulting in consistent cost and rejection guarantees across models (Perini et al., 2023).
• Adversarial Detection: Coupling standard confidence and learned "rectified confidence" metrics enables provably sharper discrimination between correctly and incorrectly classified (or adversarial) inputs, improving separation and robust accuracy under the constraints imposed by both metrics jointly (Pang et al., 2021).
• LLM Reliability: Reliability metrics couple "truthfulness" (correct answers plus correct refusals) with "accuracy" (overall correct answers), parameterized by a user appetite parameter $\alpha$ to produce a one-parameter family of coupled reliability scores (Xu et al., 2024).

2. Foundational Theoretical Properties

The coupling of rejection metrics enables properties and guarantees that remain inaccessible to independently applied rules:

• Power versus Error Tradeoff: Coupled metrics internalize inherent tradeoffs—such as those between type I error, type II error, and user priors—in ways that univariate metrics cannot. For instance, in the coupled rejection odds framework, reporting both $R_{\text{pre}}$ and $R_{\text{post}}$ aligns frequentist and Bayesian inferential practice and reveals the true strength of statistical evidence (Bayarri et al., 2015).
• Coupling Quotients and Rejection Geometry: The quotient $R_{\text{post}}(x)$0, defined as $R_{\text{post}}(x)$1, quantifies the centrality or diffuseness of a p-value pooling method, interpolating between minimal and maximal coupling regimes (e.g., between Tippett’s and Stouffer’s methods) and controlling sensitivity to different alternative hypothesis structures (Salahub et al., 2023).
• Multi-threshold Consistency and Monotonicity: In selective classification, coupling risk and coverage via AUGRC ensures monotonicity with respect to both classifier accuracy and confidence ranking, correcting interpretability flaws present in prior metrics that aggregated only conditional risks (Traub et al., 2024).

3. Construction and Mechanisms

The coupling of rejection metrics is instantiated across methodologies with:

• Explicit Formulaic Relations: In the ExCeeD-based unsupervised anomaly detection, the stability metric $R_{\text{post}}(x)$2 is computed as $R_{\text{post}}(x)$3, and a fixed threshold $R_{\text{post}}(x)$4 is derived via theoretical analysis, coupling the uncertainty distribution to a global rejection decision (Perini et al., 2023).
• Algorithmic Synchronization: For adversarial example discrimination, the RR module uses coupled heads (confidence and rectified confidence), with the rectified head trained to approximate the (inaccessible) true confidence by minimizing a cross-entropy loss, yielding theoretical separation of correct and incorrect inputs above a joint threshold (Pang et al., 2021).
• Pooled Test Statistics: The $R_{\text{post}}(x)$5-quantile family of pooled $R_{\text{post}}(x)$6-values defines, for each $R_{\text{post}}(x)$7, a family $R_{\text{post}}(x)$8 that smoothly interpolates the geometry of rejection regions, and the parameter $R_{\text{post}}(x)$9 controls the degree of coupling between the central and marginal rejection regimes (Salahub et al., 2023).
• Composite Reliability Objectives: Reliability metrics for LLMs (truthfulness, accuracy, and answer rates) are composed as $p$0, parameterizing the coupling between aversion to error and reward for informativeness (Xu et al., 2024).

4. Motivations and Applications

The development and deployment of coupled rejection metrics is motivated by:

• Calibration of Statistical Evidence: Reporting both $p$1 and $p$2 corrects for the interpretation pitfalls of conventional $p$3-values and aligns evidence quantification with the experimental and inferential context (Bayarri et al., 2015).
• Adaptive Control of Error and Coverage: In selective classification, joint risk/coverage metrics provide guarantees for system-level performance across thresholds, robust to confidence score miscalibration and model uncertainties (Traub et al., 2024).
• Adversarial Robustness: The RR module demonstrates that coupling classifier uncertainty sources enables provable separation between errors on natural and adversarial inputs, moving beyond static thresholds (Pang et al., 2021).
• Unsupervised and Label-free Guarantees: The RejEx method demonstrates that, by coupling the uncertainty score to a constant threshold, one can establish finite-sample guaranteed bounds on rejection rate and cost in unsupervised settings without recourse to labelled data (Perini et al., 2023).
• User-tunable Trustworthiness: In LLMs, the explicit parameterization of reliability via coupled metrics allows system designers to tune models to user preferences for informativeness versus caution, validated empirically under knowledge feedback-guided alignment (Xu et al., 2024).

5. Typical Metrics, Tables, and Key Results

The salient coupled rejection metrics can be concisely summarized:

Setting	Coupled Metrics	Key Properties / Equations
Hypothesis Testing (Bayarri et al., 2015)	$p$4, $p$5	Unified odds, lower bounds for Bayes factors
$p$6-value Pooling (Salahub et al., 2023)	$p$7, $p$8, $p$9	Quantifies marginal vs. central sensitivity
Selective Classification (Traub et al., 2024)	Risk, Coverage, AUGRC	Joint aggregate risk monotonic in accuracy
Unsupervised Detection (Perini et al., 2023)	$R_c(\alpha)$0, fixed $R_c(\alpha)$1, cost bounds	Detector-agnostic, label-free control
Adversarial Detection (Pang et al., 2021)	Con / R-Con, RR module	Provable error separation, robust accuracy
LLM Reliability (Xu et al., 2024)	accuracy, truthfulness, rely($R_c(\alpha)$2)	Parameterized tradeoff, empirical validation

Empirical findings include that RejEx outperforms uncoupled metric-based methods under a variety of cost structures (Perini et al., 2023); RR modules consistently improve robust accuracy over strong baselines under white-box and adaptive attacks on CIFAR-10/100 (Pang et al., 2021); and ignoring coupling in pooled $R_c(\alpha)$3-values leads to suboptimal error rates across composite null hypotheses (Salahub et al., 2023).

6. Limitations and Ongoing Research

While coupled rejection metrics provide structural and theoretical advantages, there are notable limitations and areas under investigation:

• Granularity: Methods such as RejEx, employing a constant threshold, may be coarse in their rejection and unable to address instance-level cost structures or highly accurate detectors that would benefit from adaptive coupling (Perini et al., 2023).
• Label Dependence and Calibration: Certain coupled metrics (e.g., RR) require auxiliary models or labels during training, and their optimality is contingent on the accuracy of learned rectifiers or approximators (Pang et al., 2021).
• Parameter Selection: The sensitivity of coupling parameters (such as $R_c(\alpha)$4 in $R_c(\alpha)$5) to data distribution, and the need for robust selection or scanning strategies, remains an active point of methodological research (Salahub et al., 2023).
• Interpretability: Some coupled metrics are more difficult to interpret for practitioners compared to scalar significance thresholds or classical risk measures. Interpretable metrics such as AUGRC and explicit reliability scores have been introduced as countermeasures (Traub et al., 2024, Xu et al., 2024).

7. Summary and Outlook

Coupled rejection metrics unify disparate aspects of statistical decision-making, performance characterization, and inference by synchronizing multiple rejection criteria to achieve nuanced control over error, coverage, or other task-driven desiderata. They are central to the state of the art in hypothesis testing, robust machine learning, anomaly and adversarial detection, composite null evaluation, and modern benchmarks for selective reliability in AI systems. Ongoing work seeks to refine these metrics for greater flexibility, adaptivity, and interpretability while preserving their theoretical soundness and guarantees.