Bhattacharyya Veto: Joint-Ideal Rejection Rule

• Bhattacharyya veto is a joint-ideal rejection rule that thresholds a skewed Bhattacharyya divergence between the Bayes-optimal and model posteriors.
• It is derived under a modified log-loss formulation and contrasts with Chow’s Rule by using divergence-based thresholding instead of KL-based measures.
• The method involves posterior approximation, calibration of rejection thresholds based on a preset budget, and tuning sensitivity via the parameter lambda.

Searching arXiv for the cited paper and closely related work to ground the article. {"query":"arXiv (Soen, 8 May 2025) A Connection Between Learning to Reject and Bhattacharyya Divergences", "max_results": 5} {"query":"(Soen, 8 May 2025)", "max_results": 10} The Bhattacharyya veto is a rejection rule for classification with abstention that arises when learning a joint ideal distribution over inputs and labels and then thresholding a skewed Bhattacharyya divergence between the Bayes-optimal posterior and the model posterior. In the formulation developed by Soen et al., the rule appears under a modified log-loss and contrasts with the marginal case associated with Chow’s Rule: the joint-ideal construction yields Bhattacharyya-based thresholding, whereas the marginal construction yields Kullback–Leibler thresholding. The resulting rejector is described as less aggressive than Chow’s Rule in the sense that, after suitable threshold rescaling, it rejects fewer examples for the same nominal rejection rate (Soen, 8 May 2025).

1. Learning to reject and the joint-ideal construction

Learning to reject is a learning paradigm in which a model may abstain from making a prediction. One approach learns an ideal marginal distribution with respect to the input domain and compares it to the true marginal distribution via a density ratio. The Bhattacharyya veto belongs to a different construction: it considers learning a joint ideal distribution over both inputs and labels, and links rejection to thresholding statistical divergences (Soen, 8 May 2025).

Within this framework, the central object is the discrepancy between the Bayes-optimal posterior $p^{\star}(y \mid x)$ and the model posterior, written either as $p(y \mid x)$ in the derivation or $\hat p(y \mid x)$ in the implementation outline. The decisive feature of the Bhattacharyya veto is that the rejector is not expressed directly as a threshold on conditional risk, but as a threshold on a skewed Bhattacharyya divergence induced by the joint-ideal optimization.

This construction is explicitly contrasted with the marginal case. Under log-loss, the marginal case is equivalent to a typical characterization of optimal rejection, namely Chow’s Rule, which corresponds to thresholding the Kullback–Leibler divergence. The Bhattacharyya veto therefore occupies a distinct position within rejection theory: it is a joint-ideal, divergence-thresholding rule rather than the standard marginal, KL-based rule.

2. Skewed Bhattacharyya divergence

Let $p(\cdot \mid x)$ and $q(\cdot \mid x)$ be probability-mass functions on the label set $\mathcal{Y}=\{1,\dots,L\}$. For a skew parameter $\beta \in (0,1)$, the skewed Bhattacharyya coefficient is defined as

$$B_{\beta}\bigl(q \| p\bigr) = \sum_{y \in \mathcal{Y}} q(y \mid x)^{\beta}\, p(y \mid x)^{1-\beta},$$

and the corresponding skewed Bhattacharyya divergence is

$$D^{B}_{\beta}\bigl(q \| p\bigr) = -\log B_{\beta}\bigl(q \| p\bigr).$$

These are the fundamental quantities thresholded by the veto rule (Soen, 8 May 2025).

The role of $\beta$ is described through limiting and intermediate behavior. When $p(y \mid x)$0, one recovers $p(y \mid x)$1, which is degenerate. When $p(y \mid x)$2, one recovers $p(y \mid x)$3. For intermediate values, $p(y \mid x)$4 interpolates between $p(y \mid x)$5 and $p(y \mid x)$6, and smaller $p(y \mid x)$7 makes the divergence less sensitive to modes where $p(y \mid x)$8.

This parameter dependence is significant because the veto is not based on a symmetric overlap measure. It uses a skewed coefficient, and the skew is controlled by the optimization parameter $p(y \mid x)$9 through the relation

$\hat p(y \mid x)$0

A plausible implication is that the rejection geometry can be tuned continuously between more and less sensitivity to posterior mismatch, but the formal statement provided is the dependence of the skew on $\hat p(y \mid x)$1.

3. Derivation under the modified log-loss

The Bhattacharyya veto is derived under the modified log-loss

$\hat p(y \mid x)$2

where $\hat p(y \mid x)$3 is the Bayes-optimal posterior. Under this loss, the joint-ideal density-ratio rejector is

$\hat p(y \mid x)$4

Here $\hat p(y \mid x)$5 is the Lagrange multiplier in the joint ideal optimization, and the skew is $\hat p(y \mid x)$6 (Soen, 8 May 2025).

An equivalent formulation uses the Bhattacharyya coefficient: $\hat p(y \mid x)$7 The rule therefore admits two equivalent thresholding views: one can threshold the divergence from below or the coefficient from above.

The conceptual importance of the derivation is that rejection emerges from a density-ratio formulation over a joint ideal distribution, not merely from posterior confidence heuristics. This suggests that the veto is structurally tied to the optimization problem defining the rejector, rather than being an ad hoc post-processing criterion.

4. Relation to Chow’s Rule and KL thresholding

The canonical comparison is with Chow’s Rule. In the notation given for the rejector,

$\hat p(y \mid x)$8

Chow’s Rule thresholds the conditional risk or, under log-loss, the KL divergence, whereas the Bhattacharyya veto thresholds $\hat p(y \mid x)$9 (Soen, 8 May 2025).

A concise comparison is as follows:

Rule	Thresholded quantity	Characterization
Chow’s Rule	$p(\cdot \mid x)$0	KL-based rejection
Bhattacharyya veto	$p(\cdot \mid x)$1	Joint-ideal rejector

The key inequality stated for the comparison is

$p(\cdot \mid x)$2

As a consequence, for a suitable re-scaling of the thresholds,

$p(\cdot \mid x)$3

The stated conclusion is that, for the same nominal rejection rate, Bhattacharyya will defer on fewer examples and is therefore less aggressive than KL-based rejection.

The accompanying intuition is that Bhattacharyya “softens” extreme differences between $p(\cdot \mid x)$4 and $p(\cdot \mid x)$5 via the geometric mean inside the coefficient. A common misconception is to treat the veto as merely another confidence threshold. In the formulation given here, it is instead a specific divergence-thresholding rule whose behavior is analytically related to KL thresholding but not identical to it.

5. Algorithmic realization

The implementation outline begins from the fact that $p(\cdot \mid x)$6 is not known exactly. A typical procedure is:

1. Train the base classifier $p(\cdot \mid x)$7 and collect a held-out calibration set $p(\cdot \mid x)$8.
2. Compute estimated probabilities $p(\cdot \mid x)$9.
3. Approximate the Bayes posterior $q(\cdot \mid x)$0 by either a temperature-scaled version of $q(\cdot \mid x)$1, a nonparametric estimator from the calibration set, or an ensemble of stronger models.
4. Fix $q(\cdot \mid x)$2 with $q(\cdot \mid x)$3.
5. On the calibration set compute

$q(\cdot \mid x)$4

1. Choose $q(\cdot \mid x)$5 so that the fraction of rejections meets a budget, for example by taking $q(\cdot \mid x)$6 to be the $q(\cdot \mid x)$7-th quantile of $q(\cdot \mid x)$8.
2. At test time compute

$q(\cdot \mid x)$9

and reject whenever $\mathcal{Y}=\{1,\dots,L\}$0 (Soen, 8 May 2025).

The practical significance of this outline is twofold. First, the rejector requires a surrogate for the Bayes posterior rather than only the base model’s own scores. Second, the threshold is calibrated to a rejection budget rather than fixed purely analytically. A common misunderstanding would be to assume that the method avoids posterior estimation; the stated procedure shows the opposite, since the approximation of $\mathcal{Y}=\{1,\dots,L\}$1 is an explicit step.

6. Geometric interpretation and illustrative behavior

An illustrative two-class example is given by

$\mathcal{Y}=\{1,\dots,L\}$2

The comparison examines the region in the $\mathcal{Y}=\{1,\dots,L\}$3-plane where each rule rejects. The accompanying contour-plot construction uses matched nominal thresholds for the KL and Bhattacharyya criteria (Soen, 8 May 2025).

The stated observation is that the rejection region for KL strictly contains the rejection region for Bhattacharyya for the same nominal threshold. This is presented as confirmation that the Bhattacharyya veto is less aggressive. In geometric terms, the KL rule rejects a larger subset of posterior mismatches, while the Bhattacharyya rule retains some points that KL would veto.

This example also clarifies the operational meaning of the “softening” effect. Because the Bhattacharyya coefficient combines the two posteriors through powers and a geometric-mean-like aggregation, discrepancies that produce large KL values need not produce comparably large Bhattacharyya divergence values. A plausible implication is that the veto can be preferable when one seeks a rejection rule that is conservative in the sense of abstaining less often for a calibrated budget, although the formal claim supplied is only the relative aggressiveness comparison.

7. Position within rejection theory

The Bhattacharyya veto is situated at the intersection of learning to reject, density-ratio methods, and divergence-based decision rules. Its defining contribution is the explicit connection between a joint-ideal rejector and a skewed Bhattacharyya divergence. In the same account, the marginal case aligns with a KL-based characterization of optimal rejection, thereby separating two families of abstention rules according to the ideal distribution being learned (Soen, 8 May 2025).

Within this perspective, the rule is not a replacement for Chow’s Rule in the sense of reproducing the same criterion. Rather, it is a distinct rejector induced by a different optimization setup and a different divergence. The comparison to KL is therefore structural: Bhattacharyya thresholding is analytically bounded by KL thresholding and empirically illustrated as less aggressive, but it is not claimed to be identical to KL-based optimal rejection.

The main significance of the Bhattacharyya veto is thus conceptual as well as procedural. Conceptually, it identifies a precise divergence corresponding to joint-ideal rejection under a variant of log-loss. Procedurally, it yields a calibration workflow in which posterior approximation, skew selection through $\mathcal{Y}=\{1,\dots,L\}$4, and threshold selection through a rejection budget are all explicit components of the abstention mechanism.