Bidirectional Preference Entropy (BPE)

• Bidirectional Preference Entropy (BPE) is an order-invariant uncertainty score for pairwise LLM judging that neutralizes response-order bias.
• It computes and averages probabilities from forward and reversed prompts, then converts them into binary entropy to reflect true ambiguity.
• Integrated in the SCOPE framework, BPE aids selective prediction by calibrating judgment acceptance based on a target error rate.

Searching arXiv for the named method and adjacent usages to ground the article in current papers. Bidirectional Preference Entropy (BPE) is an entropy-based uncertainty score for pairwise LLM judging, introduced as the bias-neutral scoring mechanism within SCOPE, a selective conformal framework for pairwise evaluation (Badshah et al., 13 Feb 2026). In this setting, a judge compares two candidate responses and may abstain when its judgment is not sufficiently reliable. BPE addresses a specific pathology of pairwise judging: response-order bias can make ordinary confidence scores appear decisive even when that apparent certainty is driven by positional artifacts rather than stable underlying preference. The method therefore queries the judge under both response orderings, aligns the two implied probabilities to the same latent preference, averages them to enforce response-order invariance, and converts the result into binary entropy. The resulting quantity is not a general-purpose entropy functional; it is a task-specific uncertainty score for selective LLM-as-a-judge evaluation.

1. Motivation and problem setting

BPE was proposed for pairwise evaluation, where an LLM judge receives a prompt and two candidate responses, then predicts which response is better. The core concern is that such judges are often miscalibrated and systematically biased by response order. In that regime, a raw confidence proxy such as $\max_y P_\theta(y\mid x)$ can be spuriously high because the judge consistently prefers the first or second position, not because it is genuinely certain about the underlying quality comparison (Badshah et al., 13 Feb 2026).

This matters most in selective prediction. If the evaluator is allowed to abstain, its uncertainty score must track genuine ambiguity rather than nuisance variation. Otherwise, the evaluator may accept precisely those cases on which it is confidently wrong. BPE is designed as an order-aware uncertainty score: it forces the judge to reveal its preference twice, once in each positional arrangement, then suppresses the positional asymmetry by aggregation before computing entropy. In the SCOPE formulation, this makes BPE the mechanism that determines whether a pairwise judgment is trustworthy enough to accept.

A useful conceptual distinction is that BPE does not estimate the correctness of a response directly. Instead, it estimates uncertainty in the judge’s preference decision after neutralizing the most salient nuisance factor identified in the paper, namely response order. This suggests that BPE is best understood as an uncertainty functional over judging behavior, not over response generation.

2. Formal definition

Given an instance

$x = (q, r_A, r_B),$

BPE evaluates the judge on both the forward and reversed response orderings: $$x_{\text{fwd}} = (q, r_A, r_B), \qquad x_{\text{rev}} = (q, r_B, r_A).$$ From the forward prompt, the model’s probability that $r_A$ is preferred is

$$p_{\text{fwd}} = P_\theta(y=A \mid x_{\text{fwd}}).$$

From the reverse prompt, “prefers $r_A$” corresponds to label $B$ in the swapped presentation, so the aligned reverse probability is

$$p_{\text{rev}} = P_\theta(y=B \mid x_{\text{rev}}).$$

The order-invariant preference estimate is then the arithmetic mean

$$\bar{p} = \frac{1}{2}\left(p_{\text{fwd}} + p_{\text{rev}}\right).$$

Because the task is binary, the aligned probability of preferring $r_B$ is $x = (q, r_A, r_B),$0. BPE converts this aggregated probability into binary entropy: $x = (q, r_A, r_B),$1 This entropy is maximal at $x = (q, r_A, r_B),$2 and minimal near $x = (q, r_A, r_B),$3 or $x = (q, r_A, r_B),$4, so it behaves as an uncertainty score rather than as a confidence score (Badshah et al., 13 Feb 2026).

Operationally, the implementation extracts the logits for the two output tokens corresponding to “A” and “B”, applies a two-way softmax for each ordering, aligns the reverse prediction to the same underlying candidate, averages the two probabilities, and computes entropy. The paper notes that this requires about two forward passes per instance. For confidence-oriented reporting such as AUROC or ECE, the same aggregated probability can be mapped back to

$x = (q, r_A, r_B),$5

which preserves ranking while expressing certainty on a conventional confidence scale.

3. Function within SCOPE

SCOPE, or Selective Conformal Optimized Pairwise Evaluation, uses BPE as the uncertainty score that drives abstention. The selective decision rule is

$x = (q, r_A, r_B),$6

where $x = (q, r_A, r_B),$7 is the judge’s predicted preference and $x = (q, r_A, r_B),$8 denotes abstention (Badshah et al., 13 Feb 2026).

The target risk is the accepted-set error rate,

$x = (q, r_A, r_B),$9

with $$x_{\text{fwd}} = (q, r_A, r_B), \qquad x_{\text{rev}} = (q, r_B, r_A).$$0 the selection indicator and $$x_{\text{fwd}} = (q, r_A, r_B), \qquad x_{\text{rev}} = (q, r_B, r_A).$$1 the error indicator. In other words, SCOPE does not simply seek low uncertainty; it seeks a threshold on BPE such that the error rate among accepted judgments remains below a user-specified level $$x_{\text{fwd}} = (q, r_A, r_B), \qquad x_{\text{rev}} = (q, r_B, r_A).$$2.

Calibration is performed on a labeled calibration set $$x_{\text{fwd}} = (q, r_A, r_B), \qquad x_{\text{rev}} = (q, r_B, r_A).$$3 using the linearized loss

$$x_{\text{fwd}} = (q, r_A, r_B), \qquad x_{\text{rev}} = (q, r_B, r_A).$$4

The selected threshold is the largest $$x_{\text{fwd}} = (q, r_A, r_B), \qquad x_{\text{rev}} = (q, r_B, r_A).$$5 satisfying

$$x_{\text{fwd}} = (q, r_A, r_B), \qquad x_{\text{rev}} = (q, r_B, r_A).$$6

equivalently,

$$x_{\text{fwd}} = (q, r_A, r_B), \qquad x_{\text{rev}} = (q, r_B, r_A).$$7

If no feasible threshold exists, the procedure abstains on all points.

The formal finite-sample guarantee derives from exchangeability between calibration and test samples. Under that assumption, the accepted-set test error rate is bounded by $$x_{\text{fwd}} = (q, r_A, r_B), \qquad x_{\text{rev}} = (q, r_B, r_A).$$8. BPE does not itself provide the guarantee; rather, it supplies the uncertainty score whose quality determines how effectively SCOPE can trade coverage against risk. The paper’s central claim is that an order-invariant entropy score yields a more faithful basis for this calibration than naïve confidence proxies.

4. Empirical behavior

Empirically, BPE improves uncertainty quality over predictive probability, verbalized confidence, and simulated annotators across MT-Bench, RewardBench, and Chatbot Arena (Badshah et al., 13 Feb 2026). The paper reports that BPE generally yields lower ECE and higher AUROC and AUPRC than standard single-prompt confidence measures, while remaining computationally cheaper than simulated-annotator schemes because it requires only two forward passes.

The reported examples are concrete. On MT-Bench with Qwen2.5-7B, predictive probability gives ECE $$x_{\text{fwd}} = (q, r_A, r_B), \qquad x_{\text{rev}} = (q, r_B, r_A).$$9, ROC $r_A$0, and PRC $r_A$1; verbalized confidence gives ECE $r_A$2, ROC $r_A$3, and PRC $r_A$4; BPE gives ECE $r_A$5, ROC $r_A$6, and PRC $r_A$7. On RewardBench with Qwen2.5-32B, predictive probability gives ECE $r_A$8, ROC $r_A$9, and PRC $$p_{\text{fwd}} = P_\theta(y=A \mid x_{\text{fwd}}).$$0; verbalized confidence gives ECE $$p_{\text{fwd}} = P_\theta(y=A \mid x_{\text{fwd}}).$$1, ROC $$p_{\text{fwd}} = P_\theta(y=A \mid x_{\text{fwd}}).$$2, and PRC $$p_{\text{fwd}} = P_\theta(y=A \mid x_{\text{fwd}}).$$3; BPE gives ECE $$p_{\text{fwd}} = P_\theta(y=A \mid x_{\text{fwd}}).$$4, ROC $$p_{\text{fwd}} = P_\theta(y=A \mid x_{\text{fwd}}).$$5, and PRC $$p_{\text{fwd}} = P_\theta(y=A \mid x_{\text{fwd}}).$$6. On Chatbot Arena with Qwen2.5-14B, predictive probability gives ECE $$p_{\text{fwd}} = P_\theta(y=A \mid x_{\text{fwd}}).$$7, ROC $$p_{\text{fwd}} = P_\theta(y=A \mid x_{\text{fwd}}).$$8, and PRC $$p_{\text{fwd}} = P_\theta(y=A \mid x_{\text{fwd}}).$$9; verbalized confidence gives ECE $r_A$0, ROC $r_A$1, and PRC $r_A$2; BPE gives ECE $r_A$3, ROC $r_A$4, and PRC $r_A$5.

Against simulated annotators, BPE often improves discrimination while remaining competitive in calibration. For example, on MT-Bench with Qwen2.5-7B, simulated annotators yield ECE $r_A$6, ROC $r_A$7, and PRC $r_A$8, whereas BPE yields ECE $r_A$9, ROC $B$0, and PRC $B$1. On RewardBench with Qwen2.5-14B, simulated annotators yield ECE $B$2, ROC $B$3, and PRC $B$4, whereas BPE yields ECE $B$5, ROC $B$6, and PRC $B$7.

Its downstream value appears most clearly within SCOPE. At $B$8, SCOPE consistently satisfies the target risk bound across all reported benchmarks and judge scales, with empirical risk approximately $B$9 to $$p_{\text{rev}} = P_\theta(y=B \mid x_{\text{rev}}).$$0, while retaining substantial coverage. The paper reports coverage reaching $$p_{\text{rev}} = P_\theta(y=B \mid x_{\text{rev}}).$$1 on RewardBench with Qwen-14B and $$p_{\text{rev}} = P_\theta(y=B \mid x_{\text{rev}}).$$2 on RewardBench with Qwen-32B. Compared with naïve baselines, SCOPE accepts up to $$p_{\text{rev}} = P_\theta(y=B \mid x_{\text{rev}}).$$3 more judgments on MT-Bench with Qwen-7B under the same target risk constraint. A specific MT-Bench example contrasts naïve calibration, which attains coverage $$p_{\text{rev}} = P_\theta(y=B \mid x_{\text{rev}}).$$4 and risk $$p_{\text{rev}} = P_\theta(y=B \mid x_{\text{rev}}).$$5, with SCOPE, which attains coverage $$p_{\text{rev}} = P_\theta(y=B \mid x_{\text{rev}}).$$6 and risk $$p_{\text{rev}} = P_\theta(y=B \mid x_{\text{rev}}).$$7.

These results support a narrow but important conclusion: BPE is valuable not because it increases confidence, but because it improves the ranking of judgments by reliability after removing position-induced asymmetry. This is why the paper emphasizes discrimination quality and accepted-set risk control rather than raw pointwise confidence.

5. Relation to bidirectional entropy modulation in RLVR

A distinct 2026 line of work studies entropy directionality in reinforcement learning with verifiable rewards (RLVR), especially in the paper “Rethinking Exploration in RLVR: From Entropy Regularization to Refinement via Bidirectional Entropy Modulation” (Gu et al., 6 Apr 2026). That paper does not define Bidirectional Preference Entropy and does not present a method named BPE. Instead, it argues that effective exploration in RLVR requires “entropy refinement”: preserving informative entropy on positive rollouts while suppressing spurious entropy on negative rollouts.

The mechanism there is conceptually adjacent but methodologically different. The RLVR paper claims that group-relative policy optimization already induces a two-directional entropy effect—entropy is sustained on successful rollouts and reduced on failed rollouts—and formalizes this as bidirectional entropy modulation. Its proposed algorithm, AsymGRPO, explicitly decouples the modulation strength for positive and negative rollouts. The object being controlled is policy entropy during reasoning RL, not pairwise judgment uncertainty.

The similarity is therefore one of directional structure rather than of task or formulation. In both cases, a naïve one-sided treatment of entropy is rejected: the RLVR paper rejects blind entropy maximization, while SCOPE rejects single-order confidence as a trustworthy uncertainty proxy. A plausible implication is that both frameworks treat bidirectionality as a way to separate informative variation from nuisance variation, but they do so in different domains and with different mathematical objects. Conflating the two would be incorrect: BPE belongs to selective LLM judging, whereas bidirectional entropy modulation in AsymGRPO belongs to RLVR exploration.

6. Acronym ambiguity and neighboring meanings

The acronym “BPE” is not unique. In quantum information and holography, BPE also denotes Balanced Partial Entanglement, introduced as a purification-dependent correlation measure for bipartite mixed states (Wen, 2021). That quantity is defined through partial entanglement entropy and a balance condition on a purification, satisfies entropy relations similar to entanglement of purification, and in holographic CFT$$p_{\text{rev}} = P_\theta(y=B \mid x_{\text{rev}}).$$8 equals the entanglement wedge cross section divided by $$p_{\text{rev}} = P_\theta(y=B \mid x_{\text{rev}}).$$9. In canonical purification it becomes half of the reflected entropy.

Usage	Domain	Core meaning
Bidirectional Preference Entropy	Pairwise LLM judging	Order-invariant entropy score built from forward/reverse preference probabilities
Bidirectional entropy modulation	RLVR	Positive rollouts sustain entropy; negative rollouts suppress it
Balanced Partial Entanglement	Quantum information / holography	Purification-dependent correlation measure tied to entanglement contour and EWCS

This acronym collision creates an obvious source of confusion. In the LLM-evaluation literature, BPE refers to an uncertainty score for selective pairwise judging (Badshah et al., 13 Feb 2026). In holography, BPE refers to a measure derived from partial entanglement entropy (Wen, 2021). The RLVR paper cited above is related only at the level of bidirectional entropy control and does not use BPE terminology (Gu et al., 6 Apr 2026).

A common misconception is therefore to read “BPE” as a generic label for any bidirectional entropy method. The available evidence does not support that usage. Within the supplied literature, “Bidirectional Preference Entropy” is a specific construction for LLM-as-a-judge uncertainty estimation; “Balanced Partial Entanglement” is a separate established acronym in quantum information; and “bidirectional entropy modulation” in RLVR is conceptually related but terminologically distinct.