Point-Biserial Correlation Coefficient (PBC)

• PBC is a statistical measure that quantifies the linear relationship between a binary outcome and a continuous variable, aiding in uncertainty evaluation.
• It computes Pearson correlation by comparing means and variances of uncertainty measures for dichotomous rewards, ensuring precise alignment metrics.
• PBC guides active learning in reinforcement learning by identifying instances where high uncertainty aligns with incorrect responses, optimizing sample selection.

The point-biserial correlation coefficient (PBC) is a statistical measure evaluating the association between a dichotomous (binary) variable and a continuous variable. In the context of reinforcement learning with verifiable reward (RLVR), PBC quantifies the alignment between the model's subjective uncertainty and objective correctness signals, providing a principled way to guide active learning and query selection. Strong negative point-biserial correlation indicates effective uncertainty consistency, where high model uncertainty tends to coincide with incorrect responses, and low uncertainty with correct responses, thereby informing the selection of informative samples for RL updates (Yi et al., 30 Jan 2026).

1. Formal Definition and Statistical Properties

Suppose $R\in\{0,1\}$ is a binary random variable (e.g., "response is correct" vs. "incorrect") and $U$ is a continuous variable representing model subjective uncertainty (such as entropy, margin, or perplexity). The point-biserial correlation coefficient $r_{pb}$ is defined as the Pearson correlation between $R$ and $U$. Let:

• $K_1$ = number of samples with $R=1$
• $K_0$ = number of samples with $R=0$
• $K = K_0 + K_1$
• $\overline{U}_1$ = mean of $U$ for samples with $R=1$
• $\overline{U}_0$ = mean of $U$ for samples with $R=0$
• $s_U$ = standard deviation of $U$ over all $K$ samples
• $\overline{Y}$ = overall mean of $U$

The coefficient is given by:

$$r_{pb} = \frac{\overline{U}_1 - \overline{U}_0}{s_U} \sqrt{ \frac{K_1 K_0}{K^2} }$$

Interpretation:

• $r_{pb} > 0$: Higher uncertainty aligns with correct responses.
• $r_{pb} < 0$: Higher uncertainty aligns with incorrect responses.

In RLVR for mathematical reasoning, a strong negative $r_{pb}$ is preferred, reflecting that the model expresses greater uncertainty for wrong answers, while being certain when correct (Yi et al., 30 Jan 2026).

2. Calculation in Offline RLVR Settings

For each training query $x^{(i)}$, generate $K$ candidate responses $y_1,\ldots,y_K$. For each response $k$:

• $R_k \in \{0,1\}$ is the Bernoulli reward (correctness).
• $U_k$ is the subjective uncertainty, computed under a fixed reference policy.

Mathematically, define:

• $K_1 = \sum_{k=1}^K R_k$, $K_0 = K - K_1$
• $\overline{U}_1 = \frac{1}{K_1}\sum_{k:R_k=1} U_k$
• $\overline{U}_0 = \frac{1}{K_0}\sum_{k:R_k=0} U_k$
• $$s_U = \sqrt{ \frac{1}{K} \sum_{k=1}^K (U_k - \overline{Y})^2 }$$, where $\overline{Y} = \frac{1}{K} \sum_{k=1}^K U_k$

Hence,

$$r_{pb} = \frac{ \overline{U}_1 - \overline{U}_0 }{ s_U } \sqrt{ \frac{ K_1 K_0 }{ K^2} }$$

A worked example (for $K=4$, $U = [2.0, 3.0, 5.0, 4.0]$, $R = [1, 0, 0, 1]$) yields $r_{pb} \approx -0.4475$, indicating the model is less confident about wrong answers (Yi et al., 30 Jan 2026).

3. Interpretation as an Uncertainty Consistency Metric

The PBC directly operationalizes the notion of uncertainty consistency:

• A negative $r_{pb}$ demonstrates that the model's uncertainty estimates are high precisely when its outputs are incorrect, and low when correct.
• Selecting queries with strongly negative $r_{pb}$ identifies instances where subjective and objective uncertainties are well-aligned, maximizing informativeness for RL-driven policy updates.

This methodology supports identifying queries that are likely to yield the greatest benefit during RLVR, especially when annotation budgets are constrained (Yi et al., 30 Jan 2026).

4. Online Analogue and Theoretical Relationship

In on-policy RL, it is often infeasible to compute offline PBC with large $K$ at each step due to cost and non-stationarity. An online variant is introduced:

$$r_{pb}^{online}(x;\theta) = \frac{1}{K} \left( \sum_{j:\hat{A}_j > 0} \frac{ \hat{A}_j }{U_j^\theta } + \gamma \sum_{j:\hat{A}_j < 0} \frac{ \hat{A}_j }{U_j^\theta } \right)$$

where:

• $\hat{A}_j$ is the normalized (group-standardized) advantage of the $j$th response ($>0$ for above-mean reward, $<0$ otherwise).
• $U_j^\theta$ is the subjective uncertainty for the current policy parameters $\theta$.
• $\gamma > 0$ is a balancing hyperparameter.

This measure acts as a weighted difference of (advantage)/(uncertainty) segregated by response quality. Key theoretical results:

• $\mathrm{Cov}[ r_{pb}, r_{pb}^{online} ] < 0$, so large positive $r_{pb}^{online}$ correlates with low (good) offline PBC.
• Under assumptions including gradient orthogonality and bounded magnitude, maximizing $r_{pb}^{online}$ at each step simultaneously minimizes total subjective uncertainty $\sum_j U_j$, optimizing sample informativeness for RL learning.

5. Practical Considerations and Limitations

Several constraints govern the application of PBC and its variants:

• Sample size ($K$): Offline PBC estimation requires a substantial number of candidate responses (e.g., $K \approx 64$), which is compute-intensive.
• Model non-stationarity: Since the policy evolves during RL, offline PBC values computed on initial reference policies become biased as training progresses.
• Online efficiency: The online version incurs lower cost (few samples per minibatch) but introduces a hyperparameter ($\gamma$) and relies on unverified assumptions about gradient structure.
• Reward type: Metrics require binary (dichotomous) rewards. Extension to graded/continuous rewards necessitates alternative measures (biserial, polyserial, or full Pearson/Spearman correlations).
• Uncertainty characteristics: For heavy-tailed or non-Gaussian uncertainty distributions, rank correlations (e.g., Kendall’s $\tau$, Spearman’s $\rho$) may be more robust.

6. Generalizations, Extensions, and Applications

PBC facilitates informed active learning, query screening, and policy improvement:

• Multiclass or continuous rewards: For $R \in \{0,1,2,\ldots\}$, deploy point-polyserial or correlation measures suitable for continuous data.
• Alternative uncertainty measures: In the presence of outliers or heavy tails in $U$, use rank-based metrics.
• Active learning in broader settings: Offline PBC can guide selection among unlabeled instances in other contexts by estimating $U$ via techniques like MC-dropout and calculating surrogate PBC.
• Promoting sample diversity: To avoid redundancy, combine PBC selection with diversity penalties (e.g., core-set or clustering-based approaches).

In the RLVR domain, using PBC and its online analogue to maximize the alignment between model-reported and verifiable uncertainties supports effective sample selection. This enables preservation or improvement in model performance while substantially reducing annotation and compute costs (Yi et al., 30 Jan 2026).