Score Range Adjustment

• Score Range Adjustment is a collection of methodologies that correct distortions in scoring scales caused by biases, range restrictions, and context-dependent evaluation artifacts.
• It employs rigorous statistical and algorithmic techniques—such as post hoc renormalization, contrastive decoding, and probability adjustments—to restore comparability across different empirical settings.
• These methods improve inferential validity in tests, MOS experiments, and classifier outputs, ensuring that evaluations accurately reflect underlying qualities despite operational constraints.

Score Range Adjustment refers to the family of methodologies and correction procedures for compensating or correcting distortions in the scaling, interpretability, or inferential validity of scores caused by restrictions, biases, or arbitrary choices regarding the score ranges in empirical evaluation, statistical inference, or machine annotation settings. This includes the correction for range restriction in statistical inference, post hoc renormalization of human rating scales in mean opinion score (MOS) experiments, post-processing of classifier predictions under dataset shift, and algorithmic mitigation of range biases in LLMs acting as judges. Each context features distinct sources of bias and entails rigorous mathematical or algorithmic measures for restoring or harmonizing score comparability across operational ranges.

1. Range Restriction and Its Correction in Statistical Inference

Range restriction arises when the observed sample for correlation or regression analyses has been selected in such a manner that the variance of a predictor (or predictors) is artificially reduced with respect to its distribution in the population. This phenomenon is prevalent in standardized test validity studies in higher education, where analysis is typically performed on students who have already met certain admissions criteria. The resulting attenuation in the sample variance of the predictor leads to downwardly biased estimates of the predictor–outcome correlation.

Classic psychometric correction procedures, as formalized in Thorndike's Case II and Case III, address this bias:

• For direct range restriction on a predictor $X$, the true population correlation $\rho_{XY}$ can be recovered as

$\rho_{XY} = \frac{r_{XY}^{r}}{u_X}$

where $$u_X = \sigma_X(\text{restricted}) / \sigma_X(\text{full})$$ and $r_{XY}^{r}$ is the observed (attenuated) correlation in the restricted sample.

• For indirect selection on a composite $Z$, the correction is

$$\rho_{XY} = \frac{r_{XY}^{r} - r_{XZ}^{r} r_{YZ}^{r}}{\sqrt{(1 - (r_{XZ}^{r})^2)(1 - (r_{YZ}^{r})^2)}}$$

These adjustments presuppose bivariate normality and homoscedasticity, with selection determined solely by the targeted variables. Numerical simulation confirms that observed correlations among admits may be reduced by more than half relative to the true population values, recoverable only when proper adjustment is applied (Small, 2017).

2. Range-Equalizing Bias in Human Ratings and Mean Opinion Scores

In MOS tests, especially for synthesized speech, "range-equalizing bias" denotes the tendency for human raters to utilize the full range of a scoring scale in any given context, irrespective of absolute stimulus quality. Operationally, the worst item in a presented test set is mapped to the lowest rating, and the best to the highest. This "rubber ruler" effect leads to context-dependent re-scaling of ratings, severely compromising comparability across differently constructed tests.

The MOS drop for a system $i$ under restricted range $R$ is modeled as

$$\Delta\mathrm{MOS}_i(R) = \mathrm{MOS_i}^{\mathrm{zoom}}(R) - \mathrm{MOS_i}^{\mathrm{orig}}$$

Empirically, a linear fit $\Delta\mathrm{MOS}_i(R) \simeq \alpha + \beta R$ shows $\beta \approx 0$, $\alpha \simeq -1.0$, i.e., a consistent drop of about one MOS point for the worst system as context narrows.

To harmonize scores across different test contexts, the following post hoc correction can be applied:

$$\mathrm{MOS}_i^{\mathrm{corrected}} \simeq \mathrm{MOS}_i^{\mathrm{observed}} + 1.0$$

or, using z-score normalization, scaling listener ratings to a global mean and standard deviation. Experimental evidence indicates significant right-skew in high-quality-only subsets even after equalization, but statistical separability between nearly equivalent systems increases, and observed Spearman’s $\rho$ with the full-range scores drops to as low as 0.31 at extreme restriction (Cooper et al., 2023).

3. Score Range Bias in LLMs and Contrastive Decoding Mitigation

In LLM-as-a-Judge scenarios, score range bias manifests as the model’s output distribution concentrating on particular values within the specified scoring interval, largely independent of ground truth sample quality. This results from LLMs’ reliance on prompt structure and token frequencies, with an explicit "Score range X–Y" rubric priming outputs toward anchor values within this interval. The phenomenon is amplified by shared biases within model families, leading to unstable or non-interpretable score calibrations across experiments.

Contrastive decoding mitigates range bias by exploiting matched biases in model families. For main model logits $\log p_{\mathrm{main}}(i)$ and assistant model logits $\log p_{\mathrm{asst}}(i)$ (the latter temperature-adjusted),

$$\mathrm{score}(i) = \log p_{\mathrm{main}}(i) - \lambda \log p_{\mathrm{asst}}(i)$$

with $\lambda \geq 0$ a tunable hyperparameter. By subtracting scaled assistant log-probabilities, spurious range biases are canceled, restoring sensitivity to true sample quality. Grid search over $(\lambda, t)$, with $t$ the assistant temperature, is performed to calibrate this adjustment, and substantial improvements in correlation with human ratings (Spearman and Pearson) are observed—up to 11.3% relative—across various scoring ranges. The method is robust to assistant model size and incurs minimal computational overhead (Fujinuma, 21 Oct 2025).

4. Adjustment for Dataset and Distribution Shift in Classifier Outputs

Model predictions calibrated on one distribution may become misaligned post-deployment due to shifts in the marginal distribution (class priors). To re-align probability forecasts to the current class distribution, unbounded general adjustment (UGA) and bounded general adjustment (BGA) are applied:

• For $n$ samples, $K$-class outputs $p_{ij}$, and target class distribution $\pi_j$, UGA projects predictions onto affine constraints matching the new priors:

$$a_{ij} = p_{ij} + \varepsilon_j, \quad \varepsilon_j = \pi_j - \frac{1}{n}\sum_{i=1}^n p_{ij}$$

for the Brier score, or

$a_{ij} = \frac{w_j p_{ij}}{\sum_k w_k p_{ik}}$

for log-loss, where $w_j$ recalibrates priors.

• BGA enforces probability simplex constraints and is preferred when negative probabilities are infeasible.

These adjustments are guaranteed to reduce true expected loss for the chosen proper scoring rule, provided class priors $\pi_j$ are correct. Empirically, even moderate errors ($\leq 8\%$) in estimating $\pi_j$ do not abrogate the benefit. Simulations and OpenML benchmarks confirm improved calibration and loss reduction over naive prior adjustment (Heiser et al., 2021).

5. Quantitative Characterization and Impact across Domains

Score range adjustment yields correction factors and diagnostic tools for a range of inferential and evaluative settings:

• In educational measurement, uncorrected range restriction can halve or worse the observed test–outcome correlations, leading to underestimation of test validity. Thorndike corrections recover the true effect size, conditional on baseline variance estimates.
• In MOS experiments, range-equalizing bias results in systematic context-dependent drops of up to 1.28 MOS points for the worst system in narrow-range evaluation; applying a fixed offset or z-score normalization rescales results for comparability.
• For LLM judges, contrastive decoding both removes sensitivity to score range choices and recovers alignment with human-annotated reference scores, thus stabilizing LLM-based evaluation protocols.
• In classifier adaptation, UGA/BGA correction brings the mean predicted probabilities into correspondence with new class priors, strictly reducing proper scoring-rule risk under exact or approximately known distributions.

Summary statistics from key papers are tabulated below:

Setting	Observed Bias (Uncorrected)	Correction Formula / Method
Range restriction	Correlation $r_{XY}$ biased downward	$\rho_{XY} = r_{XY}^r / u_X$
MOS/equalizing bias	$\Delta \mathrm{MOS} \sim -1$ point	$\mathrm{MOS}^{corr} \approx \mathrm{MOS}^{obs} + 1$; z-score normalization
LLM score range bias	Distribution collapse to anchor	Contrastive decoding: $\log p_{main}(i) - \lambda\log p_{asst}(i)$
Classifier calibration	Shifted means / over-/under-confidence	UGA/BGA projection; additive or multiplicative adjustment

6. Practical Guidelines and Methodological Recommendations

For robust inference or fair comparative evaluation under restricted or perturbed scoring ranges:

• Always document and, if possible, empirically estimate the variance of predictors or quality scores both before and after sample selection.
• In MOS and related human ratings, include fixed-quality anchoring stimuli and report both raw and range-adjusted scores; apply z-score or linear shift correction for between-study comparability.
• When benchmarking LLMs or similar judges, use prompt design to specify scoring ranges, family-matched assistant/main models for contrastive decoding, and tune calibration hyperparameters on held-out validation data.
• For classifier adaptation, favor UGA/BGA over naive prior rescaling, especially when prior estimates are imperfect.
• In all cases involving range adjustment, carefully inspect the implications of selection mechanisms, overlap in predictor composition, and possible multi-stage selection artifacts.

Score range adjustment is an indispensable methodological family for counteracting scale, selection, or context-driven distortions in scientific measurement, ensuring that reported experimental or inferential conclusions are valid and reliable across domains and operational regimes (Cooper et al., 2023, Fujinuma, 21 Oct 2025, Heiser et al., 2021, Small, 2017).