Reliable significance inference from ensemble score distributions

Establish reliable significance inference from the calibrated ensemble score distributions by accounting for cross-condition covariance and the within-listener dependencies required for panel-level hypothesis testing.

Background

The proposed bagging ensemble produces calibrated empirical distributions of predicted listener scores and can support approximate pairwise comparisons between audio conditions. However, the paper reports that matching marginal score distributions is insufficient for reliable paired decisions because significance tests depend on cross-condition covariance and within-listener dependencies, which are not represented by the current artificial-listener construction.

The authors identify listener-level calibration of cross-condition covariance and condition-dependent uncertainty, validated on independent listening tests, as the direction needed to move from approximate comparisons toward reliable panel-level inference. This problem is distinct from MOS prediction and marginal confidence-interval coverage: it concerns reproducing the dependence structure required for false-positive control, statistical power, and correct significance decisions.