A Reduced Basis Decomposition Approach to Efficient Data Collection in Pairwise Comparison Studies

Abstract: Comparative judgement studies elicit quality assessments through pairwise comparisons, typically analysed using the Bradley-Terry model. A challenge in these studies is experimental design, specifically, determining the optimal pairs to compare to maximize statistical efficiency. Constructing static experimental designs for these studies requires spectral decomposition of a covariance matrix over pairs of pairs, which becomes computationally infeasible for studies with more than approximately 150 objects. We propose a scalable method based on reduced basis decomposition that bypasses explicit construction of this matrix, achieving computational savings of two to three orders of magnitude. We establish eigenvalue bounds guaranteeing approximation quality and characterise the rank structure of the design matrix. Simulations demonstrate speedup factors exceeding 100 for studies with 64 or more objects, with negligible approximation error. We apply the method to construct designs for a 452-region spatial study in under 7 minutes and enable real-time design updates for classroom peer assessment, reducing computation time from 15 minutes to 15 seconds.