A New Correlation Coefficient for Aggregating Non-strict and Incomplete Rankings (1801.07793v4)

Abstract: We introduce a correlation coefficient that is designed to deal with a variety of ranking formats including those containing non-strict (i.e., with-ties) and incomplete (i.e., unknown) preferences. The correlation coefficient is designed to enforce a neutral treatment of incompleteness whereby no assumptions are made about individual preferences involving unranked objects. The new measure, which can be regarded as a generalization of the seminal Kendall tau correlation coefficient, is proven to satisfy a set of metric-like axioms and to be equivalent to a recently developed ranking distance function associated with Kemeny aggregation. In an effort to further unify and enhance both robust ranking methodologies, this work proves the equivalence of an additional distance and correlation-coefficient pairing in the space of non-strict incomplete rankings. These connections induce new exact optimization methodologies: a specialized branch and bound algorithm and an exact integer programming formulation. Moreover, the bridging of these complementary theories reinforces the singular suitability of the featured correlation coefficient to solve the general consensus ranking problem. The latter premise is bolstered by an accompanying set of experiments on random instances, which are generated via a herein developed sampling technique connected with the classic Mallows distribution of ranking data. Associated experiments with the branch and bound algorithm demonstrate that, as data becomes noisier, the featured correlation coefficient yields relatively fewer alternative optimal solutions and that the aggregate rankings tend to be closer to an underlying ground truth shared by a majority.