Non-Transitivity of the Win Ratio and the Area Under the Receiver Operating Characteristics Curve (AUC): a case for evaluating the strength of stochastic comparisons

Abstract: The win ratio (WR) is a novel statistic used in randomized controlled trials that can account for hierarchies within event outcomes. In this paper we report and study the long-run non-transitive behavior of the win ratio and the closely related Area Under the Receiver Operating Characteristics Curve (AUC) and argue that their transitivity cannot be taken for granted. Crucially, traditional within-group statistics (i.e., comparison of means) are always transitive, while the WR can detect non-transitivity. Non-transitivity provides valuable information on the stochastic relationship between two treatment groups, which should be tested and reported. We specify the necessary conditions for transitivity, the sufficient conditions for non-transitivity, and demonstrate non-transitivity in a real-life large randomized controlled trial for the WR of time-to-death. Our results can be used to rule out or evaluate the possibility of non-transitivity and show the importance of studying the strength of stochastic relationships.