Beyond Bjøntegaard: Limits of Video Compression Performance Comparisons

Abstract: For 20 years, the gold standard to evaluate the performance of video codecs is to calculate average differences between ratedistortion curves, also called the "Bj{\o}ntegaard Delta". With the help of this tool, the compression performance of codecs can be compared. In the past years, we could observe that the calculus was also deployed for other metrics than bitrate and distortion in terms of peak signal-to-noise ratio, for example other quality metrics such as video multi-method assessment fusion or hardware-dependent metrics such as the decoding energy. However, it is unclear whether the Bj{\o}ntegaard Delta is a valid way to evaluate these metrics. To this end, this paper reviews several interpolation methods and evaluates their accuracy using different performancemetrics. As a result, we propose to use a novel approach based on Akima interpolation, which returns the most accurate results for a large variety of performance metrics. The approximation accuracy of this new method is determined to be below a bound of 1.5%.