A spatial hue similarity measure for assessment of colourisation

Abstract: Automatic colourisation of grey-scale images is an ill-posed multi-modal problem. Where full-reference images exist, objective performance measures rely on pixel-difference techniques such as MSE and PSNR. These measures penalise any plausible modes other than the reference ground-truth; They often fail to adequately penalise implausible modes if they are close in pixel distance to the ground-truth; As these are pixel-difference methods they cannot assess spatial coherency. We use the polar form of the a*b* channels from the CIEL*a*b* colour space to separate the multi-modal problems, which we confine to the hue channel, and the common-mode which applies to the chroma channel. We apply SSIM to the chroma channel but reformulate SSIM for the hue channel to a measure we call the Spatial Hue Similarity Measure (SHSM). This reformulation allows spatially-coherent hue channels to achieve a high score while penalising spatially-incoherent modes. This method allows qualitative and quantitative performance comparison of SOTA colourisation methods and reduces reliance on subjective human visual inspection.