FloLPIPS: Temporal Video Quality Metric

• FloLPIPS is a video quality metric that extends LPIPS to assess both spatial and temporal errors in video frame interpolation, effectively capturing motion-consistency distortions.
• It integrates optical flow differences to weight deep feature-distance maps, emphasizing regions with significant temporal inconsistencies.
• Tested on the BVI-VFI database, FloLPIPS statistically outperforms 12 quality assessors, demonstrating robust correlation with subjective evaluations.

FloLPIPS is a full-reference video quality metric for video frame interpolation (VFI) that extends LPIPS by incorporating temporal distortion into the spatial aggregation of deep feature differences. It was introduced to address the open problem of perceptual quality assessment for interpolated video content, particularly the difficulty of capturing motion-consistency errors that are central to VFI artefacts. The metric uses differences in optical flow between reference and distorted videos to weight LPIPS feature-distance maps, thereby emphasizing regions with greater temporal inconsistency. On the BVI-VFI database, FloLPIPS was reported to show superior correlation performance, with statistical significance, against subjective ground truth over 12 popular quality assessors (Danier et al., 2022).

1. Motivation and problem setting

Video frame interpolation serves as a useful tool for many video processing applications and has also been applied in the video compression domain for enhancing both conventional video codecs and learning-based compression architectures (Danier et al., 2022). Despite increased attention to improved interpolation algorithms, perceptual quality assessment of interpolated content remained an open field of research at the time FloLPIPS was introduced.

The central motivation is that traditional quality metrics such as PSNR and SSIM, and even LPIPS, show poor correlation with subjective quality for VFI content. A key limitation identified for these metrics is the failure to consider temporal distortions, in particular errors in motion consistency across frames, which are a primary source of artefacts in VFI. The design target for FloLPIPS is therefore a metric that accounts for spatial perceptual differences, as LPIPS does, while also explicitly