Insensitivity of KEEP performance to optical flow estimator accuracy

Investigate whether the limited effect of optical flow estimator accuracy on the performance of the Kalman-Inspired fEaturE Propagation (KEEP) framework for video face super-resolution arises from two specific factors: (i) attenuation of minor pixel-space misalignments due to the 32× downsampling of latent codes, which makes latent representations less sensitive to small spatial discrepancies, and (ii) compensation of flow estimation inaccuracies by jointly trained modules within KEEP. Establish the validity of these two factors as the explanation for the observed insensitivity to flow estimator choice.

Background

KEEP models temporal dynamics by warping the previously decoded high-quality estimate to the current frame using optical flow computed from low-quality frames, and then re-encoding to predict the current latent state. To assess sensitivity to the flow component, the paper compares KEEP variants using different optical flow estimators (PWC-Net and GMFlow) and reports negligible performance differences.

The authors explicitly conjecture that two mechanisms explain this insensitivity: first, the heavy downsampling of latent codes (by a factor of 32) reduces sensitivity to small pixel-level misalignments; second, other jointly trained modules in the network can compensate for inaccuracies in the estimated optical flow. Validating this conjecture would clarify the role of flow accuracy in KEEP's design and training.