Explain oscillatory trajectories in video representation space

Investigate the cause of sinusoidal or oscillatory trajectories formed by temporal patch features in learned video representation space for clips with single-direction camera motion and limited spatial variation, including whether they arise from structured temporal correlations or approximate translational symmetries in the sequence of video representations.

Background

The paper analyzes per-temporal-patch features from V-JEPA 2 Large clips as trajectories in representation space. Although many clips produce smooth, nearly one-dimensional curves whose colour progression corresponds monotonically to camera motion, clips with single-direction camera motion and limited spatial variation can instead produce sinusoidal or oscillatory PCA trajectories, particularly as the number of frames increases.

The authors explicitly state that they lack a definitive explanation for this pattern. They hypothesize that the trajectories may reflect structured temporal correlations or approximate translational symmetries, potentially producing Fourier-like or oscillatory geometry when pairwise similarity depends primarily on distance along an underlying continuous variable. Determining the mechanism behind this phenomenon would clarify the geometric organization and dynamics of learned video manifolds.