Identify the synchronization cues responsible for AVTrace failures

Identify which synchronization cues or desynchronization rendering types drive the failures of omni models on Cat3 audio-visual synchronization verification in AVTrace.

Background

Cat3 evaluates whether audio and video are synchronized, using synchronized positives and several kinds of desynchronized negatives, including global shifts, local shifts, local audio replacement, and cross-splice rendering. Model accuracies remain low, and only the two post-trained Gemma4 variants exceed the test split’s majority-label baseline.

The reported aggregate results are not broken down by desynchronization subtype. Consequently, the study does not determine whether failures arise primarily from difficulty detecting particular temporal offsets, distinguishing content replacement, recognizing cross-spliced audio-visual events, or using other synchronization cues. The authors explicitly characterize Cat3 as unresolved and state that the available results cannot identify the source of the failures.

References

Cat3 (A/V synchronization verification) is also unresolved. Despite high valid-output rates, only the two post-trained variants exceed the 0.556 majority-label baseline on the test partition. Gold's affirmative-response rate is 0.364, below the 0.444 reference positive-label rate shown in Figure~\ref{fig:test-components}; these aggregate rates therefore do not establish reliable synchronization discrimination. The reported results are not stratified by desynchronization subtype, so they cannot identify which synchronization cues or rendering types drive these failures.

— AVTrace: Diagnosing Audio-Visual Temporal Reasoning in Omni Models  (2609.19991 - Zhang et al., 17 Sep 2026) in Section 6, paragraph beginning “Cat3 (A/V synchronization verification) is also unresolved.”