2000 character limit reached
AVATAR submission to the Ego4D AV Transcription Challenge (2211.09966v1)
Published 18 Nov 2022 in cs.CV, cs.MM, cs.SD, eess.AS, and eess.IV
Abstract: In this report, we describe our submission to the Ego4D AudioVisual (AV) Speech Transcription Challenge 2022. Our pipeline is based on AVATAR, a state of the art encoder-decoder model for AV-ASR that performs early fusion of spectrograms and RGB images. We describe the datasets, experimental settings and ablations. Our final method achieves a WER of 68.40 on the challenge test set, outperforming the baseline by 43.7%, and winning the challenge.