Impact of Non-Babble Noise Types on XLAVS-R Robustness

Determine the impact of noise types other than babble on the performance and robustness of the XLAVS-R cross-lingual audio-visual speech representation when evaluated on noisy inputs, by assessing how different noise conditions affect accuracy and error rates during inference.

Background

XLAVS-R is designed for noise-robust speech perception across more than 100 languages and is evaluated extensively on the MuAViC benchmark. In the paper’s experimental setup, noisy environments are simulated using babble noise, a common but specific type of acoustic interference.

The authors explicitly note that they only evaluated with babble noise during testing, leaving open the question of how other noise types (e.g., music, environmental sounds, mechanical noise, overlapping speech of different characteristics) might affect model robustness. Understanding sensitivity to diverse noise profiles is important for deploying XLAVS-R in real-world conditions where noise types vary.