Disentangle speaker timbre and pronunciation effects in speech-based ICL

Determine whether the performance gains from same-speaker demonstrations in speech-based in-context learning for mispronunciation detection and diagnosis arise from shared speaker timbre, pronunciation-related characteristics, or both, using disentangled speech representations.

Background

X-MDD uses annotated demonstrations during inference to adapt its mispronunciation detection and diagnosis predictions. The paper reports that demonstrations from the same speaker generally improve performance across Mandarin, English, and Arabic compared with demonstrations from other speakers, suggesting that speaker matching provides useful information for speech-based in-context learning.

However, same-speaker demonstrations simultaneously share multiple attributes, including vocal timbre and pronunciation-specific characteristics. The paper does not establish which of these factors explains the observed gains and proposes disentangled speech representations as a way to investigate their respective contributions.