Mechanism behind the multilingual-coverage peak

Identify the mechanism that produces the observed peak in out-of-domain generalization at approximately 100 to 128 pretrained languages, followed by no consistent improvement at 1,406 languages.

Background

The controlled coverage sweep finds that the approximately 128-language XLS-R encoder outperforms both lower-coverage encoders and the 1,406-language MMS encoder on the farther out-of-domain benchmarks. The discussion suggests a possible trade-off between multilingual breadth and per-language representation quality, but the experimental design does not test that explanation directly. The causal mechanism responsible for the non-monotonic coverage effect therefore remains unresolved.

References

Whether these effects generalize across training corpora and detection recipes, and what mechanism produces the coverage peak, are open questions for future work.