Lost in the Tower of Babel: The Adverse Effects of Incidental Multilingualism in LLMs

Abstract: This paper argues that contemporary multilingual NLP has converged on a fragile and misleading paradigm of incidental multilingualism. Today's LLMs appear multilingual largely because they are trained on massive, uneven web corpora, not because multilingual or multicultural competence has been treated as a core design objective. We contend that this paradigm systematically produces unequal, brittle, and opaque behavior across languages, with severe consequences in real-world and agentic deployments where models must reason, plan, and act across multiple linguistic contexts. We report a focused empirical study of two practical questions: which languages models self-report as supported and which languages they actually respond in across multilingual prompts. We additionally demonstrate how even a simple language-change attack can surface these failures and expose hidden assumptions about language in LLM-based systems. To address this, we call for a shift toward multilingualism by design: a research agenda that treats equitable multilingual performance, cultural grounding, and cross-lingual behavioral understanding as first-class goals in all aspects of the model pipeline.

• The paper shows that incidental multilingualism leads to brittle language support and inconsistent task performance across 203 languages.
• Empirical evaluations expose a stark mismatch between self-declared multilingual capabilities and actual operational competence.
• The study advocates a shift to multilingualism by design to ensure reliability, fairness, and transparency in high-stakes applications.

The Adverse Effects of Incidental Multilingualism in LLMs

Incidental Multilingualism: Fragility and Inequity

"Lost in the Tower of Babel: The Adverse Effects of Incidental Multilingualism in LLMs" (2605.01224) interrogates prevailing assumptions about multilingual NLP, positing that contemporary LLMs achieve broad language coverage not by design but as an emergent byproduct of training on heterogeneous web corpora. This paradigm, termed incidental multilingualism, fails to provide robust or equitable language support, instead yielding brittle and opaque multilingual behavior. The paper asserts that the prevailing methodology—where multilingual capacity is neither a central objective nor systematically engineered—produces models that behave inconsistently across linguistic boundaries.

Empirical investigation reveals stark discrepancies between self-declared model support and operational multilingual competence. Across five frontier LLMs, support list elicitation is highly sensitive to prompt variation: the intersection of supported languages across paraphrased prompts often collapses to a single language (English), while the union ranges widely (from 18 to 187 languages), emphasizing instability and overclaiming. Subsequent behavioral verification with language identification tools (e.g., GLOTLID) narrows the confirmed support, but even then, only a small, unstable subset is reliably supported across prompts.

Figure 1: Support claims of LLMs fluctuate sharply with prompt phrasing, and actual multilingual behavior is not well calibrated to these claims.

Task-Dependent Multilingual Performance

The paper evaluates task performance in 203 languages across generation, translation, pedagogical dialogue, and code production. Correctness and target-language retention exhibit pronounced variance by language and task. GPT-5.4 achieves the highest translation quality ($\mathrm{CHRF}++$: 35.44 EN$\rightarrow$X, 45.46 X$\rightarrow$EN), while CLAUDE OPUS 4.6 leads on code functional correctness (0.855). However, parseable code remains easier for all models than functional code, indicating a gap between syntactic fluency and true semantic comprehension. In contrast, refusal rates (explicit abstention when unable to perform a task) are virtually non-existent; models predominantly produce incorrect output rather than declining, raising concerns about operational transparency.

Figure 2: LLMs exhibit highly disparate performance across core multilingual tasks, with correctness rates trailing well behind non-refusal rates.

Long-form generation further accentuates these disparities. Target-language retention is consistent in high-resource languages (class 5, per Joshi et al., 2020), but performance deteriorates rapidly for the lowest-resource classes, revealing that broad coverage claims fail most acutely for underrepresented languages.

Figure 3: LLMs maintain target-language fidelity in long-form writing principally for high-resource languages; the hardest classes reveal substantial gaps.

Contradictions and Hidden Failures

A striking finding is the frequency of contradictory behaviors: models routinely claim lack of support for a language either in support lists or direct self-reports, yet subsequently handle downstream tasks in those languages correctly. This phenomenon suggests that self-assessment via prompts is fundamentally unreliable and underscores latent capabilities disconnected from claimed frontiers.

Failure modes typically manifest as incorrect language output rather than explicit refusal. Thus, the user-facing perception is one of broad capability, but operational reliability is much lower, especially in low-resource contexts where the stakes are higher.

Implications for Agentic Systems: The Tower of Babel Problem

The paper extrapolates these risks to multilingual agentic systems, highlighting vulnerabilities that arise when LLM-based agents operate or collaborate across languages. Through simulated collaborative tasks (e.g., MMLU-REDUX), the authors demonstrate performance degradation when agents are forced into multilingual operation—a scenario termed the Tower of Babel (ToB) problem. Language-induced divergence not only affects task accuracy but also biases agent evaluation and selection. Certain agents receive lower scores solely due to their language, regardless of content quality, mirroring real-world sociolinguistic hierarchies and risking systematic bias in AI-driven decision-making.

Figure 4: When agentic LLM systems are forced into multilingual operation, task performance and coordination degrade sharply.

Toward Multilingualism by Design

The research advocates the transition to multilingualism by design as a paradigm shift. This involves explicit integration of multilingual and multicultural objectives at every stage of the LLM pipeline—from data curation and modeling to evaluation and deployment. The authors argue that strongly shared representations (joint subword vocabularies and parameter sharing) risk erasing critical linguistic distinctions, while language-specific interventions, though potentially reducing transfer efficiency, can preserve local variation and improve culturally grounded behavior.

Provider transparency is a key recommendation: LLM vendors should publish versioned supported-language lists, language-level evaluation cards, and clear refusal policies. When support is inadequate, models should abstain and provide meaningful fallback guidance instead of overconfident, incorrect responses.

Theoretical and Practical Implications

The findings have substantial practical ramifications for high-stakes domains such as healthcare, education, legal and public services, where reliability across languages is imperative. Theoretically, they emphasize the necessity for principled research into how cross-lingual interference, language-neutral subnetworks, and culture-specific signals interact in large-scale models. The paper challenges the adequacy of translate-test paradigms and translation-based cross-lingual transfer, arguing such solutions only mitigate symptoms of the core problem.

Speculatively, the paper urges exploration of non-linguistic or hybrid inter-agent communication protocols for collaborative LLM systems, as human language may introduce ambiguities and biases detrimental to robust coordination.

Conclusion

This study demonstrates that official and self-declared multilingual support in frontier LLMs is unstable, over-claimed, and poorly calibrated to actual behavioral competence. Incidental multilingualism is not a sufficient foundation for global deployment, particularly in agentic or critical applications. The paper calls for transparency, robust refusal behavior, and a research agenda predicated on explicit, equitable multilingual and multicultural design. The transition to multilingualism by design is imperative for ensuring reliability, fairness, and safety in the next generation of LLMs.