MIMO: Multilingual Information Retrieval via Monolingual Objectives

Abstract: Multilingual Information Retrieval (MLIR) reflects real-world search environments in which queries and relevant documents may appear in different languages within a mixed-language corpus. However, existing embedding models are primarily optimized for Multi-Monolingual retrieval and their performance often degrades in MLIR settings. Moreover, directly applying conventional contrastive learning to MLIR can exacerbate language clustering and expose a trade-off between cross-lingual alignment and embedding uniformity. To address these limitations, we propose MIMO: Multilingual Information Retrieval via Monolingual Objectives, a two-stage framework that uses a stable English semantic space from a high-performing teacher model as an anchor. MIMO first initializes the student model's cross-lingual alignment through knowledge distillation, and then jointly optimizes distillation and cross-lingual contrastive learning to improve retrieval discrimination while preserving alignment. Extensive experiments show that MIMO consistently outperforms existing cross-lingual training baselines across various MLIR and Multi-Monolingual benchmarks. MIMO also remains competitive with off-the-shelf models of similar or larger parameter scales. Furthermore, our cross-lingual Alignment-Uniformity analysis clarifies the distinct roles of the two loss components and shows that their combination yields a favorable trade-off between alignment and uniformity.

• The paper introduces a two-stage framework that first aligns embeddings using English-centric knowledge distillation and then refines them with cross-lingual contrastive learning.
• Empirical results demonstrate state-of-the-art performance with an average nDCG@20 of 60.69 and enhanced language fairness across multiple MLIR datasets.
• The analysis of the alignment–uniformity trade-off offers actionable insights for overcoming language clustering and achieving unbiased multilingual retrieval.

MIMO: Monolingual Objective-Guided Multilingual Information Retrieval

Motivation and Problem Analysis

Current multilingual information retrieval (MLIR) systems confront practical challenges due to an overwhelming focus on multi-monolingual retrieval settings, where queries and documents are paired in the same language. When applied to real-world MLIR scenarios—retrieval across a mixed-language corpus—standard approaches induce language clustering phenomena in the learned embedding spaces, thereby degrading retrieval performance. Dense retriever models optimized on monolingual contrastive objectives exhibit embeddings where semantic similarity is overwritten by language-specific grouping.

This trend is visually confirmed via the embedding space of models such as xlm-roberta-large trained with the InfoNCE loss. Retrieval between queries and documents in the same language achieves high cosine similarity, whereas cross-lingual retrieval collapses to language-specific islands despite semantic identity.

Figure 1: The learned embedding space of xlm-roberta-large trained with InfoNCE demonstrates successful monolingual retrieval (a, b) but fails to maintain cross-lingual semantic isomorphism, as seen by language clustering in MLIR (c, d).

The MIMO Framework

MIMO (Multilingual Information Retrieval via Monolingual Objectives) proposes a two-stage architecture to address the core breakdown in cross-lingual alignment and uniformity that conventional training protocols exacerbate.

• Stage 1: Cross-Lingual Knowledge Distillation—Student multilingual encoders are initialized by regressing to the English-centric embedding space of a high-performing teacher (e.g., Qwen3-Embedding-8B), using parallel data.
• Stage 2: Joint Optimization—Student models are fine-tuned using a weighted objective mixing cross-lingual contrastive loss (XLCO) and continued knowledge distillation. The weighting parameter $\lambda$ interpolates between contrastive discrimination and monolingual space anchoring.

This dual-stage optimization fundamentally mitigates the alignment-uniformity trade-off: distillation drives tight, language-agnostic embedding contraction; contrastive learning enforces corpus-level dispersion and discriminative power.

Training Protocols and Experimental Apparatus

Training utilizes parallel corpora (OPUS for stage 1, mMARCO for stage 2) across 14 languages. Student architectures include xlm-roberta-large and mmBERT-base, with a linear projection to align dimensionality. The teacher model is frozen, and large-scale in-batch negatives are realized with gradient caching (batch size 2048). Downstream evaluation exploits actual MLIR scenarios (e.g., Belebele, NeuCLIR) and traditional benchmarks (e.g., MIRACL).

Empirical Results and Comparative Analysis

MLIR Performance

MIMO sets new state-of-the-art results across all MLIR datasets—substantially outperforming InfoNCE (monolingual only), XLCO (cross-lingual contrastive), and LaKDA (distributional alignment) baselines. For instance, on a suite of 6 MLIR datasets (Belebele, MLQA, XQuAD, MultiEuP, NeuCLIR22–23), xlm-roberta-large with MIMO achieves average nDCG@20 of 60.69, substantially higher than strong XLCO/LaKDA baselines. These gains are robust to backbone model scale.

Training Stages Ablation

Pure distillation (stage 1 only) yields superior alignment but lacks retrieval discrimination; stage 2 without initialization fails to organize the multilingual space, testing the necessity of the two-stage pipeline. Full MIMO achieves the optimal trade-off—furnishing both language-robust alignment and high retrieval discriminativeness.

Loss Balance and Trade-Off

The performance landscape as a function of $\lambda$ reveals an optimal region at $\lambda=0.2$. As the weight on contrastive loss increases, uniformity improves but alignment collapses; as distillation dominates, performance likewise degrades due to insufficient discrimination.

Figure 2: Average MLIR performance (nDCG@20) peaks at $\lambda=0.2$, revealing the criticality of balancing contrastive discrimination and cross-lingual alignment.

Alignment vs. Uniformity

Quantitative analysis using the extended Alignment-Uniformity diagnostic (Wang & Isola, 2020) confirms the distinct roles of each loss: knowledge distillation minimizes cross-lingual distances between parallel documents; contrastive learning regularizes the geometry to maximize retrieval discrimination. MIMO achieves a position in alignment-uniformity space unattainable by XLCO or LaKDA.

Figure 3: MIMO variants navigate the alignment–uniformity Pareto frontier, with joint optimization ($\lambda=0.2$) attaining optimal balance for MLIR.

Language Fairness and Consistency

Fairness and linguistic impartiality are quantified via the Multilingual Rank-based Consistency (MRC@10) metric, computing rank correlation for retrieved lists across all query languages. MIMO achieves the most uniform radar chart profile and highest MRC scores, demonstrating greater invariance to query language and resilience against language-specific bias compared to InfoNCE, XLCO, and LaKDA.

Figure 4: MIMO dramatically improves language consistency and retrieval fairness as measured by MRC@10.

Multi-Monolingual Retrieval

Contrary to claims that strong cross-lingual alignment degrades within-language retrieval, MIMO also sets new high-watermarks on monolingual benchmarks (MIRACL), indicating generalizable embedding improvements.

Off-the-Shelf Model Comparisons

Despite smaller parameterization (0.6B scale), MIMO approaches or surpasses heavily engineered, proprietary production models such as Qwen3-Embedding-0.6B and bge-m3. Unlike these, MIMO relies solely on open datasets and eschews complex hard-negative mining, underscoring the efficacy of the training objective.

Theoretical and Practical Implications

MIMO clarifies that absolute anchoring to a monolingual semantic space combined with joint contrastive learning is necessary to transcend the geometric barriers limiting cross-lingual retrieval systems. The framework unifies the goals of alignment and uniformity, achieving language-invariant retrieval without sacrificing discriminative power. These insights extend to broader LLM alignment regimes, cross-domain retrieval, and potential large-scale automatic alignment of low-resource languages conditioned on English representations.

Figure 5: Against all baselines, MIMO dominates the alignment-uniformity space, substantiating its superiority both in MLIR and geometric embedding diagnostics.

Future Directions

While MIMO relies on robust parallel data, advances in weakly and unsupervised cross-lingual mapping can further relax these assumptions. Extension to ultra-low-resource and morphologically rich languages, integration with hard-negative mining, and deployment in production-grade retrieval pipelines are all plausible, high-yield directions.

Conclusion

MIMO codifies a simple but powerful paradigm for MLIR: initialize to a robust, English-centric semantic anchor, and balance cross-lingual contrastive discrimination with monolingual objectives. The approach eliminates the trade-offs that have characterized multilingual retrieval, yielding superior, unbiased, and generalizable systems.