SemBridge: Language Transfer in Sparse Encoders via Multilingual Semantic Bridges

Abstract: Sparse encoders offer high-precision retrieval by representing term importance within a vocabulary space, yet their English-centric structures pose a critical impediment to language transfer for non-English languages. To overcome this structural limitation, we propose SemBridge, a novel embedding initialization method designed for cross-lingual adaptation in sparse encoders by leveraging multilingual bridge models. SemBridge establishes semantic alignments between source and target vocabularies using multilingual dense embeddings as a bridge. Rather than directly relying on all source tokens, SemBridge selects a small set of semantically related source-language tokens and uses them to initialize each target-language token, effectively filtering out semantic noise and reconstructing target tokens as precise linear combinations of core synonyms. This accelerates convergence during fine-tuning and improves training efficiency. Extensive experiments across five languages and four sparse architectures demonstrate that SemBridge achieves superior zero-shot retrieval performance and consistently improves retrieval performance after fine-tuning compared to existing baselines. These results validate SemBridge as a practical solution for deploying high-performance sparse retrieval systems in diverse linguistic environments.

• The paper introduces a method for cross-lingual adaptation in sparse encoders by transferring semantic embeddings from dense multilingual models.
• It employs a two-stage alignment process—token overlap transfer and semantic bridge mapping with Entmax filtering—to achieve efficient semantic normalization.
• Experiments across five languages show superior zero-shot and fine-tuned retrieval performance, rapid convergence, and improved computational efficiency.

SemBridge: Cross-lingual Adaptation of Sparse Encoders via Multilingual Semantic Bridges

Motivation and Structural Bottleneck in Sparse Encoders

Sparse encoders have surged in prominence for IR due to their ability to model term-level importance and direct compatibility with inverted index systems, yielding both interpretable ranking and memory efficiency. However, their explicit vocabulary-centric architecture fundamentally limits cross-lingual transfer: English-centric vocabularies dominate, with negligible representation for non-English tokens. Empirical analysis demonstrates severe token imbalance—e.g., only two Korean tokens present in granite-30m-sparse—creating an inherent structural barrier to semantic coverage in new languages.

Figure 1: Token distribution across sparse encoders. Other and ETC represent additional language and non-linguistic tokens, respectively.

Simple fine-tuning or token replacement does not address this deficiency, as sparse encoders lack latent space flexibility inherent in dense models. Thus, optimal deployment in multilingual settings requires a principled mechanism to transplant source-language semantic knowledge into precise target-language embedding spaces.

SemBridge Methodology: Embedding Initialization through Multilingual Semantic Bridging

SemBridge introduces a targeted embedding initialization strategy that leverages pre-trained multilingual dense models as semantic bridges. The process executes a two-stage alignment:

1. Token Overlap Transfer: Language-agnostic tokens (numbers, symbols, etc.) are matched and their embeddings copied verbatim, preserving universal semantic expressiveness and stability.
2. Semantic Bridge Mapping: For non-overlapping target tokens, SemBridge utilizes a multilingual dense model to project both source and target tokens into a shared vector space. Each target token is aligned to a sparse set of semantically closest source tokens via cosine similarity, filtering noise with Entmax transformation to derive sparse weight vectors. Target embeddings are then constructed as weighted linear combinations of relevant source token embeddings.

   Figure 2: Overview of the SemBridge embedding initialization process.

This approach enables precise semantic transplantation, maintaining embedding dimensionality and immediate architectural compatibility. It circumvents reliance on surface-level token overlap, lexicons, or low-rank approximations, achieving semantic fidelity across diverse scripts and morphological systems.

Experimental Evaluation: Zero-shot and Fine-tuned Retrieval

Experiments span four sparse retrievers (splade-v3, Splade-PP, Opensearch-Sparse-v1, Granite-30M-Sparse) and five languages (Arabic, Chinese, Hindi, Korean, Russian), using nDCG@10 as the retrieval metric across MIRACL and WebFAQ datasets. SemBridge consistently delivers superior zero-shot performance, evidenced by average nDCG@10 scores of 0.422 (splade-v3) and 0.409 (Splade-PP) on WebFAQ, markedly outperforming statistical and cross-lingual initialization baselines.

SemBridge's initialization produces substantially faster convergence and improved final performance after fine-tuning.

Figure 3: Training loss trajectories for Splade-v3 on Chinese, Korean, and Russian using Baseline, OFA, FOCUS, and SemBridge. The y-axis is Loss and the x-axis is Epoch. Insets zoom in on the initial training stage (0-0.2 epoch)

Loss curves demonstrate that SemBridge achieves significantly lower initial losses and rapid convergence, indicating optimal initialization state for efficient knowledge transfer and adaptation under resource constraints.

Ablation Study: Sparsity and Semantic Filtering

Ablation on Entmax hyperparameter $\alpha$ confirms the necessity of sparse weighting in cross-lingual token alignment. Softmax ($\alpha=1$) yields suboptimal performance due to noise from irrelevant source tokens. Sparsemax and Entmax ($\alpha \geq 2$) architectures consistently outperform baselines, with $\alpha=4$ achieving highest robustness by isolating core synonyms.

Figure 4: Zero-shot retrieval performance (nDCG@10) on (a) WebFAQ and (b) MIRACL. For SemBridge, the sparsity level $\alpha$ is varied from 1 to 4, with FOCUS and OFA included as baselines.

Efficiency and Retrieval-Performance Tradeoff

SemBridge achieves a superior efficiency/performance trade-off relative to competing methods. FLOPS analysis demonstrates that SemBridge maintains high sparsity and computational efficiency while yielding maximal retrieval effectiveness; it reliably suppresses extraneous token activity compared to FOCUS (which yields excessive FLOPS) and statistical baselines (which suffer performance collapse).

Figure 5: Efficiency vs. performance trade-off on the MIRACL dev set (Chinese). The y-axis shows FLOPS, and marker color indicates nDCG@10.

Qualitative Analysis: Semantic Fidelity in Token Mapping

Qualitative inspection of top-weighted source token mappings for cross-lingual 'home'-equivalents illustrates precise semantic transfer and filtering. SemBridge consistently selects direct semantic equivalents and core synonyms (e.g., 'home', 'house', 'casa', 'maison') in all languages, outperforming FOCUS and OFA which suffer from incoherent mappings and failure to suppress uninformative subwords or contextually irrelevant tokens.

Practical and Theoretical Implications

SemBridge establishes a paradigm for cross-lingual sparse retrieval model adaptation without the need for large-scale MLM pretraining or lexical resources. The method's robustness to bridge model choice (demonstrated via comparative auxiliary embedding experiments) and strong empirical performance across architectures and languages suggest its utility in broad multilingual IR contexts, especially for low-resource and structurally divergent languages.

Theoretically, the reliance on sparse semantic bridging via dense embedding spaces emphasizes the importance of token-level semantic mapping over surface overlap and statistical initialization, motivating further research into adaptive sparsity selection and semantic bridge optimization.

Conclusion

SemBridge operationalizes multilingual transfer in sparse encoders by enforcing precise token-level semantic alignment through sparse weighting over multilingual dense embeddings. The approach demonstrably outperforms previous methods in both zero-shot retrieval and fine-tuned settings, achieves rapid convergence, and provides a repeatable, robust foundation for building efficient, interpretable IR systems in diverse linguistic environments. SemBridge's design and empirical results underscore the critical role of embedding initialization in preserving and propagating retrieval capabilities beyond English-centric settings (2605.26002).