BGE-M3 Embeddings: Unified Multilingual Retrieval

• BGE-M3 Embeddings are unified multilingual text representations that combine dense, sparse, and multi-vector retrieval modes to process inputs up to 8192 tokens in over 100 languages.
• The model employs self-knowledge distillation and a hybrid ranking score to calibrate signals from different retrieval strategies, enhancing both recall and precision.
• Adapted for diverse applications such as legal QA, financial semantic tasks, and RAG pipelines, BGE-M3 serves as a versatile first-stage retriever and semantic reward proxy.

Searching arXiv for recent and foundational papers on BGE-M3 embeddings. BGE-M3 Embeddings are multilingual text representations introduced as a unified retrieval foundation that combines multi-linguality, multi-functionality, and multi-granularity within a single model family. The original M3-Embedding formulation supports more than 100 working languages, three retrieval modes—dense retrieval, sparse retrieval, and multi-vector retrieval—and inputs ranging from short passages to long documents of up to 8192 tokens (Chen et al., 2024). Subsequent work has used BGE-M3 in diverse roles: as a first-stage dense retriever in Retrieval-Augmented Generation (RAG), as a semantic evaluation model, as a finance-domain embedding backbone, and as a lightweight semantic reward proxy in reinforcement-learning-based legal QA systems (Yang et al., 8 Jan 2025, Ros et al., 21 May 2026, Lee et al., 13 Jul 2025, Akarajaradwong et al., 13 Jul 2025). The resulting literature presents BGE-M3 not as a single narrowly optimized retriever, but as a general-purpose multilingual embedding infrastructure whose utility depends on task regime, retrieval direction, corpus structure, and domain adaptation.

1. Foundational model design

BGE M3-Embedding is defined in terms of three explicit capabilities. Multi-linguality refers to support for multilingual and cross-lingual retrieval across more than 100 languages. Multi-functionality refers to a single encoder supporting dense retrieval, sparse / lexical retrieval, and multi-vector retrieval. Multi-granularity refers to processing inputs from short sentences to long documents up to 8192 tokens (Chen et al., 2024).

The original paper formulates the model as a unified retriever capable of mapping a query $q^x$ in language $x$ to a relevant document $d^y$ in language $y$, whether retrieval is monolingual or cross-lingual and whether scoring uses dense, sparse, or late-interaction mechanisms (Chen et al., 2024). This matters because the model is not framed as merely a multilingual sentence embedder. It is instead presented as a retrieval system with multiple scoring heads sharing a common encoder.

The dense mode uses normalized [CLS]-based query and passage embeddings,

$$e_q = norm(\mathbf{H_q}[0]), \qquad e_p = norm(\mathbf{H_p}[0]),$$

with dense relevance score

$s_{dense} \leftarrow \langle e_p, e_q \rangle.$

The sparse mode produces token-level lexical weights,

$$w_{q_t} \leftarrow \mathsf{ReLU}\left(\mathbf{W}_{lex}^T \mathbf{H_q}[i]\right),$$

and then scores overlapping terms by

$$s_{lex} \leftarrow \sum_{t\in q \cap p} (w_{q_t} * w_{p_t}).$$

The multi-vector mode projects token representations,

$$E_q = norm(\mathbf{W}_{mul}^T\mathbf{H_q}), \qquad E_p = norm(\mathbf{W}_{mul}^T\mathbf{H_p}),$$

and applies a ColBERT-style max-sim aggregation,

$$s_{mul} \leftarrow \frac{1}{N} \sum_{i=1}^{N} \max_{j=1}^{M} E_q[i] \cdot E_p^{T}[j].$$

The paper also defines a hybrid ranking score,

$x$0

These formulas are central because they show that BGE-M3 is not limited to a single embedding geometry or retrieval semantics (Chen et al., 2024).

A common misconception is that BGE-M3 should be understood purely as a dense embedding model. The foundational paper explicitly defines it as a model that can simultaneously support dense, sparse, and multi-vector retrieval. Many downstream papers, however, use only the dense component for practical reasons such as indexing simplicity or latency constraints. This suggests that discussions of “BGE-M3 embeddings” often refer to a reduced deployment configuration rather than the full model design (Chen et al., 2024, Yang et al., 8 Jan 2025, Cirillo et al., 22 May 2026).

2. Training objective and self-knowledge distillation

The key training innovation in BGE-M3 is self-knowledge distillation (SKD). The base retrieval loss for each mode is InfoNCE: $x$1 Here $x$2 is the positive passage, $x$3 the negatives, and $x$4 one of the mode-specific scores $x$5, $x$6, or $x$7 (Chen et al., 2024).

The distinctive step is that the model constructs an integrated teacher score

$x$8

then encourages each retrieval head to align with the soft distribution induced by this integrated signal: $x$9 with the averaged distillation loss

$d^y$0

and final objective

$d^y$1

The paper frames this as an ensemble-like teacher mechanism allowing the dense, sparse, and multi-vector views to regularize one another (Chen et al., 2024).

This design is significant because multi-objective retrieval training often induces interference across heads. BGE-M3’s SKD scheme treats the integrated multi-mode score as a supervisory prior rather than training the heads independently. A plausible implication is that the model’s empirical robustness in downstream multilingual settings partly derives from this joint calibration of semantic, lexical, and late-interaction signals.

The training pipeline is multi-stage. The base encoder is XLM-RoBERTa further pre-trained with RetroMAE; the model then undergoes unsupervised dense retrieval pretraining on about 1.2B text pairs, followed by multi-head fine-tuning on English, Chinese, multilingual, and synthetic long-document retrieval data (Chen et al., 2024). The paper reports 1.1M English examples, 386.6K Chinese examples, 88.9K multilingual examples, and 41.4K synthetic long-document examples for fine-tuning (Chen et al., 2024).

The long-input regime imposes substantial systems constraints. The implementation uses a batching strategy with length grouping, fixed random seeds across GPUs, and split-batch training. The reported effect is large: at length 8192, maximum batch size per device rises from 6 to 130, and the paper states that for 8192-token text, batch size can increase by more than 20× (Chen et al., 2024). This is part of why BGE-M3 could be trained as a long-document retriever rather than only evaluated as one.

3. Retrieval modes, long-context handling, and benchmark behavior

The foundational evaluation establishes BGE-M3 as a strong multilingual retriever across multilingual, cross-lingual, and long-document settings. On MIRACL dev, average nDCG@10 is reported as 67.8 for M3 Dense, 69.0 for M3 Multi-vec, 68.9 for M3 Dense+Sparse, and 70.0 for M3 All, compared with 65.4 for mE5-large and 62.2 for E5-mistral-7B (Chen et al., 2024). On MKQA Recall@100, M3 All reaches 75.5, above 70.9 for mE5-large and 70.1 for E5-mistral-7B (Chen et al., 2024). On long-document retrieval benchmarks, BGE-M3 is especially strong in hybrid mode: on MLDR nDCG@10, M3 All scores 65.0, and on NarrativeQA it scores 61.7 (Chen et al., 2024).

Long-context support is a defining technical property. The model handles up to 8192 tokens, and the paper introduces MCLS (Multiple CLS) as an inference-time technique that inserts a CLS token every 256 tokens and averages the resulting states to better summarize long inputs when long-document fine-tuning resources are limited. In the ablation setting, MCLS improves long-document retrieval on MLDR from 41.2 to 45.0 nDCG@10 (Chen et al., 2024).

Downstream RAG work often exploits this long-input capacity directly. A Traditional Chinese knowledge-retrieval system built on Chinese Wikipedia and Lawbank uses BGE-M3 as its dense retriever and notes that Chinese Wikipedia articles were mostly short enough to fit within the 8192-token limit without chunking, while Lawbank’s legal materials were segmented into individual legal clauses because of their specialized structure and variable length (Yang et al., 8 Jan 2025). That paper explicitly treats BGE-M3 as the embedding backbone in the pipeline query → BGE-M3 embedding → FAISS dense retrieval → BGE-reranker re-ranking → LLM generation (Yang et al., 8 Jan 2025).

The same paper describes FAISS as the vector index supporting options such as Flat, IVF, and HNSW, with similarity measures including L2 distance, dot product, and cosine similarity. In the deployed system, BGE-M3 is used for dense vector search over Chinese Wikipedia and Lawbank, and BGE-reranker, a cross-encoder, reorders the dense retrieval candidates before they are passed to the LLM (Yang et al., 8 Jan 2025). The paper argues that BGE-M3 is strong at semantic matching, but that semantic closeness alone is not always enough for precise question answering, so reranking is used to optimize for answer relevance rather than topical similarity (Yang et al., 8 Jan 2025).

A similar two-stage architecture appears in Ukrainian document QA. There BGE-M3 is used as the main dense retriever, followed by BGE reranker-v2-m3, and the best development retrieval result is achieved by bge-m3 + bge reranker with Mean $d^y$2 = 0.9219 and Mean $d^y$3 = 0.8111, compared with 0.9002 / 0.7365 for BGE-M3 alone (Sumyk et al., 16 Apr 2026). The paper interprets retrieval quality as the primary bottleneck and treats reranking as a precision-improving stage layered over BGE-M3’s recall-oriented candidate generation (Sumyk et al., 16 Apr 2026).

These findings support a recurring pattern: BGE-M3 is often used as the first-stage retriever, while a cross-encoder or reranker supplies precision. This does not contradict the original multi-mode design. Rather, it suggests that many applied systems prefer to externalize precision optimization into a dedicated reranking stage instead of relying on the model’s built-in hybrid scoring.

4. Multilingual deployment across low-resource and cross-lingual settings

A notable portion of the BGE-M3 literature concerns low-resource or morphologically challenging languages. In Khmer telecom-domain QA, BGE-M3 was compared against Jina-Embeddings-v3 (570M) and Qwen3-Embedding (597M) on a golden dataset of 200 Khmer question-answer pairs derived from official telecom and ICT documents. BGE-M3 was selected as the final retriever because it achieved the strongest dense retrieval results at the deployed top-$d^y$4 setting, k = 3: Hit Rate@3 = 0.285, File Hit Rate@3 = 0.700, MRR@3 = 0.221, and Precision@3 = 0.112 (Ros et al., 21 May 2026).

The paper highlights an instructive exception: Jina-Embeddings-v3 had the highest top-1 cosine similarity at 0.759, above 0.704 for BGE-M3 and 0.686 for Qwen3-Embedding, but this did not translate into better retrieval accuracy (Ros et al., 21 May 2026). The authors therefore argue that raw cosine similarity is not a reliable proxy for retrieval quality in Khmer, particularly under non-Latin script, segmentation difficulty, OCR noise, and mixed Khmer-English terminology (Ros et al., 21 May 2026). Their interpretation is that BGE-M3 better separates truly relevant passages from merely semantically adjacent ones.

In the Indic persona-instruction alignment benchmark spanning 12 Indian languages and four tasks, BGE-M3 again displays mixed but substantial multilingual competence. It supports all 12 benchmark languages and leads monolingual reverse retrieval, achieving Recall@1: 32.1%, Recall@5: 47.0%, and MRR@10: 0.385, slightly ahead of E5-Large-Instruct on that task (Shah et al., 15 Jan 2026). However, it is not the best model on forward persona-to-instruction retrieval or cross-lingual transfer, where E5-Large-Instruct leads (Shah et al., 15 Jan 2026). The paper explicitly suggests that BGE-M3’s multi-vector retrieval architecture may better support reverse-direction matching (Shah et al., 15 Jan 2026).

This asymmetry is important because it indicates that BGE-M3’s embedding geometry is not uniformly optimal across retrieval directions. In monolingual forward retrieval, its averages are Recall@1: 21.6%, Recall@5: 33.8%, and MRR@10: 0.269; in cross-lingual forward retrieval, Recall@1: 14.3% and MRR: 0.187; but in monolingual reverse retrieval it becomes the top system (Shah et al., 15 Jan 2026). This suggests that BGE-M3 may be especially useful where retrieval is not standard query-to-document matching but involves structured or role-reversed compatibility search.

The multilingual coverage also proved useful in finance-domain cross-lingual adaptation. In NMIXX, the public bge-m3 checkpoint was fine-tuned with 18.8K high-confidence triplets composed of in-domain paraphrases, hard negatives derived from a financial semantic-shift typology, and exact Korean-English translations (Lee et al., 13 Jul 2025). The resulting multilingual bge-m3 variant improved from 0.1969 to 0.2967 on FinSTS and from 0.0512 to 0.2732 on KorFinSTS, with corresponding gains of +0.0998 and +0.2220, while general-domain STS scores dropped from 0.8194 to 0.7803 on STS and from 0.7382 to 0.6919 on KorSTS (Lee et al., 13 Jul 2025).

The paper attributes this strong finance adaptation partly to BGE-M3’s tokenizer coverage: vocabulary size 250,002, 5,413 full Korean tokens, and 2.17% Korean token share (Lee et al., 13 Jul 2025). The authors argue that models with sparse Korean token coverage benefit less from bilingual supervision. A plausible implication is that BGE-M3’s multilingual effectiveness in low-resource settings is not merely a consequence of multilingual pretraining but also of tokenizer-level representational adequacy.

5. Use in RAG pipelines, answer selection, and semantic scoring

BGE-M3 has become a frequent component in multilingual RAG architectures. In the Traditional Chinese system built over Chinese Wikipedia and Lawbank, BGE-M3 encodes user queries and candidate knowledge chunks into vectors indexed in FAISS. Chinese Wikipedia contains about 1.38 million entries, with 1,377,100 articles under the 8,192-token limit. Lawbank contains 6,193 legal clauses drawn from 151 regulations, of which 86 were below 3,400 tokens, and 5,450 clauses were under 250 tokens (Yang et al., 8 Jan 2025). The legal materials were segmented into clauses to improve retrieval precision in financial and insurance question answering (Yang et al., 8 Jan 2025).

That system evaluates on TTQA and TMMLU+, using accuracy as the decisive metric for the reported experiments (Yang et al., 8 Jan 2025). The reported TTQA results show consistent improvements when RAG is added: Taiwan-LLM-8x7B-DPO improves from 57.28% to 88.35%, Mistral-8x7B-Instruct-v0-1 from 63.11% to 85.44%, and ChatGPT 3.5 from 74.76% to 88.35% (Yang et al., 8 Jan 2025). On more specialized financial and insurance subsets, Lawbank-based RAG outperforms Wikipedia-based RAG, with gains such as +13.74% for Gemma-7B-it on banking questions and +15.09% on insurance questions (Yang et al., 8 Jan 2025). The paper concludes that BGE-M3 + BGE-reranker is effective, but that the usefulness of retrieval depends strongly on whether the corpus matches the task domain.

The same work also uses BGE-M3 in answer refinement rather than only retrieval. The authors remove answer-format constraints, strip labels $d^y$5, $d^y$6, $d^y$7, $d^y$8 from multiple-choice options, then embed both the LLM-generated answer and each candidate option using BGE-M3. Cosine similarity determines the closest option. Their example yields similarities of 0.973, 0.771, 0.663, and 0.775, where the largest score selects the correct option (Yang et al., 8 Jan 2025). This illustrates a second use case: BGE-M3 as a semantic post-processor for answer normalization and evaluation.

In Khmer RAG, BGE-M3 similarly influences both retrieval and evaluation. After retriever selection, BGE-M3 is fixed as the retriever for all generator experiments, and it is also the embedding function used in the RAGAS-style metrics

$d^y$9

The answer correctness metric is defined as

$y$0

with

$y$1

The paper notes that context relevance scores across generators cluster around 0.72, reinforcing the claim that retriever quality, rather than generator quality, is the main bottleneck (Ros et al., 21 May 2026).

These RAG deployments collectively show that BGE-M3 is frequently used in at least three operational roles: first-stage retrieval, semantic reranking support, and embedding-based answer evaluation. This suggests a broader functional identity than that of a conventional dual-encoder retriever.

6. Domain adaptation, reranking derivatives, and reward modeling

BGE-M3 has also been adapted beyond first-stage retrieval. In Thai legal QA optimized with Group-Relative Policy Optimization (GRPO), BGE-M3 serves as a cost-efficient semantic reward proxy. The authors contrast a BGE-M3-based semantic similarity reward against a judge-based reward using Qwen2.5-72B-Instruct. The BGE-M3 setup required 104 GPU-hours and about \$y$2216, roughly 2.5× more compute time and cost (Akarajaradwong et al., 13 Jul 2025). The reward is a scalar in [0.0, 1.0] computed from semantic similarity between generated and reference answers using BGE-M3 embeddings, and is integrated with citation-focused rewards inside GRPO (Akarajaradwong et al., 13 Jul 2025).

The same Thai legal system also uses BGE-M3 for retrieval with a multi-head strategy and dense / sparse / ColBERT weights of 0.4 / 0.2 / 0.4, retrieving the top 10 legal sections under an 8192-token prompt cap (Akarajaradwong et al., 13 Jul 2025). Empirically, semantic-reward GRPO improves in-domain performance substantially. On Typhoon2 over Nitibench-CCL, semantic reward raises Citation F1 from 0.3597 to 0.6828 and Joint Score from 0.5912 to 0.7773; on OpenThaiGPT1.5, it raises Citation F1 from 0.4299 to 0.7017 and Joint Score from 0.6030 to 0.7595 (Akarajaradwong et al., 13 Jul 2025). The abstract summarizes the overall gains as up to 90% citation-F1 gains from the base model and a 31% increase in joint quality metrics over instruction tuning (Akarajaradwong et al., 13 Jul 2025). Yet the authors also report that on the harder Nitibench-Tax benchmark, the correlation between BGE-M3 semantic similarity and deeper answer quality weakens, so the proxy becomes less reliable for complex legal reasoning (Akarajaradwong et al., 13 Jul 2025).

A separate line of work uses BGE-M3 as the initialization for reranking models. ViRanker is a Vietnamese reranking model built on the BGE-M3 encoder and modified with Rotary Position Encoding (RoPE), Blockwise Parallel Transformer (BPT), a lightweight MLP scoring head, and triplet ranking loss (Dang et al., 11 Sep 2025). Here BGE-M3 is not the final embedding model exposed to retrieval; it is the multilingual semantic foundation for a cross-encoder reranker tailored to Vietnamese syntax, diacritics, and longer passages.

ViRanker is trained on an approximately 8 GB Vietnamese corpus from Vietnamese Wikipedia, Vietnamese GitHub repositories, and Vietnamese books and reports, producing around 3.5 million document chunks and using a hybrid hard-negative pipeline: BM25 retrieval, BGE-M3 embedding similarity reranking, and MMR filtering to select the three most challenging non-relevant passages (Dang et al., 11 Sep 2025). Evaluated on MMARCO-VI, ViRanker achieves NDCG@3 = 0.6815 and MRR@3 = 0.6641, surpassing multilingual BGE rerankers and beating PhoRanker at top ranks while remaining competitive deeper in the ranking (Dang et al., 11 Sep 2025). This line of work suggests that BGE-M3 is not only deployable as-is, but also productive as a pretrained semantic substrate for language-specific rerankers.

7. Comparative assessments, limitations, and practical positioning

Despite its strong foundational results, later comparative studies do not consistently place BGE-M3 at the top of dense retrieval benchmarks. In a benchmark comparing Google Embeddings 2 (GE2) against five open-source models, BGE-M3 is evaluated in dense retrieval mode only and trails both mE5-L and E5-large on zero-shot passage retrieval (Cirillo et al., 22 May 2026). Reported BEIR nDCG@10 values for BGE-M3 are 0.366 on FiQA-2018, 0.294 on NFCorpus, 0.650 on SciFact, with TREC-COVID omitted due to tokenization incompatibility, and an average of 0.437 (Cirillo et al., 22 May 2026). This is below 0.546 for mE5-L and 0.538 for E5-large (Cirillo et al., 22 May 2026).

On the synthetic Italian IT-RAG-Bench, BGE-M3 records R@1 = 0.046, R@5 = 0.253, R@10 = 0.404, MRR = 0.224, and nDCG@10 = 0.238, again below mE5-L at 0.279 and GE2 at 0.282 nDCG@10 (Cirillo et al., 22 May 2026). The same paper reports median CPU latency of 30.9 ms, standard deviation 0.7 ms, and p95 32.1 ms for BGE-M3, placing it in a practical low-latency regime even if it is not the quality leader (Cirillo et al., 22 May 2026). Its chunking ablation reports nDCG@10 at fixed-32 = 0.231, best strategy = fixed-64, and peak nDCG@10 = 0.238, with storage = 12.5 MB (Cirillo et al., 22 May 2026).

These results imply that BGE-M3’s broad functionality may not always translate into best-in-class zero-shot dense retrieval under benchmark conditions focused narrowly on dense passage ranking. The paper explicitly argues that its multi-task design may introduce trade-offs relative to retrieval-specialized E5 models (Cirillo et al., 22 May 2026). This does not negate the original results of (Chen et al., 2024); rather, it indicates that performance depends materially on whether evaluation rewards unified multi-mode flexibility or pure dense retrieval specialization.

Another comparative challenge comes from jina-embeddings-v3. That work presents a compact multilingual embedding model with 570M parameters, 8192-token context, task-specific LoRA adapters, and Matryoshka Representation Learning (Sturua et al., 2024). The paper explicitly contrasts its long-context strategy with BGE-M3’s position-extension approach and reports LongEmbed average = 70.39 for jina-embeddings-v3* versus 56.56 for baai-bge-m3 (Sturua et al., 2024). It also notes that attempts to extend positional encodings “as done in the BGE M3 model” yielded poor performance in its own setup, which the authors attribute to differences in training and pooling (Sturua et al., 2024). This should be read as a comparative result under that paper’s design choices rather than a universal statement about BGE-M3.

Across the literature, several limitations recur. First, retrieval remains a bottleneck even when BGE-M3 is the strongest available option, as in Khmer QA where Hit Rate@3 = 0.285 is explicitly described as modest (Ros et al., 21 May 2026). Second, general-purpose corpora can be too broad for specialized domains, as shown by Wikipedia underperforming Lawbank on financial and insurance questions (Yang et al., 8 Jan 2025). Third, semantic similarity is not always an adequate proxy for correctness; Thai legal reasoning results show that BGE-M3-based semantic reward weakens on more complex tax reasoning (Akarajaradwong et al., 13 Jul 2025). Fourth, reranking can be computationally expensive, and applied RAG papers note this as a deployment constraint even when it improves precision (Yang et al., 8 Jan 2025).

Taken together, these findings position BGE-M3 as a versatile multilingual retrieval backbone rather than a universally dominant embedding model. Its most distinctive contribution is the unification of dense, sparse, and multi-vector retrieval in one model with long-context support and multilingual coverage (Chen et al., 2024). Its most common downstream use is as a dense retriever in two-stage RAG stacks augmented by rerankers (Yang et al., 8 Jan 2025, Sumyk et al., 16 Apr 2026). Its strongest domain-adaptation evidence comes from Korean finance, where tokenizer coverage and bilingual triplet supervision yield large gains on financial STS (Lee et al., 13 Jul 2025). Its most distinctive non-retrieval application is as a low-cost semantic reward proxy for RL-based legal QA (Akarajaradwong et al., 13 Jul 2025). The broader literature therefore suggests that BGE-M3 is best understood not as a single benchmark leader, but as an extensible multilingual embedding platform whose value emerges most clearly in heterogeneous, cross-lingual, or systems-oriented retrieval settings.