---
title: 'BGE-M3 Embeddings: Unified Multilingual Retrieval'
url: https://www.emergentmind.com/topics/bge-m3-embeddings
type: topic
---

# BGE-M3 Embeddings: Unified Multilingual Retrieval

Searching arXiv for recent and foundational papers on BGE-M3 embeddings.
BGE-M3 Embeddings are multilingual text representations introduced as a unified retrieval foundation that combines **multi-linguality**, **multi-functionality**, and **multi-granularity** within a single model family. The original M3-Embedding formulation supports **more than 100 working languages**, three retrieval modes—**dense retrieval**, **sparse retrieval**, and **multi-vector retrieval**—and inputs ranging from short passages to long documents of up to **8192 tokens** [2402.03216]. Subsequent work has used BGE-M3 in diverse roles: as a first-stage dense retriever in Retrieval-Augmented Generation (RAG), as a semantic evaluation model, as a finance-domain embedding backbone, and as a lightweight semantic reward proxy in reinforcement-learning-based legal QA systems [2501.04635], [2605.22099], [2507.09601], [2507.09638]. The resulting literature presents BGE-M3 not as a single narrowly optimized retriever, but as a general-purpose multilingual embedding infrastructure whose utility depends on task regime, retrieval direction, corpus structure, and domain adaptation.

## 1. Foundational model design

BGE M3-Embedding is defined in terms of three explicit capabilities. **Multi-linguality** refers to support for multilingual and cross-lingual retrieval across more than 100 languages. **Multi-functionality** refers to a single encoder supporting **dense retrieval**, **sparse / lexical retrieval**, and **multi-vector retrieval**. **Multi-granularity** refers to processing inputs from short sentences to long documents up to **8192 tokens** [2402.03216].

The original paper formulates the model as a unified retriever capable of mapping a query \(q^x\) in language \(x\) to a relevant document \(d^y\) in language \(y\), whether retrieval is monolingual or cross-lingual and whether scoring uses dense, sparse, or late-interaction mechanisms [2402.03216]. This matters because the model is not framed as merely a multilingual sentence embedder. It is instead presented as a retrieval system with multiple scoring heads sharing a common encoder.

The dense mode uses normalized [CLS]-based query and passage embeddings,
\[
e_q = norm(\mathbf{H_q}[0]), \qquad e_p = norm(\mathbf{H_p}[0]),
\]
with dense relevance score
\[
s_{dense} \leftarrow \langle e_p, e_q \rangle.
\]
The sparse mode produces token-level lexical weights,
\[
w_{q_t} \leftarrow \mathsf{ReLU}\left(\mathbf{W}_{lex}^T \mathbf{H_q}[i]\right),
\]
and then scores overlapping terms by
\[
s_{lex} \leftarrow \sum_{t\in q \cap p} (w_{q_t} * w_{p_t}).
\]
The multi-vector mode projects token representations,
\[
E_q = norm(\mathbf{W}_{mul}^T\mathbf{H_q}), \qquad E_p = norm(\mathbf{W}_{mul}^T\mathbf{H_p}),
\]
and applies a ColBERT-style max-sim aggregation,
\[
s_{mul} \leftarrow \frac{1}{N} \sum_{i=1}^{N} \max_{j=1}^{M} E_q[i] \cdot E_p^{T}[j].
\]
The paper also defines a hybrid ranking score,
\[
s_{rank} \leftarrow s_{dense} + s_{lex} + s_{mul}.
\]
These formulas are central because they show that BGE-M3 is not limited to a single embedding geometry or retrieval semantics [2402.03216].

A common misconception is that BGE-M3 should be understood purely as a dense embedding model. The foundational paper explicitly defines it as a model that can simultaneously support dense, sparse, and multi-vector retrieval. Many downstream papers, however, use only the dense component for practical reasons such as indexing simplicity or latency constraints. This suggests that discussions of “BGE-M3 embeddings” often refer to a reduced deployment configuration rather than the full model design [2402.03216], [2501.04635], [2605.23618].

## 2. Training objective and self-knowledge distillation

The key training innovation in BGE-M3 is **self-knowledge distillation (SKD)**. The base retrieval loss for each mode is InfoNCE:
\[
\mathcal{L} = - \log \frac{ \exp(s(q,p^*) / \tau) }{\sum_{p \in \{p^*, P'\} \exp(s(q,p) / \tau) }}.
\]
Here \(p^*\) is the positive passage, \(P'\) the negatives, and \(s(\cdot)\) one of the mode-specific scores \(s_{dense}\), \(s_{lex}\), or \(s_{mul}\) [2402.03216].

The distinctive step is that the model constructs an integrated teacher score
\[
s_{inter} \leftarrow s_{dense} + s_{lex} + s_{mul},
\]
then encourages each retrieval head to align with the soft distribution induced by this integrated signal:
\[
\mathcal{L}'_* \leftarrow - p(s_{inter}) * \log p(s_{*}),
\]
with the averaged distillation loss
\[
\mathcal{L}' \leftarrow \big( \mathcal{L}'_{dense}+\mathcal{L}'_{lex}+\mathcal{L}'_{mul}\big) / 3,
\]
and final objective
\[
\mathcal{L}_{final} \leftarrow \mathcal{L} + \mathcal{L}'.
\]
The paper frames this as an ensemble-like teacher mechanism allowing the dense, sparse, and multi-vector views to regularize one another [2402.03216].

This design is significant because multi-objective retrieval training often induces interference across heads. BGE-M3’s SKD scheme treats the integrated multi-mode score as a supervisory prior rather than training the heads independently. A plausible implication is that the model’s empirical robustness in downstream multilingual settings partly derives from this joint calibration of semantic, lexical, and late-interaction signals.

The training pipeline is multi-stage. The base encoder is **XLM-RoBERTa further pre-trained with RetroMAE**; the model then undergoes unsupervised dense retrieval pretraining on about **1.2B text pairs**, followed by multi-head fine-tuning on English, Chinese, multilingual, and synthetic long-document retrieval data [2402.03216]. The paper reports **1.1M** English examples, **386.6K** Chinese examples, **88.9K** multilingual examples, and **41.4K** synthetic long-document examples for fine-tuning [2402.03216].

The long-input regime imposes substantial systems constraints. The implementation uses a batching strategy with length grouping, fixed random seeds across GPUs, and **split-batch** training. The reported effect is large: at length **8192**, maximum batch size per device rises from **6 to 130**, and the paper states that for **8192-token** text, batch size can increase by **more than 20×** [2402.03216]. This is part of why BGE-M3 could be trained as a long-document retriever rather than only evaluated as one.

## 3. Retrieval modes, long-context handling, and benchmark behavior

The foundational evaluation establishes BGE-M3 as a strong multilingual retriever across multilingual, cross-lingual, and long-document settings. On **MIRACL** dev, average nDCG@10 is reported as **67.8** for **M3 Dense**, **69.0** for **M3 Multi-vec**, **68.9** for **M3 Dense+Sparse**, and **70.0** for **M3 All**, compared with **65.4** for **mE5-large** and **62.2** for **E5-mistral-7B** [2402.03216]. On **MKQA** Recall@100, **M3 All** reaches **75.5**, above **70.9** for **mE5-large** and **70.1** for **E5-mistral-7B** [2402.03216]. On long-document retrieval benchmarks, BGE-M3 is especially strong in hybrid mode: on **MLDR** nDCG@10, **M3 All** scores **65.0**, and on **NarrativeQA** it scores **61.7** [2402.03216].

Long-context support is a defining technical property. The model handles up to **8192 tokens**, and the paper introduces **MCLS (Multiple CLS)** as an inference-time technique that inserts a CLS token every 256 tokens and averages the resulting states to better summarize long inputs when long-document fine-tuning resources are limited. In the ablation setting, MCLS improves long-document retrieval on MLDR from **41.2 to 45.0** nDCG@10 [2402.03216].

Downstream RAG work often exploits this long-input capacity directly. A Traditional Chinese knowledge-retrieval system built on Chinese Wikipedia and Lawbank uses BGE-M3 as its dense retriever and notes that Chinese Wikipedia articles were mostly short enough to fit within the **8192-token** limit without chunking, while Lawbank’s legal materials were segmented into **individual legal clauses** because of their specialized structure and variable length [2501.04635]. That paper explicitly treats BGE-M3 as the embedding backbone in the pipeline
**query → BGE-M3 embedding → FAISS dense retrieval → BGE-reranker re-ranking → LLM generation** [2501.04635].

The same paper describes FAISS as the vector index supporting options such as **Flat**, **IVF**, and **HNSW**, with similarity measures including **L2 distance, dot product, and cosine similarity**. In the deployed system, BGE-M3 is used for dense vector search over Chinese Wikipedia and Lawbank, and BGE-reranker, a **cross-encoder**, reorders the dense retrieval candidates before they are passed to the LLM [2501.04635]. The paper argues that BGE-M3 is strong at semantic matching, but that semantic closeness alone is not always enough for precise question answering, so reranking is used to optimize for answer relevance rather than topical similarity [2501.04635].

A similar two-stage architecture appears in Ukrainian document QA. There BGE-M3 is used as the main dense retriever, followed by **BGE reranker-v2-m3**, and the best development retrieval result is achieved by **bge-m3 + bge reranker** with **Mean \(d_i\) = 0.9219** and **Mean \(p_i\) = 0.8111**, compared with **0.9002 / 0.7365** for BGE-M3 alone [2604.14896]. The paper interprets retrieval quality as the primary bottleneck and treats reranking as a precision-improving stage layered over BGE-M3’s recall-oriented candidate generation [2604.14896].

These findings support a recurring pattern: BGE-M3 is often used as the **first-stage retriever**, while a cross-encoder or reranker supplies precision. This does not contradict the original multi-mode design. Rather, it suggests that many applied systems prefer to externalize precision optimization into a dedicated reranking stage instead of relying on the model’s built-in hybrid scoring.

## 4. Multilingual deployment across low-resource and cross-lingual settings

A notable portion of the BGE-M3 literature concerns low-resource or morphologically challenging languages. In Khmer telecom-domain QA, BGE-M3 was compared against **Jina-Embeddings-v3 (570M)** and **Qwen3-Embedding (597M)** on a **golden dataset of 200 Khmer question-answer pairs** derived from official telecom and ICT documents. BGE-M3 was selected as the final retriever because it achieved the strongest dense retrieval results at the deployed top-\(k\) setting, **k = 3**: **Hit Rate@3 = 0.285**, **File Hit Rate@3 = 0.700**, **MRR@3 = 0.221**, and **Precision@3 = 0.112** [2605.22099].

The paper highlights an instructive exception: **Jina-Embeddings-v3** had the **highest top-1 cosine similarity** at **0.759**, above **0.704** for BGE-M3 and **0.686** for Qwen3-Embedding, but this did not translate into better retrieval accuracy [2605.22099]. The authors therefore argue that raw cosine similarity is not a reliable proxy for retrieval quality in Khmer, particularly under non-Latin script, segmentation difficulty, OCR noise, and mixed Khmer-English terminology [2605.22099]. Their interpretation is that BGE-M3 better separates truly relevant passages from merely semantically adjacent ones.

In the Indic persona-instruction alignment benchmark spanning **12 Indian languages** and four tasks, BGE-M3 again displays mixed but substantial multilingual competence. It supports all 12 benchmark languages and leads **monolingual reverse retrieval**, achieving **Recall@1: 32.1%**, **Recall@5: 47.0%**, and **MRR@10: 0.385**, slightly ahead of **E5-Large-Instruct** on that task [2601.10205]. However, it is not the best model on forward persona-to-instruction retrieval or cross-lingual transfer, where E5-Large-Instruct leads [2601.10205]. The paper explicitly suggests that BGE-M3’s **multi-vector retrieval architecture** may better support reverse-direction matching [2601.10205].

This asymmetry is important because it indicates that BGE-M3’s embedding geometry is not uniformly optimal across retrieval directions. In monolingual forward retrieval, its averages are **Recall@1: 21.6%**, **Recall@5: 33.8%**, and **MRR@10: 0.269**; in cross-lingual forward retrieval, **Recall@1: 14.3%** and **MRR: 0.187**; but in monolingual reverse retrieval it becomes the top system [2601.10205]. This suggests that BGE-M3 may be especially useful where retrieval is not standard query-to-document matching but involves structured or role-reversed compatibility search.

The multilingual coverage also proved useful in finance-domain cross-lingual adaptation. In NMIXX, the public **bge-m3** checkpoint was fine-tuned with **18.8K high-confidence triplets** composed of in-domain paraphrases, hard negatives derived from a financial semantic-shift typology, and exact Korean-English translations [2507.09601]. The resulting multilingual bge-m3 variant improved from **0.1969 to 0.2967** on **FinSTS** and from **0.0512 to 0.2732** on **KorFinSTS**, with corresponding gains of **+0.0998** and **+0.2220**, while general-domain STS scores dropped from **0.8194 to 0.7803** on **STS** and from **0.7382 to 0.6919** on **KorSTS** [2507.09601].

The paper attributes this strong finance adaptation partly to BGE-M3’s tokenizer coverage: vocabulary size **250,002**, **5,413** full Korean tokens, and **2.17%** Korean token share [2507.09601]. The authors argue that models with sparse Korean token coverage benefit less from bilingual supervision. A plausible implication is that BGE-M3’s multilingual effectiveness in low-resource settings is not merely a consequence of multilingual pretraining but also of tokenizer-level representational adequacy.

## 5. Use in RAG pipelines, answer selection, and semantic scoring

BGE-M3 has become a frequent component in multilingual RAG architectures. In the Traditional Chinese system built over **Chinese Wikipedia** and **Lawbank**, BGE-M3 encodes user queries and candidate knowledge chunks into vectors indexed in FAISS. Chinese Wikipedia contains about **1.38 million entries**, with **1,377,100 articles** under the **8,192-token** limit. Lawbank contains **6,193 legal clauses** drawn from **151 regulations**, of which **86 were below 3,400 tokens**, and **5,450 clauses** were under **250 tokens** [2501.04635]. The legal materials were segmented into clauses to improve retrieval precision in financial and insurance question answering [2501.04635].

That system evaluates on **TTQA** and **TMMLU+**, using **accuracy** as the decisive metric for the reported experiments [2501.04635]. The reported TTQA results show consistent improvements when RAG is added: **Taiwan-LLM-8x7B-DPO** improves from **57.28% to 88.35%**, **Mistral-8x7B-Instruct-v0-1** from **63.11% to 85.44%**, and **ChatGPT 3.5** from **74.76% to 88.35%** [2501.04635]. On more specialized financial and insurance subsets, **Lawbank-based RAG** outperforms **Wikipedia-based RAG**, with gains such as **+13.74%** for **Gemma-7B-it** on banking questions and **+15.09%** on insurance questions [2501.04635]. The paper concludes that BGE-M3 + BGE-reranker is effective, but that the usefulness of retrieval depends strongly on whether the corpus matches the task domain.

The same work also uses BGE-M3 in answer refinement rather than only retrieval. The authors remove answer-format constraints, strip labels \((A)\), \((B)\), \((C)\), \((D)\) from multiple-choice options, then embed both the LLM-generated answer and each candidate option using BGE-M3. **Cosine similarity** determines the closest option. Their example yields similarities of **0.973, 0.771, 0.663, and 0.775**, where the largest score selects the correct option [2501.04635]. This illustrates a second use case: BGE-M3 as a semantic post-processor for answer normalization and evaluation.

In Khmer RAG, BGE-M3 similarly influences both retrieval and evaluation. After retriever selection, BGE-M3 is fixed as the retriever for all generator experiments, and it is also the embedding function used in the RAGAS-style metrics
\[
\text{ConRel} = \mathrm{sim}(E(q), E(c(q))), \qquad \text{AnsSim} = \mathrm{sim}(E(a(q)), E(gt(q))).
\]
The answer correctness metric is defined as
\[
\text{AnsCor} = w_1 \times \text{FacCor} + w_2 \times \text{AnsSim},
\]
with
\[
[w_1, w_2] = [0.5, 0.5].
\]
The paper notes that context relevance scores across generators cluster around **0.72**, reinforcing the claim that retriever quality, rather than generator quality, is the main bottleneck [2605.22099].

These RAG deployments collectively show that BGE-M3 is frequently used in at least three operational roles: first-stage retrieval, semantic reranking support, and embedding-based answer evaluation. This suggests a broader functional identity than that of a conventional dual-encoder retriever.

## 6. Domain adaptation, reranking derivatives, and reward modeling

BGE-M3 has also been adapted beyond first-stage retrieval. In Thai legal QA optimized with **Group-Relative Policy Optimization (GRPO)**, BGE-M3 serves as a **cost-efficient semantic reward proxy**. The authors contrast a BGE-M3-based semantic similarity reward against a judge-based reward using **Qwen2.5-72B-Instruct**. The BGE-M3 setup required **104 GPU-hours** and about **\$85**, whereas the judge setup required **264 GPU-hours** and about **\$216**, roughly **2.5×** more compute time and cost [2507.09638]. The reward is a scalar in **[0.0, 1.0]** computed from semantic similarity between generated and reference answers using BGE-M3 embeddings, and is integrated with citation-focused rewards inside GRPO [2507.09638].

The same Thai legal system also uses BGE-M3 for retrieval with a **multi-head strategy** and **dense / sparse / ColBERT weights of 0.4 / 0.2 / 0.4**, retrieving the **top 10 legal sections** under an **8192-token** prompt cap [2507.09638]. Empirically, semantic-reward GRPO improves in-domain performance substantially. On **Typhoon2** over **Nitibench-CCL**, semantic reward raises **Citation F1** from **0.3597 to 0.6828** and **Joint Score** from **0.5912 to 0.7773**; on **OpenThaiGPT1.5**, it raises **Citation F1** from **0.4299 to 0.7017** and **Joint Score** from **0.6030 to 0.7595** [2507.09638]. The abstract summarizes the overall gains as up to **90% citation-F1 gains from the base model** and a **31% increase in joint quality metrics over instruction tuning** [2507.09638]. Yet the authors also report that on the harder **Nitibench-Tax** benchmark, the correlation between BGE-M3 semantic similarity and deeper answer quality weakens, so the proxy becomes less reliable for complex legal reasoning [2507.09638].

A separate line of work uses BGE-M3 as the initialization for reranking models. **ViRanker** is a Vietnamese reranking model built on the BGE-M3 encoder and modified with **Rotary Position Encoding (RoPE)**, **Blockwise Parallel Transformer (BPT)**, a lightweight **MLP scoring head**, and **triplet ranking loss** [2509.09131]. Here BGE-M3 is not the final embedding model exposed to retrieval; it is the multilingual semantic foundation for a cross-encoder reranker tailored to Vietnamese syntax, diacritics, and longer passages.

ViRanker is trained on an approximately **8 GB Vietnamese corpus** from Vietnamese Wikipedia, Vietnamese GitHub repositories, and Vietnamese books and reports, producing around **3.5 million document chunks** and using a hybrid hard-negative pipeline: **BM25 retrieval**, **BGE-M3 embedding similarity** reranking, and **MMR filtering** to select **the three most challenging non-relevant passages** [2509.09131]. Evaluated on **MMARCO-VI**, ViRanker achieves **NDCG@3 = 0.6815** and **MRR@3 = 0.6641**, surpassing multilingual BGE rerankers and beating PhoRanker at top ranks while remaining competitive deeper in the ranking [2509.09131]. This line of work suggests that BGE-M3 is not only deployable as-is, but also productive as a pretrained semantic substrate for language-specific rerankers.

## 7. Comparative assessments, limitations, and practical positioning

Despite its strong foundational results, later comparative studies do not consistently place BGE-M3 at the top of dense retrieval benchmarks. In a benchmark comparing **Google Embeddings 2 (GE2)** against five open-source models, BGE-M3 is evaluated in **dense retrieval mode only** and trails both **mE5-L** and **E5-large** on zero-shot passage retrieval [2605.23618]. Reported BEIR nDCG@10 values for BGE-M3 are **0.366** on **FiQA-2018**, **0.294** on **NFCorpus**, **0.650** on **SciFact**, with **TREC-COVID** omitted due to tokenization incompatibility, and an **average of 0.437** [2605.23618]. This is below **0.546** for **mE5-L** and **0.538** for **E5-large** [2605.23618].

On the synthetic Italian **IT-RAG-Bench**, BGE-M3 records **R@1 = 0.046**, **R@5 = 0.253**, **R@10 = 0.404**, **MRR = 0.224**, and **nDCG@10 = 0.238**, again below **mE5-L** at **0.279** and **GE2** at **0.282** nDCG@10 [2605.23618]. The same paper reports **median CPU latency of 30.9 ms**, **standard deviation 0.7 ms**, and **p95 32.1 ms** for BGE-M3, placing it in a practical low-latency regime even if it is not the quality leader [2605.23618]. Its chunking ablation reports **nDCG@10 at fixed-32 = 0.231**, **best strategy = fixed-64**, and **peak nDCG@10 = 0.238**, with **storage = 12.5 MB** [2605.23618].

These results imply that BGE-M3’s broad functionality may not always translate into best-in-class zero-shot dense retrieval under benchmark conditions focused narrowly on dense passage ranking. The paper explicitly argues that its multi-task design may introduce trade-offs relative to retrieval-specialized E5 models [2605.23618]. This does not negate the original results of [2402.03216]; rather, it indicates that performance depends materially on whether evaluation rewards unified multi-mode flexibility or pure dense retrieval specialization.

Another comparative challenge comes from **jina-embeddings-v3**. That work presents a compact multilingual embedding model with **570M** parameters, **8192-token** context, task-specific LoRA adapters, and **Matryoshka Representation Learning** [2409.10173]. The paper explicitly contrasts its long-context strategy with BGE-M3’s position-extension approach and reports **LongEmbed average = 70.39** for **jina-embeddings-v3*** versus **56.56** for **baai-bge-m3** [2409.10173]. It also notes that attempts to extend positional encodings “as done in the BGE M3 model” yielded poor performance in its own setup, which the authors attribute to differences in training and pooling [2409.10173]. This should be read as a comparative result under that paper’s design choices rather than a universal statement about BGE-M3.

Across the literature, several limitations recur. First, **retrieval remains a bottleneck** even when BGE-M3 is the strongest available option, as in Khmer QA where **Hit Rate@3 = 0.285** is explicitly described as modest [2605.22099]. Second, **general-purpose corpora can be too broad** for specialized domains, as shown by Wikipedia underperforming Lawbank on financial and insurance questions [2501.04635]. Third, **semantic similarity is not always an adequate proxy for correctness**; Thai legal reasoning results show that BGE-M3-based semantic reward weakens on more complex tax reasoning [2507.09638]. Fourth, **reranking can be computationally expensive**, and applied RAG papers note this as a deployment constraint even when it improves precision [2501.04635].

Taken together, these findings position BGE-M3 as a versatile multilingual retrieval backbone rather than a universally dominant embedding model. Its most distinctive contribution is the unification of dense, sparse, and multi-vector retrieval in one model with long-context support and multilingual coverage [2402.03216]. Its most common downstream use is as a dense retriever in two-stage RAG stacks augmented by rerankers [2501.04635], [2604.14896]. Its strongest domain-adaptation evidence comes from Korean finance, where tokenizer coverage and bilingual triplet supervision yield large gains on financial STS [2507.09601]. Its most distinctive non-retrieval application is as a low-cost semantic reward proxy for RL-based legal QA [2507.09638]. The broader literature therefore suggests that BGE-M3 is best understood not as a single benchmark leader, but as an extensible multilingual embedding platform whose value emerges most clearly in heterogeneous, cross-lingual, or systems-oriented retrieval settings.

Source: https://www.emergentmind.com/topics/bge-m3-embeddings