mRAKL: Multilingual KG Construction

• mRAKL is a multilingual, retrieval-augmented generation system that reformulates KG triples as question-answer pairs for low-resource languages.
• It integrates sparse (BM25) and dense (LaBSE) retrieval methods with a fine-tuned AfriTeVa-base generator to predict missing entities.
• Empirical results demonstrate improved hit rates for Tigrinya and Amharic KGs through effective cross-lingual transfer from Arabic and English.

Searching arXiv for the specified paper and closely related work to ground the article. mRAKL is a multilingual, retrieval-augmented generation system for multilingual knowledge graph construction in low-resourced languages. It reformulates multilingual knowledge graph construction as a question answering task, using the head entity and linking relation to form a question and predicting the tail entity as the answer. The system is designed to complete and enrich knowledge graphs in settings where both structured data and high-quality pretrained models are scarce, with experiments centered on Tigrinya and Amharic and cross-lingual transfer from Arabic and English (Nigatu et al., 21 Jul 2025).

1. Definition and task reformulation

Multilingual knowledge graph construction refers to the task of automatically constructing or predicting missing entities and links in multiple languages, with special emphasis on low-resourced languages. In the formulation used by mRAKL, a knowledge graph is a set of triples

$$\mathcal{G} = \{(h, r, t) \mid h, t \in \mathcal{E}, r \in \mathcal{R}\},$$

where $h$ is the head entity, $r$ is a relation or property, and $t$ is the tail entity (Nigatu et al., 21 Jul 2025).

The central methodological move is to verbalize each triple $(h, r, t)$ into a question answering instance. The question is constructed from the head $h$ and relation $r$ using a relation-specific template, and the answer is the tail $t$. In this setting, the model predicts the tail given the head and relation, optionally augmented with retrieved textual context:

$\hat{t} = f(h, r, C),$

where $C$ denotes retrieved context. Training uses a standard conditional generation objective,

$h$0

This reformulation addresses two constraints identified for low-resourced languages. First, it allows a generator to use evidence from text rather than relying only on structured graph neighborhoods. Second, it exploits unstructured, monolingual corpora that are easier to obtain than labeled KG data. Cross-lingual transfer from Arabic and English is used to leverage richer resources and culturally relevant information.

2. System architecture

mRAKL consists of a retriever and a generator in a retrieval-augmented generation pipeline (Nigatu et al., 21 Jul 2025). The retriever fetches relevant passages $h$1 given the question formed from $h$2 and $h$3, and the generator predicts the tail entity conditioned on the question and retrieved context.

The retrieval component operates over monolingual Wikipedia articles using the 2024-07-01 snapshot from the Hugging Face dataset “olm/wikipedia.” Monolingual indexes are built for Tigrinya, Amharic, Arabic, and English by indexing the full Wikipedia articles of all head entities. Three retrieval modes are described. The first is BM25 with bm25s default settings and the standard scoring function

$h$4

The second is a LaBSE sentence embedder fine-tuned with contrastive loss using positive contexts harvested by a heuristic retriever, plus hard, head-only, and relation-only negatives. The third is an “(im)perfect retriever,” a heuristic upper bound that searches the head entity’s Wikipedia first paragraph for sentences containing the tail; it yields high-precision context when the tail is present, but it is not realistic because it uses the ground-truth tail to retrieve.

The generator is AfriTeVa-base, a seq2seq multilingual LLM, fine-tuned with LoRA for parameter-efficient adaptation. Its input format is a composed sequence with language tags and special markers: [C-LAN] C | [Q-LAN] Q ? [A-LAN], where LAN ∈ {tir, amh, eng, ara} indicates language, ? ends the question, | marks the end of context, and [A-LAN] specifies the language in which the answer should be generated. Decoding uses beam search with beam size 10, and evaluation examines top-$h$5 outputs for $h$6. Generator training uses cross-entropy over the tail text, with no explicit negative sampling.

A notable implementation detail is that constrained decoding attempts were made, but final scoring counts hits that include the tail string. This is significant because it situates mRAKL between open-form generation and entity prediction over a predefined vocabulary.

3. Multilingual setting and data construction

The target languages are Tigrinya and Amharic, both written in Ge’ez script. The transfer languages are Arabic and English. Arabic is selected because it is linguistically related through the Afro-Semitic family and culturally and regionally close, whereas English provides abundant resources (Nigatu et al., 21 Jul 2025).

Cross-lingual transfer is enabled by multilingual pretrained models, shared subword vocabularies, cross-lingual retrieval, and entity alignment through Wikidata labels. Empirically, 86.47% of tail entities in the Tigrinya KG have Amharic labels, and approximately 35.88% of Amharic and Tigrinya tails share identical spellings. This suggests that shared surface forms and multilingual label coverage are an important source of transfer signal.

The KG and QA datasets are derived from Wikidata. The construction process starts from the Wikidata Property Explorer. The resource contains 96 Tigrinya relations plus 24 manually translated from Amharic, yielding 120 total relations with textual representations across Tigrinya, Amharic, Arabic, and English. The Tigrinya KG is extracted from Wikidata for entities with Tigrinya Wikipedia pages, filtered to those 120 relations, resulting in approximately 3.5k triples and 272 unique entities. The Amharic KG contains approximately 34k triples. The same extraction pipeline is applied across languages.

Preprocessing is language-aware. Entity surface forms from Wikidata labels are used consistently across languages. Amharic and Tigrinya question templates are gender-aware as needed, and templates are manually curated for correctness in Ge’ez script. Relation labels are manually extended to ensure coverage, including 24 translations from Amharic to Tigrinya and 68 from Tigrinya to Amharic.

4. Training pipeline and evaluation protocol

The end-to-end workflow comprises KG extraction, template-based reformulation, multilingual corpus preparation, retrieval, generator fine-tuning, and inference (Nigatu et al., 21 Jul 2025). Triples are extracted from Wikidata for entities with target-language Wikipedia pages and filtered to the 120 relations. Multilingual labels for head, relation, and tail are obtained from Wikidata. Each triple is converted to a QA pair using relation-specific templates, after which the data are split into train, validation, and test with an 8:1:1 ratio. The evaluation set is used for hyperparameter tuning, and results are reported on held-out test data.

Generator variants are trained under no-context, monolingual self-context, multilingual self-context, and cross-lingual context conditions. The retriever can be BM25, LaBSE, or the heuristic upper-bound retriever. For LaBSE, training uses contrastive learning with anchors equal to the question, positives drawn from heuristic contexts containing tails, and negatives consisting of hard, head-only, and relation-only examples. Reproducibility details include LaBSE training for 50 epochs with learning rate $h$7 and warmup over 15% of steps, implemented via Sentence-Transformers. The generator is AfriTeVa-base with 428M parameters, fine-tuned with LoRA using learning rate $h$8, training batch size 15 or 30, LoRA rank 4, dropout 0.01, and compute on two Titan RTX GPUs. Baseline comparisons also include mT5-base with 580M parameters.

The principal metric is Hit@$h$9 for exact matches of the tail string among the top-$r$0 generator outputs:

$r$1

Zero-shot probing sometimes uses a contains-tail criterion instead. Reported “accuracy improvements” correspond to H@1 percentage point gains.

The baselines include no-context seq2seq verbalization in a KGT5-style format, KGT5-Description with appended Wikipedia entity descriptions, KGT5-One-Hop with appended one-hop neighbors, no-context multilingual QA setups with mT5 and AfriTeVa, and zero-shot LLM probing with GPT-4o and Aya.

5. Empirical results

Zero-shot parametric knowledge probing is weak in both low-resourced languages (Nigatu et al., 21 Jul 2025). On Tigrinya test QA questions, H@1 is 0.22 for AfriTeVa, 0.67 for Aya, and 2.23 for GPT-4o; mT5 is omitted because the unseen language produced non-meaningful outputs. On Amharic, H@1 is 0.49 for mT5, 0.61 for AfriTeVa, 1.52 for Aya, and 5.83 for GPT-4o. After fine-tuning without context, performance rises to 2.01 for mT5 and 5.13 for AfriTeVa in Tigrinya, and to 23.32 for mT5 and 29.15 for AfriTeVa in Amharic. The strongest seq2seq baseline after fine-tuning is AfriTeVa.

In monolingual closed versus open domain link prediction, retrieval-augmented self-context improves H@1 and H@10 over the no-context generator. KGT5-No-Context obtains 6.91 / 28.57 on Tigrinya and 32.58 / 52.57 on Amharic. KGT5-Description obtains 5.80 / 23.44 and 32.91 / 43.32. KGT5-One-Hop obtains 4.46 / 24.33 and 28.83 / 48.17. mRAKL No-Context obtains 5.13 / 26.11 and 29.15 / 54.81. mRAKL Self-Context reaches 11.83 / 34.59 on Tigrinya and 41.37 / 61.87 on Amharic. Relative to the best KGT5 baseline, self-context improves H@1 by 4.92 percentage points for Tigrinya and 8.79 percentage points for Amharic. The reported explanation is that descriptions and one-hop neighbors rarely contain the tail and coverage is limited.

In cross-lingual link prediction, BM25 outperforms LaBSE on average. For Tigrinya, average H@1 across context languages is 12.97 for no-context, 12.30 for LaBSE, and 14.15 for BM25. Example BM25 results are 15.75 with Amharic context, 12.30 with Arabic context, 14.73 with English context, and 13.84 with Tigrinya context. For Amharic, average H@1 is 33.12 for no-context, 32.19 for LaBSE, and 35.27 for BM25, with example BM25 results of 38.52 using Amharic context, 33.58 using Arabic context, 38.22 using English context, and 11.17 using Tigrinya context.

With the heuristic upper-bound retriever, multilingual self-context outperforms monolingual self-context. For Tigrinya, average H@1 increases from 11.83 to 15.18, a gain of 4.69 points. For Amharic, it increases from 42.92 to 43.21. For the Tigrinya KG, Arabic and English each supply approximately 25% of contexts, whereas Amharic and Tigrinya provide less than 10%. H@1 is highest when Amharic provides context, which is attributed to the 86.47% tail-label coverage and the 35.88% shared spellings with Tigrinya tails.

6. Error modes, interpretation, and significance

The reported qualitative analyses identify several recurrent failure modes (Nigatu et al., 21 Jul 2025). Retrieval misses occur when the head article’s introductory paragraph does not mention the tail or when BM25 and LaBSE retrieve off-topic passages. LaBSE underperforms when positive pairs are sparse. Entity disambiguation is difficult for many-to-many relations such as “diplomatic relations” and “neighboring country.” Generation errors arise because the decoder is unconstrained and may produce paraphrases or extra tokens. Cross-lingual alignment also has limits when transfer-language KGs omit entities specific to the target language, such as Eritrea-specific regions in the Tigrinya KG.

Two examples illustrate the contrast between successful and failed retrieval. For the query “What is Blue an instance of?”, Arabic context refers to flag colors, and the generator correctly predicts “color” in Arabic, Amharic, and English even though the Amharic and English contexts do not explicitly contain the tail. This suggests effective cross-lingual entity alignment during generation. For “What is Porto-Novo a capital of?”, BM25 and the heuristic retriever return context containing the tail “Benin,” whereas LaBSE retrieves only head-related content and fails, highlighting the sensitivity of the overall system to retriever quality.

A common misconception would be to interpret the heuristic “(im)perfect retriever” as a deployable component. The paper explicitly characterizes it as an upper bound because it uses the ground-truth tail to retrieve. Its value is diagnostic: it estimates achievable performance when the correct tail is present in retrieved sentences and supplies positive pairs for LaBSE fine-tuning. When context exists, it substantially outperforms LaBSE, for example on Tigrinya-context retrieval where H@1 is 78.05 for the heuristic retriever and 43.90 for LaBSE. At the same time, it cannot return context if the tail is absent from the head’s lead paragraph.

The broader significance of mRAKL lies in its argument that low-resource multilingual KG construction can benefit from open-domain textual evidence rather than depending exclusively on sparse graph structure or weak parametric knowledge. Unstructured monolingual corpora are more available than labeled KGs, and cross-lingual retrieval can supply culturally relevant and region-specific context from Arabic as well as abundant knowledge from English. A plausible implication is that the method’s modularity makes retriever quality, corpus coverage, and multilingual entity normalization the primary levers for future improvement. The stated future directions are explicit multilingual entity linking, constrained decoding over the KG’s entity vocabulary, improved dense cross-lingual encoders including Tigrinya, better negatives, hybrid sparse+dense retrieval, automated template and label creation, extension to more low-resourced languages and scripts, and knowledge-aware training or KG-adapters to stabilize decoding.