VecMap: Unsupervised Cross-Lingual Mapping

• VecMap is a framework for learning cross-lingual mappings of word embeddings that aligns independent monolingual spaces.
• It leverages normalization, unsupervised initialization, iterative self-learning, and symmetric refinement to create shared representation spaces.
• Empirical evaluations highlight VecMap's robustness in bilingual lexicon induction, low-resource transfer, and multilingual pretraining.

Searching arXiv for recent and foundational papers on VecMap and related evaluations/applications. VecMap is a framework for learning cross-lingual mappings of word embeddings by aligning independently trained monolingual spaces into a shared representation space. It was introduced as “a robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings,” with a fully unsupervised initialization, a robust self-learning algorithm, and symmetric refinement, and its implementation was released as an open source project at https://github.com/artetxem/vecmap (Artetxe et al., 2018). In subsequent work, VecMap became both a strong unsupervised baseline and a reusable alignment component in bilingual lexicon induction, multilingual pretraining, contextual embedding alignment, and low-resource transfer (Glavas et al., 2019).

1. Origins and formal problem setting

VecMap addresses the problem of inducing cross-lingual word embeddings from separately trained monolingual embeddings without requiring parallel corpora, bilingual dictionaries, shared numerals, or string-level cues. Its central assumption is that independently trained embedding spaces preserve enough structural similarity for translation equivalents to be recoverable through a mapping constrained by orthogonality and refined by self-learning (Artetxe et al., 2018).

In one common formulation, given monolingual embedding matrices $X$ and $Z$, VecMap seeks an orthogonal mapping such that

$W = \arg\min_{W \in O(d)} \|WX - Z\|_F ,$

where $O(d)$ denotes the group of orthogonal matrices and $\|\cdot\|_F$ is the Frobenius norm (Makgatho et al., 2021). In the fully unsupervised formulation emphasized in the original work, the optimization is written symmetrically over two orthogonal matrices: $$\max_{W_X, W_Z} \sum_i \sum_j D_{ij} \left((X_{i*}W_X)\cdot(Z_{j*}W_Z)\right),$$ with the optimum obtained from the SVD

$$USV^\top = X^\top D Z,\qquad W_X = U,\qquad W_Z = V .$$

This symmetric treatment is one of the method’s characteristic design choices (Artetxe et al., 2018).

The framework is usually discussed within the projection-based cross-lingual word embedding literature, but later work extended it beyond static embeddings. It was applied to contextual ELMo embeddings through context-aligned anchor points (Ulčar et al., 2021), used to align static embeddings extracted from XLM-R across 40 languages (Hämmerl et al., 2022), and integrated into multilingual BERT pretraining by replacing masked tokens with cross-lingual nearest neighbors in VecMap space (Dufter et al., 2020).

2. Core algorithmic design

VecMap’s original algorithm comprises four interlocking components: embedding normalization, unsupervised initialization, iterative self-learning, and final refinement (Artetxe et al., 2018).

Normalization standardizes both embedding spaces before alignment. In later expositions of the unsupervised algorithm, both source and target embeddings are described as length-normalized, mean-centered across dimensions, and length-normalized again (Rathnayake et al., 2024). This preprocessing is complemented in broader evaluations by ZCA whitening on both languages, and by postprocessing steps such as cross-correlational re-weighting, de-whitening, and dimensionality reduction (Glavas et al., 2019).

The unsupervised initialization is based on the idea that a word’s similarity distribution with respect to other words acts as a structural fingerprint. For normalized embeddings $X$ and $Z$, VecMap computes

$M_X = XX^\top,\qquad M_Z = ZZ^\top .$

Each row of these similarity matrices is then sorted independently, and the values are square-rooted to reduce variance and facilitate comparability. The resulting representations are used to induce an initial, noisy dictionary entirely from structural similarity (Artetxe et al., 2018).

Self-learning alternates between mapping estimation and dictionary induction. Given a current dictionary $Z$0, VecMap computes the optimal orthogonal mapping by SVD, maps the embeddings into the current shared space, induces a new dictionary, and repeats until convergence. Retrieval is typically performed with nearest neighbors or, more robustly, with CSLS: $Z$1 CSLS is explicitly used to mitigate hubness and is described as critical in both the original ablations and later comparative evaluations (Artetxe et al., 2018).

The robustness of self-learning depends on several engineering choices. The original method uses stochastic dictionary induction, frequency-based vocabulary cutoff, bidirectional dictionary induction, and symmetric re-weighting. The symmetric re-weighting step applies

$Z$2

after convergence, with whitening and de-whitening borrowed from prior work to further align the spaces optimally (Artetxe et al., 2018). Later analyses identify VecMap’s distinctive robustness as deriving largely from these normalization and stochastic self-learning procedures rather than from a radically different linear objective (Glavas et al., 2019).

3. Empirical profile and evaluation

The original paper positioned VecMap against adversarial unsupervised methods and argued that those approaches often fail in realistic scenarios, whereas VecMap succeeds in all tested scenarios and obtains the best published results in standard datasets, even surpassing previous supervised systems (Artetxe et al., 2018). A particularly emphasized case was English–Finnish on harder monolingual corpora, where other unsupervised or adversarial systems often failed while VecMap always converged to meaningful dictionaries (Artetxe et al., 2018).

A broader comparative analysis later evaluated supervised and unsupervised cross-lingual word embedding models on 28 language pairs and three downstream tasks. In that study, VecMap was the most robust unsupervised model in bilingual lexicon induction, with MAP $Z$3 on “All LPs,” MAP $Z$4 on “Filt. LPs,” and successful runs on $Z$5 language pairs; by contrast, Muse obtained MAP $Z$6 on “All LPs” and succeeded on $Z$7 pairs (Glavas et al., 2019). The same study emphasized that simple supervised mapping methods still outperform VecMap on BLI when a seed dictionary is available, and that the performance gap widens for distant language pairs (Glavas et al., 2019).

The downstream profile was more nuanced. VecMap achieved XNLI accuracies of $Z$8 for en–de, $Z$9 for en–fr, $W = \arg\min_{W \in O(d)} \|WX - Z\|_F ,$0 for en–tr, and $W = \arg\min_{W \in O(d)} \|WX - Z\|_F ,$1 for en–ru, with an average of $W = \arg\min_{W \in O(d)} \|WX - Z\|_F ,$2 (Glavas et al., 2019). On CLDC, its micro-averaged $W = \arg\min_{W \in O(d)} \|WX - Z\|_F ,$3 was $W = \arg\min_{W \in O(d)} \|WX - Z\|_F ,$4, and on CLIR its average MAP was $W = \arg\min_{W \in O(d)} \|WX - Z\|_F ,$5 (Glavas et al., 2019). These results supported a recurring conclusion in the evaluation literature: optimizing a cross-lingual embedding model for BLI can deteriorate downstream performance, and VecMap’s importance lies partly in its balance between lexicon induction and transfer utility rather than in absolute dominance on any single benchmark (Glavas et al., 2019).

This evaluation history also corrected a common misconception. Claims that unsupervised models match or surpass supervised models in BLI were explicitly challenged: in the systematic setup of the 2019 analysis, VecMap remained the strongest unsupervised baseline, but supervised projection approaches still performed better whenever even small bilingual dictionaries were available (Glavas et al., 2019).

4. Integration into multilingual representation learning

VecMap has repeatedly been used as an auxiliary alignment mechanism in multilingual pretrained models. In “Identifying Necessary Elements for BERT’s Multilinguality,” monolingual fastText embeddings from different languages are projected into a shared embedding space with VecMap and then used to modify masked language modeling. The resulting “knn-replace” masking replaces masked tokens according to

$W = \arg\min_{W \in O(d)} \|WX - Z\|_F ,$6

where the $W = \arg\min_{W \in O(d)} \|WX - Z\|_F ,$7 mass corresponds to replacement by a cross-lingual nearest neighbor in VecMap space from another language (Dufter et al., 2020). On XNLI, after pretraining on English, German, and Hindi and fine-tuning on English, the VecMap-based masking yielded accuracies of $W = \arg\min_{W \in O(d)} \|WX - Z\|_F ,$8 for English, $W = \arg\min_{W \in O(d)} \|WX - Z\|_F ,$9 for German, and $O(d)$0 for Hindi, improving zero-shot performance over the baseline on German and Hindi (Dufter et al., 2020). The paper interprets this as faster and stronger emergence of multilinguality early during pretraining.

VecMap was also used to align static embeddings extracted from XLM-R. In “Combining Static and Contextualised Multilingual Embeddings,” static embeddings for 40 languages were extracted from layer 6 of XLM-R using X2Static and then aligned with VecMap. The aligned resource, denoted X2S-MA, achieved $O(d)$1 on MUSE BLI for en$O(d)$2xx and $O(d)$3 for xx$O(d)$4en, compared with $O(d)$5 and $O(d)$6 for fastText+VecMap, and it then served as an anchor for continued pre-training of XLM-R with alignment losses such as MSE and DCCA (Hämmerl et al., 2022).

Contextual alignment work extended VecMap further. In “Cross-lingual alignments of ELMo contextual embeddings,” anchor points were created from parallel corpora and bilingual dictionaries so that contextual ELMo vectors for translation-equivalent words in matching contexts could be aligned. VecMap was then applied layer-wise as a supervised linear mapping method. The paper reports that VecMap is more stable than MUSE on contextual ELMo mappings, that it is generally competitive on NER, and that it consistently outperforms ELMoGAN and MUSE on dependency parsing while being best or tied best in sentiment analysis (Ulčar et al., 2021). A plausible implication is that linearity remains surprisingly effective when the contextual alignment dataset is carefully constructed.

5. Low-resource, semi-supervised, and domain-specific bilingual lexicon induction

VecMap has been especially prominent in low-resource and semi-supervised BLI. In “Unsupervised Bilingual Lexicon Induction for Low Resource Languages,” the unsupervised version of VecMap was used as the backbone for English–Sinhala, English–Tamil, and English–Punjabi. Baseline Precision@1 values were $O(d)$7, $O(d)$8, and $O(d)$9 with Word2Vec, and $\|\cdot\|_F$0, $\|\cdot\|_F$1, and $\|\cdot\|_F$2 with FastText, for EnSi, EnTa, and EnPa respectively (Rathnayake et al., 2024). The best-performing combinations added linear transformation and, in some configurations, static-contextual combination, reaching $\|\cdot\|_F$3 on EnSi with Word2Vec for “+ Lin. Transform (M3)” and $\|\cdot\|_F$4 on EnTa with FastText for “CSCBLI + Linear Transformation + UVecMap” (Rathnayake et al., 2024). The paper’s broader conclusion is that combining techniques consistently improves or maintains pr@1 over VecMap alone, but that fusion methods and aggressive dimensionality reduction can harm performance (Rathnayake et al., 2024).

Later work treated VecMap as a backbone to be enhanced rather than replaced. “Improving Bilingual Lexicon Induction with Cross-Encoder Reranking” used VecMap as one of the main CLWE backbones and showed that BLICEr improved VecMap from $\|\cdot\|_F$5 to $\|\cdot\|_F$6 on XLING 5k, from $\|\cdot\|_F$7 to $\|\cdot\|_F$8 on XLING 1k, and from $\|\cdot\|_F$9 to $$\max_{W_X, W_Z} \sum_i \sum_j D_{ij} \left((X_{i*}W_X)\cdot(Z_{j*}W_Z)\right),$$0 on PanLex-BLI 1k (Li et al., 2022). “Semi-Supervised Bilingual Lexicon Induction with Two-way Interaction” evaluated on the challenging VecMap dataset and reported that its CSS and PSS variants outperformed prior semi-supervised methods across EN-ES, EN-IT, and EN-DE, with averages of $$\max_{W_X, W_Z} \sum_i \sum_j D_{ij} \left((X_{i*}W_X)\cdot(Z_{j*}W_Z)\right),$$1 for CSS under “100 unique,” $$\max_{W_X, W_Z} \sum_i \sum_j D_{ij} \left((X_{i*}W_X)\cdot(Z_{j*}W_Z)\right),$$2 for PSS under “5K unique,” and $$\max_{W_X, W_Z} \sum_i \sum_j D_{ij} \left((X_{i*}W_X)\cdot(Z_{j*}W_Z)\right),$$3 for PSS under “5K all” (Zhao et al., 2020).

Domain adaptation exposed clear limits of the classical static-embedding setting. In “Cross-Domain Bilingual Lexicon Induction via Pretrained LLMs,” unsupervised VecMap with CSLS obtained an average P@1 of $$\max_{W_X, W_Z} \sum_i \sum_j D_{ij} \left((X_{i*}W_X)\cdot(Z_{j*}W_Z)\right),$$4 across Medicine, Law, and Financial domains, whereas the supervised VecMap variant achieved $$\max_{W_X, W_Z} \sum_i \sum_j D_{ij} \left((X_{i*}W_X)\cdot(Z_{j*}W_Z)\right),$$5 (Ding et al., 29 May 2025). The same paper attributed degradation in specialized domains to domain mismatch, vocabulary coverage, and the limitation of static embeddings when word meaning is strongly context-dependent (Ding et al., 29 May 2025).

A representative selection of reported numbers is given below.

Study	VecMap setting	Reported result
(Rathnayake et al., 2024)	Baseline pr@1	EnSi $$\max_{W_X, W_Z} \sum_i \sum_j D_{ij} \left((X_{i*}W_X)\cdot(Z_{j*}W_Z)\right),$$6, EnTa $$\max_{W_X, W_Z} \sum_i \sum_j D_{ij} \left((X_{i*}W_X)\cdot(Z_{j*}W_Z)\right),$$7, EnPa $$\max_{W_X, W_Z} \sum_i \sum_j D_{ij} \left((X_{i*}W_X)\cdot(Z_{j*}W_Z)\right),$$8
(Li et al., 2022)	VecMap $$\max_{W_X, W_Z} \sum_i \sum_j D_{ij} \left((X_{i*}W_X)\cdot(Z_{j*}W_Z)\right),$$9 VecMap + BLICEr	XLING 5k $$USV^\top = X^\top D Z,\qquad W_X = U,\qquad W_Z = V .$$0; XLING 1k $$USV^\top = X^\top D Z,\qquad W_X = U,\qquad W_Z = V .$$1; PanLex $$USV^\top = X^\top D Z,\qquad W_X = U,\qquad W_Z = V .$$2
(Ding et al., 29 May 2025)	Domain BLI	Vecmap_unsupervisedCSLS avg $$USV^\top = X^\top D Z,\qquad W_X = U,\qquad W_Z = V .$$3; Vecmap_supervisedCSLS avg $$USV^\top = X^\top D Z,\qquad W_X = U,\qquad W_Z = V .$$4

VecMap has also been used in genuinely low-resource language work outside standard European benchmarks. “Training Cross-Lingual embeddings for Setswana and Sepedi” used the unsupervised VecMap mode to align independently trained Word2Vec and fastText spaces for Setswana–Sepedi, then evaluated intrinsic quality with translated SimLex and WordSim datasets released as part of the work (Makgatho et al., 2021). The paper reports coverage values ranging from approximately $$USV^\top = X^\top D Z,\qquad W_X = U,\qquad W_Z = V .$$5 to $$USV^\top = X^\top D Z,\qquad W_X = U,\qquad W_Z = V .$$6, with sample cross-lingual results such as $$USV^\top = X^\top D Z,\qquad W_X = U,\qquad W_Z = V .$$7 coverage and Spearman’s $$USV^\top = X^\top D Z,\qquad W_X = U,\qquad W_Z = V .$$8 on Setswana–Sepedi SimLex for cross-lingual word2vec + VecMap (Makgatho et al., 2021).

6. Limitations, debates, and legacy

VecMap’s main strengths are robustness, strong unsupervised performance, and reusability. It is repeatedly identified as the most robust unsupervised reference point for cross-lingual word embedding research, and later surveys explicitly recommend that both Proc-B and VecMap be included in future evaluations as indicative baselines (Glavas et al., 2019). At the same time, the literature is consistent that VecMap is not a universal optimum.

The first limitation is methodological: VecMap is a linear mapping approach with an isomorphism assumption. Work on contextual ELMo alignment reports that nonlinear ELMoGAN methods can outperform linear approaches on some distant language pairs and tasks where isomorphism is not a good assumption (Ulčar et al., 2021). Cross-domain BLI work similarly argues that static embeddings are insufficient in professional fields where context substantially changes word meaning (Ding et al., 29 May 2025).

The second limitation is dependence on the quality of induced dictionaries and structural comparability. In the XLM-R static-alignment study, alignment improved BLI by as much as 17 points for languages such as Bengali and Hindi, but some languages, including Korean, Thai, Tagalog, and Vietnamese, saw drops after alignment, attributed to less reliable induced dictionaries or divergent embedding spaces (Hämmerl et al., 2022). This suggests that VecMap’s strengths are most pronounced when monolingual spaces are sufficiently comparable and the induced anchor structure is stable.

The third limitation concerns what counts as success. Later evaluation work showed that BLI is not a sufficient proxy for downstream transfer quality, and that retrieval-oriented methods optimized narrowly for lexicon induction can underperform on sentence- or document-level tasks (Glavas et al., 2019). VecMap’s lasting legacy therefore lies not only in its original claim to fully unsupervised mapping, but also in its role as a disciplined baseline that made these trade-offs visible.

In that sense, VecMap occupies a distinctive position in multilingual NLP. It is simultaneously a standalone mapping algorithm, a benchmark-grade unsupervised baseline, and an alignment primitive embedded into later systems for multilingual masked language modeling, contextual embedding transfer, low-resource dictionary induction, and hybrid static-contextual multilingual representation learning (Artetxe et al., 2018).