---
title: Entity-Aligned Translation (EAT)
url: https://www.emergentmind.com/topics/entity-aligned-translation-eat
type: topic
---

# Entity-Aligned Translation (EAT)

Entity-Aligned Translation (EAT) denotes, in the literature summarized here, a set of methods that make entities explicit objects of translation, alignment, or reconstruction rather than treating them as ordinary tokens. Across neural machine translation, cross-lingual named entity recognition, knowledge graph alignment, and entity-focused pre-training, the common pattern is to preserve entity identity under language transfer by introducing dedicated mechanisms such as dictionary lookup, transliteration, translation-mediated span matching, dual translation with large language models, adversarial knowledge translation, or code-switched denoising objectives [2306.02242]. This suggests that EAT is best understood not as a single model family with a fixed architecture, but as a recurrent design principle: entity handling is externalized and then reintegrated into a downstream sequence or graph model [1909.05356].

## 1. Conceptual scope

In neural machine translation, EAT appears explicitly as the “Extract-and-Attend” approach, motivated by the observation that when humans encounter an unknown entity, they may first look it up in a dictionary and then incorporate the result into a fluent sentence [2306.02242]. In cross-lingual NER, the same label has been used informally for translation-mediated entity projection pipelines in which sentences are translated, entity spans are matched across languages, and source tags are projected to target text [1909.05356]. In zero-shot cross-lingual NER, EAT is formulated as a dual-translation strategy using large language models to align entities between non-Latin-script languages and English [2509.01147]. In cross-lingual knowledge graph alignment, the term is associated with adversarial knowledge translation over random walks once entities have been embedded in a common space [2104.07837]. A related but distinct realization appears in DEEP, where entity-aligned translation is induced through denoising pre-training on code-switched monolingual text linked to a knowledge base [2111.07393].

A recurring technical motivation is data sparsity. In NMT, inaccurate translation of person, organization, and location names is attributed to the lack of entity training instances [2306.02242]. In CL-NER, transfer quality degrades when orthographic, phonetic, or structural differences prevent direct span projection, especially for non-Latin-script languages [1909.05356]. In knowledge graphs, alignment suffers from feature inconsistency and sequence context unawareness [2104.07837]. In entity pre-training, purely phonetic transliteration is described as ignoring sentence context and being limited in domain and language coverage [2111.07393].

The principal unifying idea is therefore explicit entity preservation under cross-lingual transformation. Depending on the task, the preserved object may be a surface span, a dictionary candidate, a masked node in a random walk, or a linked knowledge-base item. A plausible implication is that “alignment” in EAT should be read broadly: it can mean token-level lexical preservation, span-level correspondence, latent-space consistency, or reconstruction of the correct target-language entity form.

## 2. Prefix-based entity integration in neural machine translation

The most direct EAT formulation is the “Extract-and-Attend” method for NMT, which implements a two-step workflow: entity extraction and candidate lookup, followed by integration of the extracted candidates into the decoder input [2306.02242]. A source sentence $X=(x_1,\dots,x_I)$ is processed by a named-entity recognition model,
$$
\mathrm{NER}:X\;\longrightarrow\;\mathcal{E}=\{(s_j,e_j,t_j)\}_{j=1}^N,
$$
where each detected entity span $x_{s_j}\dots x_{e_j}$ has type $t_j\in\{\mathrm{PER,ORG,LOC}\}$.

For each detected source entity $u_j=x_{s_j:e_j}$, candidate translations are retrieved from a bilingual dictionary $\mathcal{D}$ through
$$
\mathrm{lookup}(u_j)=\{v_{j,1},v_{j,2},\dots\}\subseteq\mathcal{V}_{\mathrm{tgt}}.
$$
If the lookup set is non-empty, training selects
$$
v_j = \arg\min_{v\in\mathrm{lookup}(u_j)}\mathrm{LD}(v,y^*_j),
$$
where $\mathrm{LD}$ is Levenshtein distance to the reference entity $y^*_j$; at inference, the most frequent $v_j$ is chosen [2306.02242]. If lookup fails, a nationality-aware transliteration model is used, with nationality predicted by $\mathrm{ClassifyNationality}(u_j,X)$ and the candidate produced by $\mathrm{Transliterate}(u_j\Vert t_j^{\mathrm{nat}})$.

Architecturally, the method keeps a standard Transformer encoder-decoder. Let $C=(v_1,\dots,v_N)$ denote the candidate sequence in source-entity order. The decoder input is formed as
$$
Z = [c_1,\dots,c_{|C|},\langle\mathrm{BOS}\rangle,y_1,\dots,y_{t-1}],
$$
so that the entity candidates appear as a prefix [2306.02242]. Separate positional embeddings are assigned to the candidate prefix and the output tokens. In each decoder layer, self-attention is computed over the entire concatenated sequence:
$$
Q = ZW_Q,\quad K = ZW_K,\quad V = ZW_V,
$$
$$
A = \mathrm{Softmax}(QK^T/\sqrt{d}),\quad \mathrm{SelfAtt}_\ell(Z)=AV.
$$
Because output-token queries can attend jointly to both prefix keys and ordinary decoder-state keys, the decoder can draw entity candidate embeddings into its prediction context.

The training objective remains the standard cross-entropy,
$$
\mathcal{L}(\theta) = -\sum_{t=1}^{T}\log p_\theta(y_t\mid y_{<t},X,C),
$$
while ignoring loss on the prefix tokens [2306.02242]. No auxiliary loss is introduced; the model must learn from the sequence modeling objective to use the prefix for both entity accuracy and fluency.

On En→Zh and En→Ru, the method improves both entity translation and global quality. Reported gains include Entity Error Rate reduction by up to 35%, BLEU improvement of 0.85, and COMET improvement of 13.8 points; an example given for En→Zh is a reduction from 42.7% to 27.7% EER and an increase from 26.32 to 27.17 BLEU and from 34.8 to 48.6 COMET [2306.02242]. Ablation results indicate that removing transliteration raises EER by 13.2 points and lowers BLEU by 0.33, placing the prefix in the decoder outperforms placing it in the encoder by approximately 2 BLEU, 2 COMET points, and about 2% EER, and using a single candidate performs similarly to using multiple candidates while being simpler.

The paper also identifies characteristic failure modes. NER errors can provide an incorrect prefix, but the decoder may still override it because attention is soft. Dictionary noise or low coverage triggers transliteration back-off, which can fail if nationality is misclassified. In addition, the entity error metric may undercount correct alternate translations [2306.02242]. These caveats matter because they show that the method is not a hard lexically constrained decoder; it is a soft conditioning mechanism.

## 3. Translation-mediated span alignment in cross-lingual named entity recognition

A different EAT lineage arises in cross-lingual NER through the “Translate–Match–Project” pipeline, sometimes referred to informally as Entity-Aligned Translation [1909.05356]. Here the objective is not sentence translation quality but creation of a synthetic labeled target-language corpus from an annotated English source corpus.

The pipeline has three stages. First, each English sentence $x^{S_i}$ in an annotated corpus $D_A^S=\{(x^{S_i},y^{S_i})\}$ is translated by a machine translation system to produce target-language sentences $D^T=\{x^{T_i}\}$ [1909.05356]. Second, source entities are aligned to spans in the translated sentence. For each source entity $e^S$, the method generates hypotheses
$$
T(e^S)=\{T_G(e^S), e^S, T_L(e^S)\},
$$
consisting of machine translation of the entity in isolation, a copied source string, and lexicon translations. Third, once a target span $e^T$ is identified, the source IOB labels are projected to the target sentence.

Candidate matching combines orthographic, phonetic, and distributional heuristics. For each hypothesis token $h$ and target token $x_l^T$, a soft affix score is computed from the length of the longest common prefix or suffix:
$$
s^w(h,x_l^T)=\min\{n_l/|h|,\;n_l/|x_l^T|\},
$$
where the same procedure is repeated on IPA transliterations via Epitran, and the higher score is retained [1909.05356]. Entity-to-token scores are then defined by maximizing over all hypothesis tokens. Adjacent target-token sequences whose scores exceed a threshold $\delta$ become candidate spans. To choose the best span, the method enumerates permutations of the hypothesis translations and minimizes character-level Levenshtein distance. Residual unmatched entities are handled by corpus-wide tf–idf ranking:
$$
\mathrm{tfidf}(s,t)=\mathrm{tf}(s,t)\times \log\bigl(|\mathrm{Sentences}|/\mathrm{df}(t)\bigr).
$$

Evaluation uses conventional NER $F_1$, with source training from English CoNLL-2003 for Spanish, German, Dutch, Hindi, and Tamil, OntoNotes-English for Chinese, and additional analysis on Armenian [1909.05356]. Reported scores include Spanish $73.5\pm0.4$, German $61.5\pm0.4$, Dutch $69.9\pm0.4$, Chinese $50.1\pm0.2$, Hindi $41.7\pm1.3$, and Tamil $33.8\pm2.2$, with an average gain of 4.1 $F_1$ points over the best prior cross-lingual baseline. On Armenian, the Reverse paradigm reaches $F_1=62.6$, outperforming fast-align at 44.8 and a monolingual Armenian model at 62.2 [1909.05356].

This formulation broadens the meaning of EAT. The translated sentence is not itself the final product; rather, translation is an intermediate mechanism for entity projection. The entity-alignment problem is local at the span level but supported by sentence translation, isolated entity translation, copying, lexicons, and corpus-wide statistics. A plausible implication is that EAT in CL-NER is less about generative fluency than about preserving annotation-bearing spans under translation noise.

## 4. LLM-based dual translation for zero-shot cross-lingual NER

In zero-shot cross-lingual NER for non-Latin-script languages, EAT is reframed as a dual-translation procedure driven by large language models [2509.01147]. The framework contains three stages: a Dual-Translation module, source-oriented cross-lingual corpus collection (Eacl), and entity-level alignment fine-tuning.

The Dual-Translation module first translates a target-language sentence $x$ into English using multi-round chain-of-thought prompting, denoted MrCoT. The forward pass is described as
$$
o_1^t = L_m(p_1^t; x, a\to b),\qquad
o_2^t = L_m(o_1^t; p_2^t, x, a\to b),
$$
followed by the filtered English output
$$
T_{a\to b}^t(x)=L_m(o_2^t; p^f, x, a\to b).
$$
An English text-to-text extractor then predicts entity spans $E=\{(l_1:r_1,\mathrm{tag})\}$ from the translated English sentence [2509.01147]. For each English entity span $e_s=T_{a\to b}^t(x)_{l_1:r_1}$, a backward MrCoT pass translates the span back into the target language:
$$
o_1^e = L_m(p_1^e; e_s, x, a\leftarrow b),\qquad
o_2^e = L_m(p_2^e; o_1^e, x, a\leftarrow b),
$$
and finally
$$
x_{l_2:r_2}=L_m(p^f; o_2^e).
$$
If the result matches a contiguous substring of the original sentence, the alignment is accepted [2509.01147].

Entity preservation is encouraged by the prompting design. The two-round chain-of-thought prompts explicitly instruct the model to consider and describe entities before translation, and a filtering prompt removes hallucinated or non-entity tokens. The paper notes that it does not introduce an explicit alignment score, though it sketches a possible formulation $S(e_s,e_t)=P_{\mathrm{align}}(e_t\mid e_s)$ and a corresponding loss $\mathcal{L}_{\mathrm{align}}$ as a potential extension [2509.01147]. This is important because it distinguishes the proposed system from conventional latent alignment models: acceptance depends on substring presence after backward translation rather than on a separately optimized scoring function.

The Eacl stage uses Wikipedia interlanguage links. English-entity descriptions are gathered from CoNLL2003, totaling 8,082 entities, and linked tuples $\{(u^a,v_1^a)_e\mid a\in A\}$ are built, where $u^a$ is the target-language title and $v_1^a$ its first-sentence description [2509.01147]. These corpora are then used to fine-tune the backward translation LLM by QLoRA with cross-entropy
$$
L_T(y,\hat y) = -\sum_i \hat y_i \log y_i,
$$
where $\hat y_i$ is the true entity title $u^a$ and $y_i=L_m(e,v_1^a)$ is the model prediction.

The fine-tuned backbone is Qwen2.5-14B-Instruct quantized to 4-bit NF4, with LoRA rank 64, scaling 16, a 90/10 train/valid split, 5 epochs, and learning rate $1\mathrm{e}{-4}$ [2509.01147]. On WikiANN zero-shot transfer to AR, HI, HY, JA, KA, KO, RU, and ZH, the reported average $F_1$ is 65.38 without fine-tuning and 65.81 with fine-tuning. The paper states that EAT without fine-tuning outperforms DenKD by approximately 4.2 points and that gains are largest on Japanese, at +14 points versus DenKD [2509.01147]. Additional results show that smaller Qwen backbones reduce both BLEU and $F_1$, extractor choice has minor effect, in-context GPT-4 and Qwen do not match EAT, and three MrCoT rounds offer the best trade-off among 1, 3, and 5 rounds.

The method also makes explicit several limitations: dependence on the multilingual pre-training coverage of the LLM, possible harmful inductive biases from fine-tuning, and higher computational cost than single-pass teacher-student methods [2509.01147]. These observations qualify the gains by locating them within a trade-off space of precision, latency, and model capacity.

## 5. Entity alignment beyond text: adversarial knowledge translation in knowledge graphs

In cross-lingual knowledge graph alignment, EAT is realized through a dual adversarial framework called DAEA, which addresses feature inconsistency and sequence context unawareness [2104.07837]. The first stage, Adversarial Kernel Embedding, encodes source and target knowledge graphs with a shared GCN into embeddings $H_s\in\mathbb{R}^{n_s\times D}$ and $H_t\in\mathbb{R}^{n_t\times D}$. Distributional divergence is measured in a reproducing kernel Hilbert space via Maximum Mean Discrepancy:
$$
\mathrm{MMD}(H_s,H_t)=\|\mu[p]-\mu[q]\|_{\mathcal H},
$$
with $\mu[p]=E_{v\sim p}[K(v,\cdot)]$ and $\mu[q]=E_{u\sim q}[K(u,\cdot)]$ [2104.07837].

Because direct MMD minimization is intractable, a neural discriminator $F$ approximates the RKHS mapping, yielding the empirical objective
$$
\mathrm{MMD}(H_s,H_t)\approx
\left\|
\frac{1}{n_s}F(H_s)^T1_{n_s}-
\frac{1}{n_t}F(H_t)^T1_{n_t}
\right\|_F^2.
$$
Training follows a GAN-style minimax between the shared encoder $E$ and the discriminator $F$, with discriminator loss maximizing empirical MMD and generator loss minimizing empirical MMD plus a supervised triplet-ranking term over known aligned seeds [2104.07837]. The purpose is to extract graph-invariant information and project entities from both graphs into a shared space.

The second stage, Adversarial Knowledge Translation, introduces long-sequence context through paired random walks. For a target anchor entity, a random walk
$$
W_t=(w_t^1,\dots,w_t^L)
$$
is sampled; a masked source walk
$$
W_s=(w_s^1,\dots,w_s^L),\quad M=(M_1,\dots,M^L)
$$
keeps aligned anchors and replaces unknown positions with a mask symbol $\phi$ [2104.07837]. A filler network reconstructs the masked positions, and a translator maps the completed source walk into a target-space walk, while a critic distinguishes real target walks from translated ones. The adversarial loss is
$$
L_{\mathrm{adv}}=
- E_{W_t}[\log D(W_t)]-
E_{W_s,M}[\log(1-D(T(F(W_s))))].
$$
Additional reconstruction terms regularize both the filler and the translator, and the resulting generator loss is
$$
L_T=L_{\mathrm{adv}}+\sum_{l=1}^{L}\bigl[d(F(w_s^l),w_s^l)+d(T(F(w_s^l)),w_t^l)\bigr].
$$

This is the most structurally distant use of EAT among the papers considered here. The aligned object is no longer a text span but an entity embedding enriched by graph structure, attributes, relations, and sequence context [2104.07837]. Nevertheless, the notion of translation remains literal within the model: the translator $T$ maps source-space entity representations into context-enhanced target-space representations. On DBP15K for ZH–EN, JA–EN, and FR–EN, with 15,000 seed alignments per subset and a 30/70 train/test split, DAEA reports ZH–EN Hits@1 of 0.828, compared with 0.787 for CEA and lower scores for MTransE, IPTransE, and BootEA [2104.07837].

A plausible implication is that EAT can be generalized from lexical transfer to representational transfer. Under this reading, entity alignment and entity translation become two aspects of the same operation: preserving identity under a change of language-specific representation.

## 6. Entity-focused pre-training, evaluation issues, and research directions

DEEP provides another realization of entity-aligned translation by embedding entity correspondence directly into pre-training data construction rather than adding an explicit alignment module [2111.07393]. Target-language monolingual sentences are tagged with entity mention spans using SLING, each span is linked to a Wikidata item, and multilingual surface forms are used to retrieve the English counterpart of each entity. If $y$ is a target-language sentence and $t$ an entity mention linked to knowledge-base item $e$, the English translation is defined as
$$
s=\mathrm{lookup}(T_e,\text{``en''}),
$$
where $T_e$ is the set of multilingual surface forms [2111.07393].

Noise is injected in three stages. First, code-switch replacement substitutes each detected target-language entity span with its English counterpart:
$$
y_{\mathrm{rep}}=\mathit{swap}(y;\,t\mapsto s).
$$
Second, if replaced tokens account for less than 35% of the sentence, additional non-entity spans are masked until roughly 35% of tokens are replaced or masked. Third, sentence order is randomly permuted [2111.07393]. The denoising objective reconstructs the original sentence from this corrupted input:
$$
L_{\mathrm{DEEP}}(\mathcal{D}_Y,\mathrm{KB})
=
\sum_{y\in\mathcal{D}_Y}
-\log P_\theta(y\mid f(y,\mathrm{KB})).
$$
In fine-tuning, this objective is combined with the standard MT loss
$$
L_{\mathrm{MT}}(\mathcal{D}_{X,Y})
=
\sum_{(x,y)\in\mathcal{D}_{X,Y}}
-\log P_\theta(y\mid x),
$$
with balanced token sampling so that the combined loss is
$$
L_{\mathrm{multi}}=L_{\mathrm{MT}}(\mathcal{D}_{X,Y})+L_{\mathrm{DEEP}}(\mathcal{D}'_Y,\mathrm{KB}).
$$

The architecture is a standard 12×12-layer Transformer with task tokens “[DEEP]” and “[MT]” and separate task embeddings [2111.07393]. The paper explicitly states that no additional alignment module is needed; the cross-linguistic signal arises from the denoising objective itself. This contrasts with the prefix-based and dual-translation approaches, which expose alignment more directly.

Reported gains cover En–Ru, En–Uk, and En–Ne. For En→Ru, DEEP→DEEP+MT reaches 19.6 BLEU versus 18.9 for DAE→DAE+MT, a gain of 1.3 BLEU, and 56.4% entity translation accuracy versus 47.2%, a gain of 9.2 percentage points [2111.07393]. Qualitative examples show that DEEP+MT can correctly render multiple Russian place names where DAE-based models either copy English forms or generate incorrect target forms. The paper frames earlier named entity translation methods as mainly focusing on phonetic transliteration and therefore missing context, which situates DEEP within a broader move from isolated entity conversion toward context-conditioned entity reconstruction [2111.07393].

Across the literature, evaluation is task-specific and not fully commensurate. NMT work reports BLEU, COMET, and Entity Error Rate defined by checking whether each reference entity appears in the output [2306.02242]. CL-NER work reports entity-level $F_1$ [1909.05356; 2509.01147]. Knowledge graph alignment reports Hits@1, Hits@10, and MRR [2104.07837]. The NMT paper also notes that an entity error metric may undercount correct alternate translations [2306.02242]. This is a substantive methodological caution: strong entity preservation may not be fully captured by surface-form matching, while span-level and graph-level evaluation may privilege different aspects of alignment quality.

Several future directions are stated explicitly. For Extract-and-Attend, possible extensions include unifying multiple entity types through knowledge graphs, integrating the approach into multilingual or low-resource NMT by sharing prefix-attention parameters across languages, and leveraging large pre-trained models such as mBART to improve NER and transliteration [2306.02242]. For LLM-based zero-shot CL-NER, proposed directions include lighter entity-aware LLMs, dynamic prompting or self-consistent ensembling, extension to nested or overlapping NER by iterating dual-translation stages, and explicit alignment scoring or contrastive alignment losses [2509.01147]. Taken together, these proposals suggest an ongoing convergence between symbolic resources, multilingual pre-training, and explicit entity-centric objectives.

Source: https://www.emergentmind.com/topics/entity-aligned-translation-eat