Entity-Aligned Translation (EAT)

• Entity-Aligned Translation (EAT) is a design principle that treats entities as explicit objects in translation, alignment, or reconstruction to preserve their identity.
• It spans neural machine translation, cross-lingual NER, knowledge graph alignment, and pre-training, addressing data sparsity and representation challenges.
• EAT methods enhance performance metrics like BLEU, COMET, and F1 by leveraging tailored mechanisms such as dictionary lookup, dual translation, and adversarial learning.

Entity-Aligned Translation (EAT) denotes, in the literature summarized here, a set of methods that make entities explicit objects of translation, alignment, or reconstruction rather than treating them as ordinary tokens. Across neural machine translation, cross-lingual named entity recognition, knowledge graph alignment, and entity-focused pre-training, the common pattern is to preserve entity identity under language transfer by introducing dedicated mechanisms such as dictionary lookup, transliteration, translation-mediated span matching, dual translation with LLMs, adversarial knowledge translation, or code-switched denoising objectives (Zeng et al., 2023). This suggests that EAT is best understood not as a single model family with a fixed architecture, but as a recurrent design principle: entity handling is externalized and then reintegrated into a downstream sequence or graph model (Jain et al., 2019).

1. Conceptual scope

In neural machine translation, EAT appears explicitly as the “Extract-and-Attend” approach, motivated by the observation that when humans encounter an unknown entity, they may first look it up in a dictionary and then incorporate the result into a fluent sentence (Zeng et al., 2023). In cross-lingual NER, the same label has been used informally for translation-mediated entity projection pipelines in which sentences are translated, entity spans are matched across languages, and source tags are projected to target text (Jain et al., 2019). In zero-shot cross-lingual NER, EAT is formulated as a dual-translation strategy using LLMs to align entities between non-Latin-script languages and English (Zhang et al., 1 Sep 2025). In cross-lingual knowledge graph alignment, the term is associated with adversarial knowledge translation over random walks once entities have been embedded in a common space (Zhang et al., 2021). A related but distinct realization appears in DEEP, where entity-aligned translation is induced through denoising pre-training on code-switched monolingual text linked to a knowledge base (Hu et al., 2021).

A recurring technical motivation is data sparsity. In NMT, inaccurate translation of person, organization, and location names is attributed to the lack of entity training instances (Zeng et al., 2023). In CL-NER, transfer quality degrades when orthographic, phonetic, or structural differences prevent direct span projection, especially for non-Latin-script languages (Jain et al., 2019). In knowledge graphs, alignment suffers from feature inconsistency and sequence context unawareness (Zhang et al., 2021). In entity pre-training, purely phonetic transliteration is described as ignoring sentence context and being limited in domain and language coverage (Hu et al., 2021).

The principal unifying idea is therefore explicit entity preservation under cross-lingual transformation. Depending on the task, the preserved object may be a surface span, a dictionary candidate, a masked node in a random walk, or a linked knowledge-base item. A plausible implication is that “alignment” in EAT should be read broadly: it can mean token-level lexical preservation, span-level correspondence, latent-space consistency, or reconstruction of the correct target-language entity form.

2. Prefix-based entity integration in neural machine translation

The most direct EAT formulation is the “Extract-and-Attend” method for NMT, which implements a two-step workflow: entity extraction and candidate lookup, followed by integration of the extracted candidates into the decoder input (Zeng et al., 2023). A source sentence $X=(x_1,\dots,x_I)$ is processed by a named-entity recognition model,

$$\mathrm{NER}:X\;\longrightarrow\;\mathcal{E}=\{(s_j,e_j,t_j)\}_{j=1}^N,$$

where each detected entity span $x_{s_j}\dots x_{e_j}$ has type $t_j\in\{\mathrm{PER,ORG,LOC}\}$.

For each detected source entity $u_j=x_{s_j:e_j}$, candidate translations are retrieved from a bilingual dictionary $\mathcal{D}$ through

$$\mathrm{lookup}(u_j)=\{v_{j,1},v_{j,2},\dots\}\subseteq\mathcal{V}_{\mathrm{tgt}}.$$

If the lookup set is non-empty, training selects

$$v_j = \arg\min_{v\in\mathrm{lookup}(u_j)}\mathrm{LD}(v,y^*_j),$$

where $\mathrm{LD}$ is Levenshtein distance to the reference entity $y^*_j$; at inference, the most frequent $$\mathrm{NER}:X\;\longrightarrow\;\mathcal{E}=\{(s_j,e_j,t_j)\}_{j=1}^N,$$0 is chosen (Zeng et al., 2023). If lookup fails, a nationality-aware transliteration model is used, with nationality predicted by $$\mathrm{NER}:X\;\longrightarrow\;\mathcal{E}=\{(s_j,e_j,t_j)\}_{j=1}^N,$$1 and the candidate produced by $$\mathrm{NER}:X\;\longrightarrow\;\mathcal{E}=\{(s_j,e_j,t_j)\}_{j=1}^N,$$2.

Architecturally, the method keeps a standard Transformer encoder-decoder. Let $$\mathrm{NER}:X\;\longrightarrow\;\mathcal{E}=\{(s_j,e_j,t_j)\}_{j=1}^N,$$3 denote the candidate sequence in source-entity order. The decoder input is formed as

$$\mathrm{NER}:X\;\longrightarrow\;\mathcal{E}=\{(s_j,e_j,t_j)\}_{j=1}^N,$$4

so that the entity candidates appear as a prefix (Zeng et al., 2023). Separate positional embeddings are assigned to the candidate prefix and the output tokens. In each decoder layer, self-attention is computed over the entire concatenated sequence:

$$\mathrm{NER}:X\;\longrightarrow\;\mathcal{E}=\{(s_j,e_j,t_j)\}_{j=1}^N,$$5

$$\mathrm{NER}:X\;\longrightarrow\;\mathcal{E}=\{(s_j,e_j,t_j)\}_{j=1}^N,$$6

Because output-token queries can attend jointly to both prefix keys and ordinary decoder-state keys, the decoder can draw entity candidate embeddings into its prediction context.

The training objective remains the standard cross-entropy,

$$\mathrm{NER}:X\;\longrightarrow\;\mathcal{E}=\{(s_j,e_j,t_j)\}_{j=1}^N,$$7

while ignoring loss on the prefix tokens (Zeng et al., 2023). No auxiliary loss is introduced; the model must learn from the sequence modeling objective to use the prefix for both entity accuracy and fluency.

On En→Zh and En→Ru, the method improves both entity translation and global quality. Reported gains include Entity Error Rate reduction by up to 35%, BLEU improvement of 0.85, and COMET improvement of 13.8 points; an example given for En→Zh is a reduction from 42.7% to 27.7% EER and an increase from 26.32 to 27.17 BLEU and from 34.8 to 48.6 COMET (Zeng et al., 2023). Ablation results indicate that removing transliteration raises EER by 13.2 points and lowers BLEU by 0.33, placing the prefix in the decoder outperforms placing it in the encoder by approximately 2 BLEU, 2 COMET points, and about 2% EER, and using a single candidate performs similarly to using multiple candidates while being simpler.

The paper also identifies characteristic failure modes. NER errors can provide an incorrect prefix, but the decoder may still override it because attention is soft. Dictionary noise or low coverage triggers transliteration back-off, which can fail if nationality is misclassified. In addition, the entity error metric may undercount correct alternate translations (Zeng et al., 2023). These caveats matter because they show that the method is not a hard lexically constrained decoder; it is a soft conditioning mechanism.

3. Translation-mediated span alignment in cross-lingual named entity recognition

A different EAT lineage arises in cross-lingual NER through the “Translate–Match–Project” pipeline, sometimes referred to informally as Entity-Aligned Translation (Jain et al., 2019). Here the objective is not sentence translation quality but creation of a synthetic labeled target-language corpus from an annotated English source corpus.

The pipeline has three stages. First, each English sentence $$\mathrm{NER}:X\;\longrightarrow\;\mathcal{E}=\{(s_j,e_j,t_j)\}_{j=1}^N,$$8 in an annotated corpus $$\mathrm{NER}:X\;\longrightarrow\;\mathcal{E}=\{(s_j,e_j,t_j)\}_{j=1}^N,$$9 is translated by a machine translation system to produce target-language sentences $x_{s_j}\dots x_{e_j}$0 (Jain et al., 2019). Second, source entities are aligned to spans in the translated sentence. For each source entity $x_{s_j}\dots x_{e_j}$1, the method generates hypotheses

$x_{s_j}\dots x_{e_j}$2

consisting of machine translation of the entity in isolation, a copied source string, and lexicon translations. Third, once a target span $x_{s_j}\dots x_{e_j}$3 is identified, the source IOB labels are projected to the target sentence.

Candidate matching combines orthographic, phonetic, and distributional heuristics. For each hypothesis token $x_{s_j}\dots x_{e_j}$4 and target token $x_{s_j}\dots x_{e_j}$5, a soft affix score is computed from the length of the longest common prefix or suffix:

$x_{s_j}\dots x_{e_j}$6

where the same procedure is repeated on IPA transliterations via Epitran, and the higher score is retained (Jain et al., 2019). Entity-to-token scores are then defined by maximizing over all hypothesis tokens. Adjacent target-token sequences whose scores exceed a threshold $x_{s_j}\dots x_{e_j}$7 become candidate spans. To choose the best span, the method enumerates permutations of the hypothesis translations and minimizes character-level Levenshtein distance. Residual unmatched entities are handled by corpus-wide tf–idf ranking:

$x_{s_j}\dots x_{e_j}$8

Evaluation uses conventional NER $x_{s_j}\dots x_{e_j}$9, with source training from English CoNLL-2003 for Spanish, German, Dutch, Hindi, and Tamil, OntoNotes-English for Chinese, and additional analysis on Armenian (Jain et al., 2019). Reported scores include Spanish $t_j\in\{\mathrm{PER,ORG,LOC}\}$0, German $t_j\in\{\mathrm{PER,ORG,LOC}\}$1, Dutch $t_j\in\{\mathrm{PER,ORG,LOC}\}$2, Chinese $t_j\in\{\mathrm{PER,ORG,LOC}\}$3, Hindi $t_j\in\{\mathrm{PER,ORG,LOC}\}$4, and Tamil $t_j\in\{\mathrm{PER,ORG,LOC}\}$5, with an average gain of 4.1 $t_j\in\{\mathrm{PER,ORG,LOC}\}$6 points over the best prior cross-lingual baseline. On Armenian, the Reverse paradigm reaches $t_j\in\{\mathrm{PER,ORG,LOC}\}$7, outperforming fast-align at 44.8 and a monolingual Armenian model at 62.2 (Jain et al., 2019).

This formulation broadens the meaning of EAT. The translated sentence is not itself the final product; rather, translation is an intermediate mechanism for entity projection. The entity-alignment problem is local at the span level but supported by sentence translation, isolated entity translation, copying, lexicons, and corpus-wide statistics. A plausible implication is that EAT in CL-NER is less about generative fluency than about preserving annotation-bearing spans under translation noise.

4. LLM-based dual translation for zero-shot cross-lingual NER

In zero-shot cross-lingual NER for non-Latin-script languages, EAT is reframed as a dual-translation procedure driven by LLMs (Zhang et al., 1 Sep 2025). The framework contains three stages: a Dual-Translation module, source-oriented cross-lingual corpus collection (Eacl), and entity-level alignment fine-tuning.

The Dual-Translation module first translates a target-language sentence $t_j\in\{\mathrm{PER,ORG,LOC}\}$8 into English using multi-round chain-of-thought prompting, denoted MrCoT. The forward pass is described as

$t_j\in\{\mathrm{PER,ORG,LOC}\}$9

followed by the filtered English output

$u_j=x_{s_j:e_j}$0

An English text-to-text extractor then predicts entity spans $u_j=x_{s_j:e_j}$1 from the translated English sentence (Zhang et al., 1 Sep 2025). For each English entity span $u_j=x_{s_j:e_j}$2, a backward MrCoT pass translates the span back into the target language:

$u_j=x_{s_j:e_j}$3

and finally

$u_j=x_{s_j:e_j}$4

If the result matches a contiguous substring of the original sentence, the alignment is accepted (Zhang et al., 1 Sep 2025).

Entity preservation is encouraged by the prompting design. The two-round chain-of-thought prompts explicitly instruct the model to consider and describe entities before translation, and a filtering prompt removes hallucinated or non-entity tokens. The paper notes that it does not introduce an explicit alignment score, though it sketches a possible formulation $u_j=x_{s_j:e_j}$5 and a corresponding loss $u_j=x_{s_j:e_j}$6 as a potential extension (Zhang et al., 1 Sep 2025). This is important because it distinguishes the proposed system from conventional latent alignment models: acceptance depends on substring presence after backward translation rather than on a separately optimized scoring function.

The Eacl stage uses Wikipedia interlanguage links. English-entity descriptions are gathered from CoNLL2003, totaling 8,082 entities, and linked tuples $u_j=x_{s_j:e_j}$7 are built, where $u_j=x_{s_j:e_j}$8 is the target-language title and $u_j=x_{s_j:e_j}$9 its first-sentence description (Zhang et al., 1 Sep 2025). These corpora are then used to fine-tune the backward translation LLM by QLoRA with cross-entropy

$\mathcal{D}$0

where $\mathcal{D}$1 is the true entity title $\mathcal{D}$2 and $\mathcal{D}$3 is the model prediction.

The fine-tuned backbone is Qwen2.5-14B-Instruct quantized to 4-bit NF4, with LoRA rank 64, scaling 16, a 90/10 train/valid split, 5 epochs, and learning rate $\mathcal{D}$4 (Zhang et al., 1 Sep 2025). On WikiANN zero-shot transfer to AR, HI, HY, JA, KA, KO, RU, and ZH, the reported average $\mathcal{D}$5 is 65.38 without fine-tuning and 65.81 with fine-tuning. The paper states that EAT without fine-tuning outperforms DenKD by approximately 4.2 points and that gains are largest on Japanese, at +14 points versus DenKD (Zhang et al., 1 Sep 2025). Additional results show that smaller Qwen backbones reduce both BLEU and $\mathcal{D}$6, extractor choice has minor effect, in-context GPT-4 and Qwen do not match EAT, and three MrCoT rounds offer the best trade-off among 1, 3, and 5 rounds.

The method also makes explicit several limitations: dependence on the multilingual pre-training coverage of the LLM, possible harmful inductive biases from fine-tuning, and higher computational cost than single-pass teacher-student methods (Zhang et al., 1 Sep 2025). These observations qualify the gains by locating them within a trade-off space of precision, latency, and model capacity.

5. Entity alignment beyond text: adversarial knowledge translation in knowledge graphs

In cross-lingual knowledge graph alignment, EAT is realized through a dual adversarial framework called DAEA, which addresses feature inconsistency and sequence context unawareness (Zhang et al., 2021). The first stage, Adversarial Kernel Embedding, encodes source and target knowledge graphs with a shared GCN into embeddings $\mathcal{D}$7 and $\mathcal{D}$8. Distributional divergence is measured in a reproducing kernel Hilbert space via Maximum Mean Discrepancy:

$\mathcal{D}$9

with $$\mathrm{lookup}(u_j)=\{v_{j,1},v_{j,2},\dots\}\subseteq\mathcal{V}_{\mathrm{tgt}}.$$0 and $$\mathrm{lookup}(u_j)=\{v_{j,1},v_{j,2},\dots\}\subseteq\mathcal{V}_{\mathrm{tgt}}.$$1 (Zhang et al., 2021).

Because direct MMD minimization is intractable, a neural discriminator $$\mathrm{lookup}(u_j)=\{v_{j,1},v_{j,2},\dots\}\subseteq\mathcal{V}_{\mathrm{tgt}}.$$2 approximates the RKHS mapping, yielding the empirical objective

$$\mathrm{lookup}(u_j)=\{v_{j,1},v_{j,2},\dots\}\subseteq\mathcal{V}_{\mathrm{tgt}}.$$3

Training follows a GAN-style minimax between the shared encoder $$\mathrm{lookup}(u_j)=\{v_{j,1},v_{j,2},\dots\}\subseteq\mathcal{V}_{\mathrm{tgt}}.$$4 and the discriminator $$\mathrm{lookup}(u_j)=\{v_{j,1},v_{j,2},\dots\}\subseteq\mathcal{V}_{\mathrm{tgt}}.$$5, with discriminator loss maximizing empirical MMD and generator loss minimizing empirical MMD plus a supervised triplet-ranking term over known aligned seeds (Zhang et al., 2021). The purpose is to extract graph-invariant information and project entities from both graphs into a shared space.

The second stage, Adversarial Knowledge Translation, introduces long-sequence context through paired random walks. For a target anchor entity, a random walk

$$\mathrm{lookup}(u_j)=\{v_{j,1},v_{j,2},\dots\}\subseteq\mathcal{V}_{\mathrm{tgt}}.$$6

is sampled; a masked source walk

$$\mathrm{lookup}(u_j)=\{v_{j,1},v_{j,2},\dots\}\subseteq\mathcal{V}_{\mathrm{tgt}}.$$7

keeps aligned anchors and replaces unknown positions with a mask symbol $$\mathrm{lookup}(u_j)=\{v_{j,1},v_{j,2},\dots\}\subseteq\mathcal{V}_{\mathrm{tgt}}.$$8 (Zhang et al., 2021). A filler network reconstructs the masked positions, and a translator maps the completed source walk into a target-space walk, while a critic distinguishes real target walks from translated ones. The adversarial loss is

$$\mathrm{lookup}(u_j)=\{v_{j,1},v_{j,2},\dots\}\subseteq\mathcal{V}_{\mathrm{tgt}}.$$9

Additional reconstruction terms regularize both the filler and the translator, and the resulting generator loss is

$$v_j = \arg\min_{v\in\mathrm{lookup}(u_j)}\mathrm{LD}(v,y^*_j),$$0

This is the most structurally distant use of EAT among the papers considered here. The aligned object is no longer a text span but an entity embedding enriched by graph structure, attributes, relations, and sequence context (Zhang et al., 2021). Nevertheless, the notion of translation remains literal within the model: the translator $$v_j = \arg\min_{v\in\mathrm{lookup}(u_j)}\mathrm{LD}(v,y^*_j),$$1 maps source-space entity representations into context-enhanced target-space representations. On DBP15K for ZH–EN, JA–EN, and FR–EN, with 15,000 seed alignments per subset and a 30/70 train/test split, DAEA reports ZH–EN Hits@1 of 0.828, compared with 0.787 for CEA and lower scores for MTransE, IPTransE, and BootEA (Zhang et al., 2021).

A plausible implication is that EAT can be generalized from lexical transfer to representational transfer. Under this reading, entity alignment and entity translation become two aspects of the same operation: preserving identity under a change of language-specific representation.

6. Entity-focused pre-training, evaluation issues, and research directions

DEEP provides another realization of entity-aligned translation by embedding entity correspondence directly into pre-training data construction rather than adding an explicit alignment module (Hu et al., 2021). Target-language monolingual sentences are tagged with entity mention spans using SLING, each span is linked to a Wikidata item, and multilingual surface forms are used to retrieve the English counterpart of each entity. If $$v_j = \arg\min_{v\in\mathrm{lookup}(u_j)}\mathrm{LD}(v,y^*_j),$$2 is a target-language sentence and $$v_j = \arg\min_{v\in\mathrm{lookup}(u_j)}\mathrm{LD}(v,y^*_j),$$3 an entity mention linked to knowledge-base item $$v_j = \arg\min_{v\in\mathrm{lookup}(u_j)}\mathrm{LD}(v,y^*_j),$$4, the English translation is defined as

$$v_j = \arg\min_{v\in\mathrm{lookup}(u_j)}\mathrm{LD}(v,y^*_j),$$5

where $$v_j = \arg\min_{v\in\mathrm{lookup}(u_j)}\mathrm{LD}(v,y^*_j),$$6 is the set of multilingual surface forms (Hu et al., 2021).

Noise is injected in three stages. First, code-switch replacement substitutes each detected target-language entity span with its English counterpart:

$$v_j = \arg\min_{v\in\mathrm{lookup}(u_j)}\mathrm{LD}(v,y^*_j),$$7

Second, if replaced tokens account for less than 35% of the sentence, additional non-entity spans are masked until roughly 35% of tokens are replaced or masked. Third, sentence order is randomly permuted (Hu et al., 2021). The denoising objective reconstructs the original sentence from this corrupted input:

$$v_j = \arg\min_{v\in\mathrm{lookup}(u_j)}\mathrm{LD}(v,y^*_j),$$8

In fine-tuning, this objective is combined with the standard MT loss

$$v_j = \arg\min_{v\in\mathrm{lookup}(u_j)}\mathrm{LD}(v,y^*_j),$$9

with balanced token sampling so that the combined loss is

$\mathrm{LD}$0

The architecture is a standard 12×12-layer Transformer with task tokens “[DEEP]” and “[MT]” and separate task embeddings (Hu et al., 2021). The paper explicitly states that no additional alignment module is needed; the cross-linguistic signal arises from the denoising objective itself. This contrasts with the prefix-based and dual-translation approaches, which expose alignment more directly.

Reported gains cover En–Ru, En–Uk, and En–Ne. For En→Ru, DEEP→DEEP+MT reaches 19.6 BLEU versus 18.9 for DAE→DAE+MT, a gain of 1.3 BLEU, and 56.4% entity translation accuracy versus 47.2%, a gain of 9.2 percentage points (Hu et al., 2021). Qualitative examples show that DEEP+MT can correctly render multiple Russian place names where DAE-based models either copy English forms or generate incorrect target forms. The paper frames earlier named entity translation methods as mainly focusing on phonetic transliteration and therefore missing context, which situates DEEP within a broader move from isolated entity conversion toward context-conditioned entity reconstruction (Hu et al., 2021).

Across the literature, evaluation is task-specific and not fully commensurate. NMT work reports BLEU, COMET, and Entity Error Rate defined by checking whether each reference entity appears in the output (Zeng et al., 2023). CL-NER work reports entity-level $\mathrm{LD}$1 (Jain et al., 2019, Zhang et al., 1 Sep 2025). Knowledge graph alignment reports Hits@1, Hits@10, and MRR (Zhang et al., 2021). The NMT paper also notes that an entity error metric may undercount correct alternate translations (Zeng et al., 2023). This is a substantive methodological caution: strong entity preservation may not be fully captured by surface-form matching, while span-level and graph-level evaluation may privilege different aspects of alignment quality.

Several future directions are stated explicitly. For Extract-and-Attend, possible extensions include unifying multiple entity types through knowledge graphs, integrating the approach into multilingual or low-resource NMT by sharing prefix-attention parameters across languages, and leveraging large pre-trained models such as mBART to improve NER and transliteration (Zeng et al., 2023). For LLM-based zero-shot CL-NER, proposed directions include lighter entity-aware LLMs, dynamic prompting or self-consistent ensembling, extension to nested or overlapping NER by iterating dual-translation stages, and explicit alignment scoring or contrastive alignment losses (Zhang et al., 1 Sep 2025). Taken together, these proposals suggest an ongoing convergence between symbolic resources, multilingual pre-training, and explicit entity-centric objectives.