Interlingua Representation in NLP

Updated 4 April 2026

Interlingua Representation is a language-neutral intermediate formalism that encodes semantic and syntactic structures to bridge linguistic differences.
It integrates symbolic methods like AMR and DRT with neural subspace techniques in multilingual language models and translation systems.
Empirical evaluations highlight improved zero-shot translation and robust alignment metrics, while challenges persist for low-resource languages.

An interlingua representation is a language-neutral, intermediate formalism that encodes semantic, syntactic, or knowledge structure in a way that is not tied to any single natural language. It serves as a pivot for cross-lingual applications such as machine translation, document retrieval, or semantic parsing, enabling robust transfer, modularity, and extensibility. Interlingua approaches have been instantiated across symbolic meaning representation frameworks (e.g., Discourse Representation Structures, Abstract Meaning Representation, Universal Networking Language), multilingual lexical databases, and most recently, high-dimensional neural representations within LLMs and neural machine translation architectures. The following sections provide a comprehensive account of interlingua representations’ conceptual basis, formalizations, modeling approaches, empirical evaluation, and research directions.

1. Formal Definitions and Conceptual Foundations

The core property of an interlingua is language independence: a mapping from language-specific input to a shared meaning space, such that equivalent content from different languages yields equivalent or near-equivalent representations.

Symbolic Interlingua. Systems such as Discourse Representation Theory (DRT), Abstract Meaning Representation (AMR), and Universal Networking Language (UNL) realize the interlingua as a formal structure—typically a graph or set of logic expressions. For example, the Parallel Meaning Bank (PMB) encodes the meaning of a sentence as a DRS:

$\text{DRS} = \langle U, C \rangle, \quad U = \{x_1, x_2, \dots\}~\text{(referents)},~C = \{\text{atomic or compound conditions}\}$

with a fully language-neutral predicate and entity symbol inventory (Abzianidze et al., 2017).

Graph-Theoretic Models. AMR abstracts sentences as rooted, directed, acyclic graphs $(V, E)$ , where nodes are "concepts" (predicates, entities), edges are semantic roles, and the structure is invariant across languages (Wein et al., 2022, Takhshid et al., 2022).
Neural Interlingua. In multilingual NMT and LLMs, the interlingua is a high-dimensional subspace $\mathcal{M}_c \subset \mathbb{R}^d$ in the hidden state space $\mathcal{H}$ , shared (ideally) across all languages, possibly coexisting with fragmented, language-specific subspaces $\mathcal{M}_{f_\ell}$ (Wilie et al., 14 Mar 2025, Lizzo et al., 10 Jan 2026). Encoder outputs from any source language are mapped into this subspace and consumed by any decoder, supporting zero-shot and plug-and-play transfer (Lu et al., 2018, Mao et al., 2023).
Lexical Databases. In multilingual lexical resources, a hub layer of "interlingual concepts" provides a graph to which all language-specific synsets are linked, supporting equivalence, hypernymy, and untranslatability relations (Giunchiglia et al., 2023).

2. Interlingua in Multilingual Neural Architectures

Modern multilingual NLP systems operationalize interlingua representations at scale, particularly in LLMs and NMT.

Multilingual LLMs: Models trained on parallel or comparable corpora develop a high-dimensional "core interlingua subspace" $\mathcal{M}_c$ in the model's hidden state space. Parallel sentences in diverse languages are expected to map to nearby points in $\mathcal{M}_c$ . Empirical investigation with metrics such as Average Neuron-wise Correlation (ANC) and Interlingual Local Overlap (ILO) scores shows that such alignment is partial: high-resource and typologically close languages map closely in $\mathcal{M}_c$ , while low-resource languages often remain in fragmented subspaces (Wilie et al., 14 Mar 2025).
Interlingual Subspace Decomposition: The parameter space of a multilingual transformer can be decomposed as:

$\mathbb{R}^d = S_{\text{shared}} \oplus \bigoplus_{\ell} S^{(\ell)}_{\text{res}} \oplus S_{\text{other}}$

where $S_{\text{shared}}$ (the interlingua) encodes universal semantics, and $(V, E)$ 0 are language-specific (Lizzo et al., 10 Jan 2026). Forgetting experiments show that ablating $(V, E)$ 1 universally destroys knowledge across all languages, confirming its centrality for cross-lingual knowledge transfer.

Neural MT Architectures: Many-to-many and modular NMT systems construct explicit or implicit interlingua layers:
- Explicit: An attentional LSTM or Transformer block shared by all language-pair encoders, supporting both supervised and zero-shot translation (Lu et al., 2018, Mao et al., 2023).
- Implicit: Jointly trained modular encoder–decoder blocks, with combinatorial training pressure enforcing compatible representations in a shared intermediate space H (Escolano et al., 2020, Liao et al., 2021).
- Alignment Losses: Pearson correlation–based losses or denoising auto-encoding objectives explicitly force the outputs of encoders from different languages for parallel sentences to be highly correlated (Escolano et al., 2019, Liao et al., 2021).
Variable-Length Interlingua: Limitations of fixed-size interlingua representations in Transformer architectures are addressed by dynamically predicting the interlingua length per input, with alignment losses enforcing similarity of parallel representations at each position (Mao et al., 2023).

3. Symbolic Interlingua Approaches

Classic meaning representation frameworks provide detailed, language-independent annotation schemes suitable for typologically diverse languages.

Discourse Representation Structures in the PMB: Each sentence is annotated with a compositional, scoped, and universally interpreted structure. Symbolization maps lexical items to a small, normalized inventory; cross-lingual projection of annotations leverages alignments, ensuring identical meaning structures between translations (Abzianidze et al., 2017).
AMR Adaptations: AMR's use as interlingua is extended to languages such as Spanish and Persian, with role sets, predicate inventories, and phenomena-specific extensions (pro-drop, LVCs, gender/number) defined via modular guidelines (Wein et al., 2022, Takhshid et al., 2022). Alignment of semantic graphs across languages enables cross-lingual parsing, generation, and evaluation (Wein et al., 2022).
Universal Networking Language (UNL): UNL employs a system of Universal Words, semantic relation labels, and attributes to form language-agnostic conceptual graphs. Nodes correspond to language-neutral concepts, and directed relations encode predicate-argument structure and event semantics. The representation is readily serialized for cross-lingual knowledge exchange (Ripon et al., 2014).

4. Empirical Characterization and Evaluation

Interlingua representations are assessed both intrinsically (alignment, language-independence) and extrinsically (task transfer, retrieval, translation).

Intrinsic Metrics: Similarity of parallel sentences' representations is measured via cosine/sum aggregation (España-Bonet et al., 2017) or more sophisticated locality-based scores (ILO) (Wilie et al., 14 Mar 2025). High ANC and ILO values correspond to well-aligned, language-neutral interlingua subspaces; t-SNE visualization and PCA support geometric analysis.
Cross-Lingual Generalization: Empirical benchmarks validate that strong interlingua alignment correlates with improved zero-shot translation BLEU (up to +12 points for variable-length interlingua vs. fixed-length) (Mao et al., 2023, Liao et al., 2021). For modular NMT, the ability to add languages without retraining and robust plug-and-play transfer hinge on all encoders mapping into the same intermediate space (Escolano et al., 2020, Escolano et al., 2019).
Semantic Transfer and Information Retrieval: NMT-derived interlingual context vectors yield near-perfect F1 for sentence-alignment tasks in comparable corpora (e.g., 98.2–98.9%) (España-Bonet et al., 2017), and AMR interlingua enables reduction of translationese while preserving high fluency and meaning (Wein et al., 2023).
Limitations: Fragmented representational pockets persist for low-resource and typologically distant languages (Wilie et al., 14 Mar 2025). Over-regularization or poor alignment loss calibration can degrade translation for high-resource pairs (Escolano et al., 2019). In AMR, adaptability requires language-specific conventions for phenomena such as dropped pronouns or complex verbs (Wein et al., 2022, Takhshid et al., 2022).

5. Interlingua in Multilingual Lexical Databases and Pivot-Based Models

Lexical interlingua models serve as a pivot to align, disambiguate, and reason about lexical meanings across hundreds of languages.

Hub-and-Spoke Designs: Multilingual lexical databases introduce a layer of interlingual concepts (e.g., CILI, BabelSynsets, UKC concepts) to which each language's synsets are mapped by equivalence, hypernymy, and gap (untranslatability) relations (Giunchiglia et al., 2023).
Structural Trade-offs: The expressivity of mapping—especially for fine-grained, culturally specific concepts and lexical gaps—hinges on whether the interlingua layer is unbiased (open to all concepts) or dominated by a major language (e.g., PWN in EuroWordNet) (Giunchiglia et al., 2023). Full-coverage systems such as UKC model 100% of equivalence, hypernymy, and gaps; English-centric systems fare substantially worse.
Information Flow: Applications include semantic search, cross-lingual alignment, and modular extension to new languages or lexical domains, limited only by the granularity and coverage of the interlingual graph.

6. Broader Applications, Multimodal Interlingua, and Future Directions

Interlingua methodology extends beyond text, into multimodal and cross-domain settings.

Speech and Vision: Multilingual semantic embedding models for speech and vision use visual context as an "interlingua," aligning speech representations across languages to a shared semantic space. Shared vision–speech models enable direct cross-lingual speech retrieval, with empirical boosts in recall over monolingual or direct audio–audio models (Harwath et al., 2018).
Spoken LLMs: The nature of the interlingua in SLMs depends on the training objective of speech encoders. Translation-trained encoders coupled with modality adapters produce meaning-based interlingua representations; recognition-only encoders yield phonetic transliterations. Whisper-based architectures naturally give rise to a semantic English-centric interlingua (Ògúnrèmí et al., 2 Oct 2025).
Forgetting and Subspace Manipulation: Causal ablation experiments on multilingual LLMs demonstrate that the shared interlingua subspace is critical for all-language knowledge—removal of this subspace via subspace-projection unlearning eliminates facts across languages. Fine-grained manipulation of language-specific and interlingua subspaces provides new tools for safe model editing and domain adaptation (Lizzo et al., 10 Jan 2026).
Open Challenges: Expanding the coverage of the interlingua subspace for low-resource languages, bridging fragmented subspaces via data augmentation or typology-aware objectives, and enhancing alignment via architectural or multi-modal extensions remain active research areas (Wilie et al., 14 Mar 2025). For symbolic frameworks, modularization and typologically informed extensions are required for robust universal semantics (Takhshid et al., 2022).

7. Comparative Table: Core Interlingua Frameworks

Framework / Model	Formalism Type	Language Independence	Key Mechanism	Core Limitation
PMB (DRT) (Abzianidze et al., 2017)	Symbolic Graph	High	Cross-lingual projection, language-neutral annotation	Dependency on high-quality alignments
AMR (Wein et al., 2022, Takhshid et al., 2022)	Symbolic Graph	High (with adaptation)	Predicate roles, concept normalization	Requires language-specific customization
Multilingual LLMs (Wilie et al., 14 Mar 2025, Lizzo et al., 10 Jan 2026)	Neural Subspace	Partial (fragmented for LRLs)	Joint training, subspace alignment	Representation fragmentation
Modular NMT (Escolano et al., 2020, Liao et al., 2021, Lu et al., 2018, Mao et al., 2023)	Neural Subspace	High (for supported lang pairs)	Shared interlingua layer, alignment loss	Zero-shot gaps for unseen languages
MLDBs (Giunchiglia et al., 2023)	Graph Hub	Variable	Synset-concept mapping, hypernymy, gap marking	Lexical bias, expressivity tradeoff
UNL (Ripon et al., 2014)	Graph/Hypergraph	High (ontology-based)	Universal Words, relation/attribute graphs	Manual development, limited NLP toolchain

This table summarizes the principal approaches to interlingua representation, the degree of language independence achieved, the core modeling or algorithmic technology, and their main current limitations according to the cited literature.

In summary, interlingua representations—whether symbolic or neural—have become pivotal in multilingual and cross-modal NLP, supporting modularity, transfer, and scalable extension to new languages or domains. Empirical research has demonstrated both impressive successes and persistent limitations, with active work on improving robustness, coverage, and alignment, especially for under-resourced and typologically distant languages. The continued convergence of symbolic and neural methodologies, combined with geometric and statistical analysis of interlingua subspaces, defines an evolving frontier for computational semantics and cross-lingual technology.