EIKE: Dual-Space Ontology Embedding

• EIKE is a dual-space ontology embedding framework that combines instance-level structure and concept-level semantics to capture richer semantic relationships.
• It employs geometry-based modeling in an extensional space alongside language model–derived embeddings in an intensional space to jointly represent ontology elements.
• Experimental results demonstrate that the dual-space approach yields 2–5% improvements in triple classification and link prediction over traditional geometry-only models.

EIKE is a framework for ontology embedding that incorporates both extensional (instance-level, assertional) and intensional (concept-level, terminological) knowledge by optimizing dual representation spaces. Unlike previous approaches that model only structure or textual semantics, EIKE jointly embeds instances, concepts, and relations by combining geometry-based modeling in an extensional space with LLM–derived embeddings in an intensional space. The method advances ontology embeddings for tasks such as triple classification and link prediction by enabling a richer, two-space encoding of semantic relationships (Wang et al., 2024).

1. Motivation and Context

Ontology embeddings aim to map entities (instances), concepts, and relations of an ontology into low-dimensional continuous spaces, facilitating efficient computation for downstream tasks including knowledge graph completion, reasoning, and semantic search. Ontological knowledge comprises:

• Extensional Knowledge: Instance-level assertions that specify which individuals are instances of which concepts, and the relations between individuals.
• Intensional Knowledge: Terminological axioms and textual semantics that characterize conceptual hierarchies, properties, and semantic similarity (e.g., subclass and lexical definitions).

Traditional models either emphasize the geometric structure of individual–concept relations (e.g., TransC, TransEllipsoid) or focus on textual semantics (e.g., OWL2Vec*). EIKE integrates both, motivated by the observation that real-world ontologies benefit from concurrent modeling of structural and lexical knowledge.

2. Formal Framework and Representation Spaces

Given an ontology $O = \{C, I, R, T, L\}$ with:

• $C$ = set of concepts (classes)
• $I$ = set of instances (individuals)
• $R = \{r_e, r_c\} \cup R_l$ (instance–concept, subclass, and instance–instance relations)
• $T$ consisting of instance–instance, subclass, and instance–concept triples
• $L$ = set of textual/literal descriptions for concepts

EIKE constructs two parallel vector spaces:

• Extensional Space ($\mathbb{R}^d$): Instance $i$ maps to a point $i^e \in \mathbb{R}^d$; concept $c$ maps to an ellipsoidal region $C$0, with $C$1 center and $C$2 axes.
• Intensional Space ($C$3): Concept $C$4 maps to $C$5 using a pretrained LLM (Sentence-BERT) on its textual description; instance $C$6 is a linear transformation ($C$7) of $C$8 to align with the intensional space.

The scoring functions for evaluating triples depend on both spaces, mixing geometric inclusion/containment and cosines of semantic vectors.

3. Modeling Components

3.1 Extensional-Space Modeling

• Concept Representation: Concepts as ellipsoids in $C$9 support instance inclusion and subclass nesting, following the TransEllipsoid paradigm.
• Instance–Concept (InstanceOf) Score:

$I$0

Enforces that $I$1 is within the margin $I$2 of $I$3.

• Subclass Score:

$I$4

• Relational Triples: Modeled by

$I$5

3.2 Intensional-Space Modeling

• Concept Embeddings: Obtained via a pretrained LLM applied to each concept’s textual literal; $I$6.
• Virtual Instance Embeddings:

$I$7

with either $I$8 (EYE) or learned (MAT).

• Scoring: Instance–concept similarity by cosine,

$I$9

and subclass similarity by

$R = \{r_e, r_c\} \cup R_l$0

No direct modeling of instance–instance relations in intensional space.

4. Unified Optimization and Training

Joint learning combines extensional and intensional scoring for instance-of and subclass triples with a weighting parameter $R = \{r_e, r_c\} \cup R_l$1. For each triple, the composite score is: $R = \{r_e, r_c\} \cup R_l$2 Similar for $R = \{r_e, r_c\} \cup R_l$3, with $R = \{r_e, r_c\} \cup R_l$4 as above.

Training uses a margin-ranking loss over positive and negative samples for each relation type, with parameters updated via SGD or Adam, including optional regularization by vector normalization or norm penalties. Two variants for $R = \{r_e, r_c\} \cup R_l$5 allow for either fixed or learnable cross-space transformation.

5. Experimental Validation

Experiments conducted on YAGO39K, M-YAGO39K, and DB99K-242 (DBpedia) evaluate triple classification and link prediction. Baselines include TransE, TransEllipsoid, DistMult, and other geometry/text/mixed methods.

Key quantitative results:

Model	YAGO39K (InstanceOf, %)	DB99K-242 (SubclassOf, %)	YAGO39K (MRR, rel. link pred.)
TransEllipsoid	87.12	38.0	0.536
EIKE-PRE-EYE	89.32	78.8	0.577

Ablation reveals that pretrained intensional encoders (PRE) and identity mapping for $R = \{r_e, r_c\} \cup R_l$6 (EYE) outperform random/text-untrained and learnable transformations on several metrics, particularly when concept semantics is critical. The dual-space approach consistently yields 2–5% improvements over geometry-only models in classification.

6. Implications, Limitations, and Future Directions

EIKE demonstrates that simultaneous modeling of extensional and intensional information in parallel vector spaces produces ontology embeddings capturing both structural and lexical semantics, with measurable improvements for assertional and taxonomic predictivity. Pretrained LLMs substantially boost concept-level representation quality, and a fixed bridge from instance geometry to concept text aligns spaces with stable optimization.

Limitations include imperfect preservation of isA-transitivity in long subclass chains, omission of advanced axiom types (e.g., property domains/ranges, role hierarchies), and reliance on fixed or pretrained transformer representations without in-training adaptation. The intensional encoder is currently restricted to SubClassOf logic.

Future research includes extending the framework to wider OWL constructs, employing graph neural networks for high-order extensional modeling, dynamic fine-tuning of the LLM, and developing more flexible (potentially non-ellipsoidal) region models to represent highly polysemous or compositional concepts (Wang et al., 2024).

7. Position within the Field

EIKE constitutes a methodological advance in ontology embedding by enforcing a principled, dual-space architecture—distinct from singular geometry/text approaches—while explicitly interpolating structured instance-concept and semantic concept-concept associations. It benchmarks favorably against established geometric (TransEllipsoid) and hybrid models and suggests the viability of further two-space or multi-modal extensions, particularly as ontology-encoded knowledge sources grow in size and complexity.