Entity Embeddings: Models & Applications

• Entity embeddings are learned continuous representations that map discrete entities to geometric spaces, replacing one-hot encodings with dense vectors.
• Multiple methodologies—including supervised lookup, co-occurrence modeling, and graph-based approaches—shape embeddings for diverse applications such as NLP and knowledge graphs.
• Their practical utility is demonstrated in enhanced prediction tasks, improved entity linking, and efficient retrieval, making them versatile tools across domains.

Searching arXiv for recent and foundational papers on entity embeddings to ground the article. Searching arXiv for "entity embeddings" and related terms. Entity embeddings are learned representations that assign discrete entities to continuous spaces so that similarity, structure, or task-specific behavior can be computed geometrically. Across the literature, the term covers several distinct constructions: dense vectors for categorical values learned during supervised training (Guo et al., 2016), entity points constrained to type-specific conceptual subspaces (Jameel et al., 2016), embeddings of Wikipedia, Wikidata, and knowledge-graph entities (Nielsen, 2017, Gerritse et al., 2020), fine-grained type-aware entity representations for entity linking and related NLP tasks (Hou et al., 2021, Onoe et al., 2020), ontology models in which classes are balls and relations are translations (Kulmanov et al., 2019), social account embeddings induced by co-follow patterns (Lotan et al., 2021), shared latent representations of dynamic game entities (Khameneh et al., 2020), and hyperbolic prototypes of geographic entities for image geolocation (Gadi et al., 30 Jan 2026). The unifying idea is that discrete identity is replaced by a learned geometry, but the semantics of that geometry vary substantially across domains.

1. Conceptual scope and historical development

A foundational formulation appears in work on categorical variables, where an entity embedding is a learned dense vector in a Euclidean space that replaces sparse one-hot encoding. For a categorical variable $x_i$ with values $\{1,\dots,m_i\}$, the embedding is a map $e_i: x_i \mapsto \mathbf{x}_i$, learned jointly with the rest of a neural network during supervised training. In that setting, the central claim is that categories with similar effects on the prediction task should be placed near one another, yielding lower-dimensional inputs, reduced memory use, faster training, and better generalization on sparse or high-cardinality structured data (Guo et al., 2016).

A second early line treats entity embeddings as a geometric substrate for conceptual knowledge. In that formulation, entities are points, salient features are directions, and natural properties are convex regions; entities of the same semantic type are constrained to lie in lower-dimensional subspaces so that the learned geometry supports ranking by attributes, induction from examples, and analogy making. This perspective explicitly connects entity embeddings to conceptual spaces rather than treating them only as task-specific features (Jameel et al., 2016).

Subsequent work broadened the term from categorical values to named entities, graph nodes, social accounts, dynamic game entities, and geographic regions. Wikidata items and properties were embedded as nodes in a low-dimensional space for similarity search (Nielsen, 2017). Wikipedia- and graph-based embeddings were used for entity retrieval and linking (Gerritse et al., 2020, Hou et al., 2021). Description-logic classes and roles were embedded so that the geometry itself approximates a model of the ontology (Kulmanov et al., 2019). SocialVec learned embeddings for popular Twitter accounts from co-follow patterns (Lotan et al., 2021). HierLoc treated country, region, subregion, and city as geographic entities embedded in hyperbolic space for hierarchical visual geolocation (Gadi et al., 30 Jan 2026). This suggests that “entity” in this literature is not restricted to named individuals; it can denote any discrete unit whose behavior, semantics, or relational role is to be encoded geometrically.

2. Representation families and learning mechanisms

The simplest mechanism is the learned lookup layer. In the categorical-variable formulation, the embedding layer is equivalent to placing a learned linear layer on top of one-hot encoding, so that the vector for category $x_i$ is the corresponding row of the weight matrix: $$\mathbf{x}_i \equiv \sum_\alpha w_{\alpha\beta}\,\delta_{x_i\alpha} = w_{x_i\beta}.$$ This makes the geometry directly shaped by the supervised loss for $y = f(x_1, x_2, \dots, x_n)$, with the embedding dimension $D_i$ chosen as a hyperparameter satisfying $1 \le D_i \le m_i - 1$ (Guo et al., 2016).

A second family learns entity vectors from co-occurrence. Classical word2vec- and GloVe-style objectives were adapted to entities in multiple ways: raw or entity-annotated text (Almasian et al., 2019); Wikipedia2Vec, which combines word-context, graph-structure, and anchor-text objectives (Gerritse et al., 2020); Wikidata triples treated as short graph walks and fed to Gensim’s Word2Vec (Nielsen, 2017); and social co-follow sets adapted to skip-gram and CBOW in SocialVec (Lotan et al., 2021). In these models, entities become first-class tokens whose local or graph neighborhoods determine their position in the space.

A third family imposes stronger geometric structure. EL Embeddings for $\mathcal{EL}^{++}$ map each class to an open $n$-ball in $\{1,\dots,m_i\}$0 and each binary relation to a translation vector, with separate losses enforcing the normal forms $\{1,\dots,m_i\}$1, $\{1,\dots,m_i\}$2, $\{1,\dots,m_i\}$3, and $\{1,\dots,m_i\}$4. If the total loss is zero and $\{1,\dots,m_i\}$5, the theory has a model under the induced interpretation (Kulmanov et al., 2019). HierLoc similarly uses a non-Euclidean geometry, embedding geographic entities in the Lorentz model of hyperbolic space and optimizing a Geo-Weighted Hyperbolic InfoNCE objective that directly incorporates haversine distance into the contrastive loss (Gadi et al., 30 Jan 2026).

A fourth family replaces opaque dense vectors with semantically labeled coordinates. Interpretable entity representations through large-scale typing define an embedding as a vector of posterior probabilities over a fine-grained type inventory: $\{1,\dots,m_i\}$6 Each coordinate corresponds to a named type, making the embedding human-readable while still usable as a continuous representation for cosine- or dot-product comparison (Onoe et al., 2020). This is a materially different notion of embedding from purely latent Euclidean vectors.

3. Entity embeddings in natural language processing

In neural entity linking, entity embeddings are central because they enter both local mention-context scoring and global coherence terms. In the Ganea and Hofmann style summarized by later work, a local score is written as

$\{1,\dots,m_i\}$7

and global models use pairwise coherence terms such as

$\{1,\dots,m_i\}$8

This setup motivates semantic reinforcement of entity vectors: the standard Wikipedia-trained embeddings are useful but “too distinctive” for learning contextual commonality (Hou et al., 2021).

FGS2EE modifies existing entity embeddings by injecting fine-grained semantic type information extracted from Wikipedia. It constructs a dictionary of 3,227 fine-grained semantic words, extracts up to $\{1,\dots,m_i\}$9 semantic words per entity from the first sentence of the entity’s Wikipedia article, remaps uncommon semantic words when necessary, averages their Word2Vec vectors to obtain

$e_i: x_i \mapsto \mathbf{x}_i$0

and linearly aggregates this semantic embedding with the original Wikitext entity embedding: $e_i: x_i \mapsto \mathbf{x}_i$1 Evaluated by replacing the entity embeddings inside mulrel and wnel without changing the linking models themselves, the semantic reinforced embeddings improve micro F1 on standard corpora including AIDA-B, MSNBC, AQUAINT, ACE2004, CWEB, and WIKI, and training converges much faster than with Wikitext-only embeddings (Hou et al., 2021).

E-BERT addresses a different problem: injecting factual entity knowledge into pretrained BERT without additional encoder pretraining. It aligns Wikipedia2Vec vectors with BERT’s wordpiece space by fitting a linear transformation $e_i: x_i \mapsto \mathbf{x}_i$2 on shared vocabulary and then treats transformed entity vectors as if they were ordinary BERT input embeddings: $e_i: x_i \mapsto \mathbf{x}_i$3 Because the encoder remains unchanged, the method is positioned as efficient relative to approaches that add entity-specific layers and require further pretraining. On unsupervised QA, relation classification, and entity linking, E-BERT outperforms BERT and the baselines discussed in that work; on LAMA-UHN in particular, the aligned entity vectors mitigate BERT’s reliance on surface-form cues (Poerner et al., 2019).

A third NLP line makes entity embeddings explicitly interpretable. Large-scale typing models generate vectors whose coordinates are posterior probabilities over 10k or 60k fine-grained types and then use cosine similarity directly for coreference arc prediction and named entity disambiguation. On CoNLL-YAGO NED, this parameter-free cosine ranking reaches 84.8 test accuracy, exceeding the ELMo and BERT baselines reported there; on WLNED it reaches 75.6 (Onoe et al., 2020). By contrast, work on named entity linking with multiple embeddings argues that a knowledge-base entity should not be represented by a single vector but by several contextual prototypes. In that setting, the minimal number of mentions required to create a KB entity is reported to be very important for NEL performance, whereas the number of embeddings is less important and can be kept small, within as few as 10 or less (Vasilyev et al., 2022).

The literature also records a cautionary result: simply training standard word embeddings on entity-annotated corpora does not reliably yield better entity vectors. When named entities in a corpus are replaced by canonical identifiers and word2vec or GloVe are trained naively, entity embeddings are less useful than expected and non-entity word embeddings can degrade relative to raw-text training. Graph-based node embeddings over a co-occurrence graph can restore performance, especially for entity-centric relatedness (Almasian et al., 2019).

4. Knowledge graphs, ontology models, and structured reasoning

Knowledge-graph-oriented work often treats entity embeddings as carriers of relational structure. Wembedder is a deliberately simple instance: it trains 100-dimensional embeddings for Wikidata items and properties using Gensim’s Word2Vec on triples extracted from the wikidata-20170613-truthy-BETA.nt.bz2 dump, treating each $e_i: x_i \mapsto \mathbf{x}_i$4 triple as a three-token sentence. The resulting vocabulary contains 609,471 total tokens, including 608,733 Wikidata items and 738 properties, and is exposed through a Flask web service with nearest-neighbor and pairwise-similarity endpoints (Nielsen, 2017).

More sophisticated graph-based models jointly use text, links, and anchors. In graph-embedding-empowered entity retrieval, Wikipedia2Vec is trained with the combined objective

$e_i: x_i \mapsto \mathbf{x}_i$5

where $e_i: x_i \mapsto \mathbf{x}_i$6 encodes linked-entity neighborhoods and $e_i: x_i \mapsto \mathbf{x}_i$7 ties entities to words around anchor mentions. Candidate entities are then reranked by an embedding-based score

$e_i: x_i \mapsto \mathbf{x}_i$8

interpolated with a first-stage lexical ranker. On DBpedia-Entity V2, graph-enhanced embeddings improve retrieval more than context-only embeddings, and the paper attributes part of the gain to better clustering of relevant entities in embedding space (Gerritse et al., 2020).

Entity Context Graphs generalize the knowledge-graph paradigm to semi-structured text sources where no formal KG exists. Instead of fixed symbolic relations, they use context triples of the form $e_i: x_i \mapsto \mathbf{x}_i$9, where the “relation” is a free-form text span extracted around a secondary entity mention in an entity-centric document. A multi-layer CNN encodes the context text, and a TransE-style margin loss is optimized over positive and corrupted triples. The resulting ECG embeddings are reported to be comparable to known KG-based embeddings, better than a contextual LLM-based entity embedding baseline, and useful in domains where a knowledge graph is not readily available (Gunaratna et al., 2021).

At the strongest model-theoretic end, EL Embeddings seek a vector-space embedding that is itself an approximate model of a normalized $x_i$0 TBox. Classes are open balls, relations are translation vectors, the bottom concept is handled with explicit losses, and ABox axioms are rewritten into TBox form. The method is motivated by large life-science ontologies and is evaluated on protein–protein interaction prediction, where it outperforms semantic similarity measures and standard knowledge-graph embeddings on yeast and human datasets (Kulmanov et al., 2019).

Jointly learning entity and relation representations for entity alignment occupies an intermediate position. Entities are first encoded by a Highway-GCN, relation vectors are then approximated from the embeddings of their head and tail entities, and joint entity representations concatenate entity embeddings with aggregated relation embeddings. On DBP15K cross-lingual alignment benchmarks, the joint models outperform the state of the art reported there, and the paper’s central claim is that entity alignment benefits when relation information is learned and fed back into entity embeddings rather than treated as a byproduct (Wu et al., 2019).

5. Extensions beyond text and knowledge graphs

In tabular learning, entity embeddings remain closely tied to categorical variables, but later work revisits the preprocessing assumptions that shape them. The original supervised formulation shows that learned embeddings can be reused as features for k-nearest neighbors, random forests, and gradient boosted trees, with large reported improvements on the Rossmann Store Sales task; it also argues that embeddings define a distance measure on categorical variables and can support visualization and clustering (Guo et al., 2016). A later benchmark on tabular encoding challenges the use of ordinal encoding, comparing entity embeddings and transformer-based context embeddings under several encodings. Its reported conclusion is that ordinal encoding is often not well suited for categorical data, whereas String Similarity encoding often performs better; the transformer architecture improves performance for ordinal and similarity encoding on multi-label classification tasks (Reusser, 2024).

SocialVec adapts the distributional principle to social-network structure. Here the entities are popular Twitter accounts followed by at least $x_i$1 sampled users, yielding about 201,247 embedded accounts from 1.265 million distinct Twitter users and 1.236 billion follow relationships. The training objective replaces word windows with full co-follow sets, using skip-gram or CBOW over the accounts followed by an individual user. The resulting embeddings are designed to capture “social world knowledge,” including political orientation, audience demographics, and topical affinity. On personal-trait prediction and news-source political-bias estimation, SocialVec is reported to outperform Wikipedia2Vec and Wikidata-based baselines (Lotan et al., 2021).

In procedural content generation via machine learning, entity embeddings are used to represent dynamic game entities by their mechanical properties rather than appearance. A Variational AutoEncoder maps a unified 1 × 1600 one-hot representation of mechanical state variables to a 25-dimensional latent representation, trained jointly over entities from Centipede and Space Invaders. The paper argues that the embedding space captures functional or mechanical similarity across games and reports better reconstruction performance than PCA on held-out entities (Khameneh et al., 2020).

HierLoc reframes visual geolocation as image-to-entity alignment over a hierarchy of geographic entities. Each country, region, subregion, and city is represented by a hyperbolic embedding derived from averaged image features, a text embedding of the entity name, and averaged coordinates. On OSV5M, the method uses about 240k entity embeddings instead of over 5 million image embeddings, and the headline result is a 19.5% reduction in mean geodesic error together with a 43.2% subregion accuracy improvement relative to the strongest baselines discussed there (Gadi et al., 30 Jan 2026). This is a clear example of entity embeddings serving as compact multimodal prototypes rather than as simple token vectors.

6. Interpretability, misconceptions, limitations, and prospective directions

A persistent misconception is that entity embeddings are necessarily opaque dense vectors. Several strands of the literature dispute this directly. Large-scale typing uses coordinates that correspond to named types and permits post-hoc rule-based modification of type probabilities to inject domain knowledge (Onoe et al., 2020). EmbeddingTree treats learned embeddings as objects for hierarchical feature analysis rather than as uninterpretable end products: it recursively splits the embedding cloud by the feature whose Embedding-BIC score best explains a one-dimensional PCA projection, and its associated visualization tool is intended to reveal dominant and local feature structure, support feature denoising or injecting, and initialize embeddings for unseen entities (Zheng et al., 2023).

A second misconception is that entity embeddings are always single vectors attached to entity IDs. The surveyed work includes classes represented as balls and relations as translations (Kulmanov et al., 2019), KB entities represented by multiple contextual prototypes (Vasilyev et al., 2022), fine-grained type-probability vectors (Onoe et al., 2020), and multimodal geographic prototypes in hyperbolic space (Gadi et al., 30 Jan 2026). A forward-looking perspective goes further and proposes treating conceptual entities themselves as implicit modalities in LLMs, so that locations, corporations, numbers, and dates would be encoded by entity-specific encoders into a shared latent linguistic space (Unlu et al., 2023). This is a speculative extension rather than an established standard, but it broadens the design space considerably.

The main limitations are also recurrent across domains. Quality depends heavily on upstream extraction, coverage, and alignment. FGS2EE depends on the quality of the semantic word dictionary, the first-sentence extraction heuristic, and remapping for rare semantic words; it is also a preprocessing step rather than an end-to-end learned component (Hou et al., 2021). E-BERT depends on coverage of entity IDs in Wikipedia2Vec and on the quality of the linear alignment into BERT space (Poerner et al., 2019). Graph-based retrieval is highly sensitive to entity-linker accuracy (Gerritse et al., 2020). ECG assumes entity-centric documents and uses surrounding text as noisy contextual relations (Gunaratna et al., 2021). SocialVec reflects the biases of Twitter users and requires sufficient co-follow context (Lotan et al., 2021). Named entity linking with multiple embeddings is strongly affected by the minimum support required to build a KB entity (Vasilyev et al., 2022). Wembedder is explicitly not state of the art and uses only direct triples with modest evaluation performance (Nielsen, 2017).

Taken together, the literature presents entity embeddings less as a single technique than as a representational paradigm. They can be supervised lookup vectors, graph- and co-occurrence-based node representations, logical models constrained by ontology semantics, interpretable type distributions, multiple contextual prototypes, or multimodal hierarchical entities. Their success depends on how well the chosen geometry matches the structure of the domain, whether that structure is categorical, lexical, relational, social, mechanical, or geographic.