Triple2Vec: Direct Triple Embedding Technique

• Triple2Vec is a graph embedding technique that directly learns triple (edge) embeddings by modeling knowledge graph triples with a novel triple line graph formalism.
• It introduces a dual-mode edge weighting mechanism that leverages semantic predicate relatedness and node centrality to guide random walk corpus generation.
• Empirical results demonstrate that Triple2Vec outperforms traditional node-centric approaches in classification and clustering tasks by enhancing semantic fidelity.

Triple2Vec is a graph embedding technique designed to learn direct embeddings for edges—specifically, knowledge graph triples—rather than the traditional focus on node embeddings. Triple2Vec addresses limitations in prior methodologies, which relied on aggregating node embeddings to approximate edge representations. The method introduces conceptual and algorithmic innovations, including the triple line graph formalism, a novel edge weighting strategy, and a walk-based embedding mechanism adapted from word embedding models. Empirical evaluation demonstrates that Triple2Vec generates high-quality triple embeddings with enhanced semantic fidelity, outperforming prominent node-centric baselines across classification and clustering tasks.

1. Motivation and Scope

Triple2Vec was developed to overcome two principal deficiencies in extant graph embedding techniques: (i) they exclusively target node embeddings; (ii) their edge embeddings, when needed, are superficial aggregations of endpoint node vectors, inadequately reflecting multiple edge semantics. In knowledge graphs, edges expressing relations (triples of the form $(s,p,o)$) encode rich semantic dependencies, and multiple edges can link the same node pair with different predicates. Aggregation methods obliterate these distinctions and degrade semantic accuracy. Triple2Vec introduces direct triple embedding, aiming to capture semantic and topological proximity of triples, thus expanding the representational capacity for edge-centric downstream tasks.

2. Line Graph versus Triple Line Graph Formalism

Triple2Vec formalizes the embedding process through the construction of a triple line graph, a generalization of the classic line graph. In traditional undirected graphs, the line graph $L(G)$ has nodes corresponding to edges in $G$, and adjacency in $L(G)$ is determined by edge endpoint sharing in $G$. Triple2Vec extends this to directed graphs and multigraphs by focusing on triples $(s,p,o)$.

In the triple line graph:

• Each node corresponds to a triple from the knowledge graph.
• Adjacency is defined: two triple nodes are connected if their underlying triples share at least one endpoint (subject or object), regardless of directionality.
• The triple line graph preserves the richness of the original knowledge graph by embedding edge-level semantics and topological proximity.
• For connected original graphs, the triple line graph inherits connectedness, which is essential for effective random walk corpus generation.

3. Edge Weighting Mechanism

Unweighted triple line graphs yield excessively dense and noisy structures. Triple2Vec introduces a dual-mode edge weighting mechanism:

A. Knowledge Graphs:

Edge weights reflect semantic relatedness between triple predicates:

• For triple nodes $(s_1,p_1,o_1)$ and $(s_2,p_2,o_2)$, the weight is computed using predicate relatedness $R(p_1,p_2)$.
• $R(p_1,p_2)$ employs cosine similarity between predicate co-occurrence vectors and is adjusted by predicate population counts, leading to semantically guided edge transitions during walk sampling.

B. Homogeneous Graphs:

Edge weights encode node centrality:

• For an edge representing the path $i$–$j$–$k$ in the original graph, weight is calculated as:

$$w(n_p, n_q) = \alpha \cdot cb(i) + \beta \cdot cb(j) + \gamma \cdot cb(k)$$

where $cb(x)$ is the current-flow betweenness centrality of node $x$ and $\alpha + \beta + \gamma = 1$.

This weighting guides random walks to favor triple nodes with higher semantic or structural importance, mitigating combinatorial explosion and improving context fidelity.

4. Random Walk Corpus Generation

Embedding inference is predicated on the generation of a corpus via random walks in the weighted triple line graph:

• Walks originate at each triple node, traversing edges stochastically with transition probabilities proportional to edge weights.
• Semantic weights induce walks that cluster semantically related triples (knowledge graphs); centrality weights aggregate walks around structurally relevant regions (homogeneous graphs).
• Each walk is treated analogously to a sentence in word embedding models, with sequence proximity reflecting latent semantic or structural similarity.

This corpus forms the basis for Skip-Gram–based embedding optimization.

5. Skip-Gram Based Embedding Optimization

The corpus of walks is input to a Skip-Gram embedding model:

• Each “word” is a triple node from the triple line graph.
• Context for a triple is constituted by nodes within a fixed window in its walk.
• The Skip-Gram objective maximizes the log-probability of a triple’s context, modeled via dot product of embedding vectors:

$$p((e_{v_i}, e_{v_{i+1}}) \in \mathcal{R}) = \sigma(e_{v_i}^\top e_{v_{i+1}})$$

where $\sigma(x)$ is the sigmoid function.

• Full softmax computation is replaced by negative sampling for scalability:

$$\mathcal{O}(\theta) = \log \sigma(e_{v_i}^\top e_{v_{i+1}}) + \sum_{j=1}^{k} \mathbb{E}_{v_j}\left[\log \sigma(- e_{v_i}^\top e_{v_j})\right]$$

with $k$ negative samples per positive pair. The learned map $f$: triple nodes $\rightarrow \mathbb{R}^d$, $d \ll |V|$, constitutes the desired low-dimensional triple embeddings.

6. Empirical Evaluation

Triple2Vec was benchmarked on both knowledge graphs and homogeneous graph datasets:

Knowledge Graphs:

• Triple labels were generated by propagating node labels; evaluated using one-vs-rest Logistic Regression.
• Performance was assessed via Micro-F1 and Macro-F1 scores.
• Triple2Vec significantly outperformed baselines (node2vec, DeepWalk, and metapath2vec), whose edge embeddings used aggregation functions.
• t-SNE visualizations revealed that Triple2Vec embeddings clustered triples by semantic label with greater clarity.

Homogeneous Graphs:

• Edge embeddings were applied to edge classification and clustering tasks using datasets such as Karate Club and USA Power Grid.
• Triple2Vec attained superior classification accuracy compared to node2vec and DeepWalk.

A key result is that directly embedding triples yields representations with richer semantic and structural information than constructing edge vectors via endpoint aggregation.

7. Mathematical Summary

Triple2Vec’s principal mathematical definitions are:

• Skip-Gram context probability: $$p((e_{v_i}, e_{v_{i+1}}) \in \mathcal{R}) = \sigma(e_{v_i}^\top e_{v_{i+1}})$$
• Negative sampling objective: $$\mathcal{O}(\theta) = \log \sigma(e_{v_i}^\top e_{v_{i+1}}) + \sum_{j=1}^{k} \mathbb{E}_{v_j}\left[\log \sigma(- e_{v_i}^\top e_{v_j})\right]$$
• Centrality-based edge weighting: $$w(n_p, n_q) = \alpha \cdot cb(i) + \beta \cdot cb(j) + \gamma \cdot cb(k)$$, with $\alpha + \beta + \gamma = 1$

These formulations collectively enable the unbiased representation of triple semantics and structural significance in the embedding space.

Conclusion

Triple2Vec is the first graph embedding technique to directly optimize triple (edge) embeddings for both knowledge and homogeneous graphs. By integrating the triple line graph formalism, semantic and centrality-aware weighting, walk-based corpus generation, and Skip-Gram driven optimization, Triple2Vec produces embeddings that encode nuanced semantic and structural relationships unattainable by node-centric aggregation methods. Its empirical superiority across multiple datasets and tasks substantiates its efficacy in representing edge-level graph information.