Embedding Knowledge Graph in Function Spaces (2409.14857v2)

Abstract: We introduce a novel embedding method diverging from conventional approaches by operating within function spaces of finite dimension rather than finite vector space, thus departing significantly from standard knowledge graph embedding techniques. Initially employing polynomial functions to compute embeddings, we progress to more intricate representations using neural networks with varying layer complexities. We argue that employing functions for embedding computation enhances expressiveness and allows for more degrees of freedom, enabling operations such as composition, derivatives and primitive of entities representation. Additionally, we meticulously outline the step-by-step construction of our approach and provide code for reproducibility, thereby facilitating further exploration and application in the field.

• The paper introduces a novel paradigm by embedding knowledge graphs in dynamic function spaces instead of traditional static vector spaces.
• It proposes three distinct methodologies—FMult_n, FMult^i_n, and FMult—that utilize polynomial, trigonometric, and neural network functions, respectively.
• Experimental results demonstrate that these functional embeddings outperform conventional models in link prediction tasks on benchmark datasets.

Embedding Knowledge Graphs in Function Spaces

This paper, titled "Embedding Knowledge Graphs in Function Spaces," presents a novel approach to knowledge graph embedding (KGE) by employing function spaces rather than conventional finite vector spaces. The authors introduce three distinct methodologies: $FMult_n$, $FMult^i_n$, and $FMult$, which utilize polynomial functions, trigonometric functions, and neural networks, respectively, to represent entities and relations within knowledge graphs (KGs). This marks a significant divergence from traditional KGE techniques, which typically embed these entities and relations into static $d$-dimensional vector spaces like $\mathbb{R}^d$, $\mathbb{C}^d$, or $\mathbb{H}^d$.

Methodology and Implementation

The primary innovation in this paper is the conceptual shift from static vector embeddings to dynamic functional embeddings. This change is motivated by the potential for higher expressiveness and enhanced flexibility in capturing complex relationships within KGs. The authors provide rigorous formulations for each of the three proposed methods.

1. $FMult_n$: Embedding entities and relations as polynomial functions, denoted as $\mathbb{R}_n[x]$. This method captures the interactions between entities and relations by modeling them through polynomial functions of degree $n$. For example, the embeddings for a triple $\langle h, r, t \rangle$ are computed as:

   $$\mathbf{h}(x) = \sum_{i=0}^{deg} a_i x^i, \quad \mathbf{r}(x) = \sum_{i=0}^{deg} b_i x^i, \quad \mathbf{t}(x) = \sum_{i=0}^{deg} c_i x^i$$

   Here, the scoring function is given by:

   $$FMult_n(\langle h, r, t \rangle) = \langle \mathbf{h}(x) \otimes \mathbf{r}(x), \mathbf{t}(x) \rangle_{\mathcal{L}^2(\Omega)}$$

2. $FMult^i_n$: Extends the previous approach by using trigonometric functions within the complex space, $\mathbb{C}_n[x]$. It aims to capture cyclic and periodic patterns in KGs:

   $$\mathbf{h}(x) = \sum_{k=0}^{deg} a_k e^{i k x}, \quad \mathbf{r}(x) = \sum_{k=0}^{deg} b_k e^{i k x}, \quad \mathbf{t}(x) = \sum_{k=0}^{deg} c_k e^{i k x}$$

   With the scoring function:

   $$FMult^i_n(\langle h, r, t \rangle) = \mathcal{R}e \left( \langle \mathbf{h}(x) \otimes \mathbf{r}(x), \mathbf{t}(x) \rangle_{\mathcal{L}^2(\Omega)} \right)$$

3. $FMult$: Utilizes neural networks to learn more intricate, high-dimensional functional embeddings. Entities and relations are represented dynamically through layers of neural networks, allowing for considerable expressive power:

   $$\mathbf{u}(x) = \sigma \left( W_{\mathbf{u}_n} x + b_{\mathbf{u}_n} \right) \circ \ldots \circ \sigma \left( W_{\mathbf{u}_1} x + b_{\mathbf{u}_1} \right)$$

   Thus, the scoring function becomes:

   $$FMult(\langle h, r, t \rangle) = \langle (\mathbf{h} \circ \mathbf{r})(x), \mathbf{t}(x) \rangle_{\mathcal{L}^2(\Omega)}$$

Experimental Results

The experiments conducted demonstrate that this functional approach to KGE is competitive with, and often surpasses, traditional methods. Evaluations across several benchmark datasets (e.g., UMLS, KINSHIP, and different subsets of the NELL dataset) show that:

• $FMult$ consistently achieves superior performance in metrics such as Mean Reciprocal Rank (MRR) and Hits@N, indicating its strength in capturing complex, hierarchical, or symmetric relationships.
• $FMult_n$ performs comparably to state-of-the-art models, particularly excelling where polynomial functions suit the dataset's nature.
• Insights from experiments highlight the importance of properly selecting the degree of polynomial ($n$) and layer complexity in neural networks to balance between overfitting and generalizing effectively.

Implications and Future Work

This paper opens new avenues for KGE by embedding entities and relations in function spaces, thus providing a more versatile and expressive representation framework. The results indicate that functional embeddings can inherently capture more complex dynamics, yielding better performance in link prediction tasks.

Practical Implications:

• Improved performance, especially on datasets with hierarchical or symmetric relationships, suggests potential applications in domains requiring sophisticated relational reasoning, such as biomedical informatics and natural language understanding.

Theoretical Implications:

• The transition from static to dynamic representations challenges existing assumptions in KGE, suggesting a paradigm where embeddings are seen more as evolving representations rather than fixed points in space.

Future research could further explore:

1. The potential of trigonometric-based embeddings ($FMult^i_n$), whose full capacity remains underexplored.
2. Optimization techniques for neural network-based embeddings to alleviate computational overhead in larger datasets.
3. Extensions to incorporate temporal dynamics, enhancing the framework's applicability to time-varying knowledge graphs.

In conclusion, embedding knowledge graphs in function spaces offers a promising new direction, combining mathematical rigor with computational flexibility to better model the intricacies of real-world datasets.