InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions (1911.00219v3)

Abstract: Most existing knowledge graphs suffer from incompleteness, which can be alleviated by inferring missing links based on known facts. One popular way to accomplish this is to generate low-dimensional embeddings of entities and relations, and use these to make inferences. ConvE, a recently proposed approach, applies convolutional filters on 2D reshapings of entity and relation embeddings in order to capture rich interactions between their components. However, the number of interactions that ConvE can capture is limited. In this paper, we analyze how increasing the number of these interactions affects link prediction performance, and utilize our observations to propose InteractE. InteractE is based on three key ideas -- feature permutation, a novel feature reshaping, and circular convolution. Through extensive experiments, we find that InteractE outperforms state-of-the-art convolutional link prediction baselines on FB15k-237. Further, InteractE achieves an MRR score that is 9%, 7.5%, and 23% better than ConvE on the FB15k-237, WN18RR and YAGO3-10 datasets respectively. The results validate our central hypothesis -- that increasing feature interaction is beneficial to link prediction performance. We make the source code of InteractE available to encourage reproducible research.

• The paper introduces the InteractE model that increases feature interactions via permutation, checkered reshaping, and circular convolution to boost link prediction accuracy in knowledge graphs.
• Empirical evaluations reveal significant improvements over ConvE, with notable gains of 9% to 23% in Mean Reciprocal Rank across multiple datasets.
• The rigorous theoretical analysis and practical insights establish InteractE’s potential to drive future innovations in graph embedding methodologies.

An Examination of InteractE: Enhancing Convolution-Based Knowledge Graph Embeddings

The paper "InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions" explores advancements in knowledge graph completion by promoting rich interactions within convolution-based embeddings. This approach is encapsulated in the proposed model, InteractE, which enhances structure learning capabilities through increased feature interactions. The paper provides rigorous theoretical analysis alongside robust empirical evidence, underscoring the model's efficacy against established baselines such as ConvE.

Within knowledge graphs, the goal of completing the graph by predicting missing links presents an ongoing challenge. Traditional methods like TransE and recent neural approaches like ConvE have each contributed solutions by embedding entities and relations into low-dimensional spaces. However, these approaches have limitations in maximizing the interactions between the embeddings, which can restrict cognitive inferences concerning relational patterns. InteractE addresses these constraints by substantially augmenting the richness of feature interactions, an endeavor that results in improved link prediction accuracies.

Key Methodological Innovations:

1. Feature Permutation: InteractE incorporates a mechanism of feature permutations where different permutations of entity and relation embeddings are processed, thereby capturing diverse interaction patterns. This permutation approach exponentially increases potential feature interactions, offering greater expressive power to the model.
2. Checkered Reshaping: The paper introduces a novel checkered reshaping operation where embeddings are arranged such that no two adjacent cells contain components from the same entity or relation. This setup ensures maximum heterogeneity in interactions, thus allowing convolutional operations to capture more complex relational dynamics.
3. Circular Convolution: InteractE utilizes depth-wise circular convolution, extending standard convolution techniques to enhance feature interaction by capturing a broader interaction spectrum across permutations. This choice is theoretically advantageous, yielding more interactions than traditional convolutional approaches.

The empirical findings are compelling; InteractE demonstrates superior performance against ConvE with notable improvements in Mean Reciprocal Rank (MRR) across datasets FB15k-237, WN18RR, and YAGO3-10. The gains are marked: a 9% improvement in MRR on FB15k-237, 7.5% on WN18RR, and 23% on YAGO3-10, highlighting the model’s robust generalization capabilities and its effective handling of complex interaction patterns within data.

The paper’s core proposition—that increasing heterogeneous feature interactions improves link prediction performance—is convincingly argued both through theoretical proofs and empirical validations. The comprehensive evaluation establishes a correlation between interaction richness and relational inference efficacy, reinforcing the model's theoretical foundations with practical applicability.

Implications and Future Directions:

The potential of InteractE to influence future research in AI and knowledge graph completion is substantial. By emphasizing increased feature interactions, it offers a framework that others might extend, exploring more sophisticated interaction patterns or experimenting with deeper network architectures. Additionally, the principles underlying InteractE could be beneficially adapted to adjacent areas like multi-relational graph neural networks or explainable AI, where the elucidation of latent interactions holds particular significance.

Innovation in graph embeddings continues to be pivotal in leveraging structured information for machine intelligence, and InteractE represents a forward step in refining how models perceive and synthesize relational data. Looking forward, new methodologies integrating InteractE's approach might further discern intricate facets of knowledge representation, optimizing both interpretability and performance in holistic AI applications.