Traversing Knowledge Graphs in Vector Space

Abstract: Path queries on a knowledge graph can be used to answer compositional questions such as "What languages are spoken by people living in Lisbon?". However, knowledge graphs often have missing facts (edges) which disrupts path queries. Recent models for knowledge base completion impute missing facts by embedding knowledge graphs in vector spaces. We show that these models can be recursively applied to answer path queries, but that they suffer from cascading errors. This motivates a new "compositional" training objective, which dramatically improves all models' ability to answer path queries, in some cases more than doubling accuracy. On a standard knowledge base completion task, we also demonstrate that compositional training acts as a novel form of structural regularization, reliably improving performance across all base models (reducing errors by up to 43%) and achieving new state-of-the-art results.

• The paper introduces a compositional vector space model that mitigates cascading errors in path queries over incomplete knowledge graphs.
• It proposes a novel training objective that recursively composes vector transformations to preserve multi-step relational information.
• Empirical results on WordNet and Freebase show that the approach more than doubles query performance and reduces errors by up to 43% compared to previous methods.

Traversing Knowledge Graphs in Vector Space: A Compositional Approach

This paper presents a novel methodology for performing path queries on knowledge graphs by implementing a compositional vector space model to more effectively manage graph incompleteness. Knowledge graphs often possess incomplete edges, which challenge standard path queries. The authors demonstrate that recent advancements in knowledge base completion (KBC) can be leveraged to perform recursive path queries; however, these methods are prone to cascading errors. To address this problem, a new "compositional" training objective is proposed, which significantly enhances the models’ abilities to answer path queries and also serves as an avenue for structural regularization in KBC, achieving substantial improvements over state-of-the-art methods.

Core Contributions

1. Compositional Vector Space Model for Path Queries: The paper defines a framework for using vector space models to answer path queries by defining recursive transformations that can predict paths via vector compositions. This method interprets the embedded knowledge graph's structure and enhances the efficacy of path queries.
2. Training Objective: A new training objective is proposed that focuses on reducing cascading errors inherent in previous models. This approach trains vector transformations to preserve the compositional path information throughout the entire path, enhancing the model's ability to answer complex queries up to length 5 better than prior approaches.
3. Empirical Results: The application of this compositional training objective yields remarkable accuracy improvements in path query performance, more than doubling it in some cases. Additionally, it achieves new state-of-the-art results on standard KBC tasks by consistently reducing errors across various models by up to 43%.

Methodology

The efficacy of the compositional vector space model is evaluated using datasets derived from WordNet and Freebase. The models are first trained to perform on graph paths of varying lengths through recursive application of traversal operators. Training includes paths that form random walks on the graph, ensuring diverse types of path queries that expand the model’s generalization capabilities.

For evaluation, mean quantile and hits at 10 are used as metrics, with improvements in both metrics demonstrating the efficacy of the new compositional training objective over single-edge training across multiple base models, including TransE and Bilinear-Diag.

Key Findings

• Compositional training reduces errors in path querying by addressing downstream cascading errors, a phenomenon where slight inaccuracies in initial stages of path evaluation exacerbate errors at later stages.
• Models employing compositional training outperform previous state-of-the-art models in KBC tasks, suggesting that paths do more than just traverse existing edges—they also inform the structural generalizations for learning other relations.

Implications and Future Work

The introduction of compositional training provides a robust avenue for structural regularization of vector space models, suggesting that this framework could be extended beyond relation modeling to more complex inference tasks in AI. Future work could focus on further refining this approach, exploring its applications in more complex knowledge graphs, and integrating these advancements into real-world systems where robust knowledge inference is critical.

In conclusion, by investigating new ways to conduct inference over knowledge graphs, this paper marks a significant step in optimizing knowledge representation techniques, particularly through the lens of vector compositions and recursive query handling. The proposed framework stands as a versatile model with promising implications for the development of more intelligent, comprehensive systems aimed at overcoming data sparsity in knowledge-based applications.