Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings (2002.05969v2)

Abstract: Answering complex logical queries on large-scale incomplete knowledge graphs (KGs) is a fundamental yet challenging task. Recently, a promising approach to this problem has been to embed KG entities as well as the query into a vector space such that entities that answer the query are embedded close to the query. However, prior work models queries as single points in the vector space, which is problematic because a complex query represents a potentially large set of its answer entities, but it is unclear how such a set can be represented as a single point. Furthermore, prior work can only handle queries that use conjunctions ($\wedge$) and existential quantifiers ($\exists$). Handling queries with logical disjunctions ($\vee$) remains an open problem. Here we propose query2box, an embedding-based framework for reasoning over arbitrary queries with $\wedge$, $\vee$, and $\exists$ operators in massive and incomplete KGs. Our main insight is that queries can be embedded as boxes (i.e., hyper-rectangles), where a set of points inside the box corresponds to a set of answer entities of the query. We show that conjunctions can be naturally represented as intersections of boxes and also prove a negative result that handling disjunctions would require embedding with dimension proportional to the number of KG entities. However, we show that by transforming queries into a Disjunctive Normal Form, query2box is capable of handling arbitrary logical queries with $\wedge$, $\vee$, $\exists$ in a scalable manner. We demonstrate the effectiveness of query2box on three large KGs and show that query2box achieves up to 25% relative improvement over the state of the art.

• The paper introduces box embeddings that represent queries as hyper-rectangles, overcoming the limitations of traditional point-based models.
• It systematically models logical operations such as projection, intersection, and disjunction through translations, scaling, and attention mechanisms.
• Experiments on FB15k, FB15k-237, and NELL995 demonstrate up to 25% performance gains, validating its superior generalization in query reasoning.

An Overview of "Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings"

The paper "Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings" introduces a novel approach to handle logical queries over incomplete knowledge graphs (KGs) by representing queries as boxes (hyper-rectangles) in a vector space. This research addresses the challenge of answering complex logical queries, such as those in first-order logic, over large-scale and incomplete KGs, which are crucial for knowledge base reasoning and question answering applications.

Core Contributions

The authors propose a method, referred to as Query2box (), that advances the current state-of-the-art in several ways:

1. Box Embeddings: Unlike existing models that represent queries as single points, the authors argue for and utilize box embeddings, which can naturally enclose sets of entities. This representation addresses the limitations of the point-based models which struggle with defining logical operations such as set intersections.
2. Modeling Logical Operators: provides a systematic approach to model logical operations in vector space. Specifically:
   • Projection is modeled by translating and scaling boxes, corresponding to the logical progression of a query in a KG.
   • Intersection leverages an attention mechanism to create intersections of boxes, thus capturing common entities in logical conjunctions.
3. Handling Disjunctions: The paper tackles the challenge of incorporating disjunctions (logical 'or') into query embeddings, a task known to require embedding dimensions proportional to the KG's size. By transforming queries into their Disjunctive Normal Form (DNF), efficiently handles arbitrary Existential Positive First-order (EPFO) queries in a scalable manner.
4. Empirical Validation: Through extensive experiments on standard KGs like FB15k, FB15k-237, and NELL995, the framework shows up to 25% relative improvement over existing methods. It highlights s ability to generalize to new query structures, even those not encountered during training.

Implications and Future Prospects

The implications of this work are significant both theoretically and practically. By leveraging the spatial properties of box embeddings, bridges an essential gap in graph-based machine learning tasks, providing a robust method for representing and reasoning about complex query semantics. Practically, this has potential applications in areas which rely on efficient query answering over large, sparse, and incomplete datasets.

This paper lays the groundwork for several future explorations. One potential direction is optimizing the computational efficiency further, especially considering real-time applications. Additionally, exploring other geometric shapes or extending the model to handle more complex logical operations could enhance the expressive power of vectorized logical reasoning.

Query2box represents a step forward in embedding-based KG reasoning, showing promising results in effectively handling syntactically complex queries while highlighting the potential advantages and flexibility introduced by non-point embeddings.

In conclusion, this research adds a valuable layer of logical sophistication to the field of knowledge graphs, fostering new opportunities for advancements in AI systems' deductive capabilities.