Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Foundation Model for Zero-shot Logical Query Reasoning (2404.07198v2)

Published 10 Apr 2024 in cs.AI and cs.LG

Abstract: Complex logical query answering (CLQA) in knowledge graphs (KGs) goes beyond simple KG completion and aims at answering compositional queries comprised of multiple projections and logical operations. Existing CLQA methods that learn parameters bound to certain entity or relation vocabularies can only be applied to the graph they are trained on which requires substantial training time before being deployed on a new graph. Here we present UltraQuery, the first foundation model for inductive reasoning that can zero-shot answer logical queries on any KG. The core idea of UltraQuery is to derive both projections and logical operations as vocabulary-independent functions which generalize to new entities and relations in any KG. With the projection operation initialized from a pre-trained inductive KG reasoning model, UltraQuery can solve CLQA on any KG after finetuning on a single dataset. Experimenting on 23 datasets, UltraQuery in the zero-shot inference mode shows competitive or better query answering performance than best available baselines and sets a new state of the art on 15 of them.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Query embedding on hyper-relational knowledge graphs. In International Conference on Learning Representations, 2022.
  2. Neural methods for logical reasoning over knowledge graphs. In International Conference on Learning Representations, 2022.
  3. Complex query answering with neural link predictors. In International Conference on Learning Representations, 2021.
  4. Adapting neural link predictors for complex query answering. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=1G7CBp8o7L.
  5. Answering complex logical queries on knowledge graphs via query computation tree optimization. In Proceedings of the 40th International Conference on Machine Learning, pp.  1472–1491, 2023. URL https://proceedings.mlr.press/v202/bai23b.html.
  6. Weisfeiler and leman go relational. In The First Learning on Graphs Conference, 2022. URL https://openreview.net/forum?id=wY_IYhh6pqj.
  7. A neuro-symbolic framework for answering conjunctive queries. arXiv preprint arXiv:2310.04598, 2023.
  8. Generalizing to unseen elements: A survey on knowledge extrapolation for knowledge graphs. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pp.  6574–6582, 2023.
  9. Message passing query embedding. Proceedings of the ICML 2020 Workshop on Graph Representation Learning and Beyond, 2020.
  10. LitCQD: Multi-hop reasoning in incomplete knowledge graphs with numeric literals. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp.  617–633, 2023.
  11. Nodepiece: Compositional and parameter-efficient representations of large knowledge graphs. In International Conference on Learning Representations, 2022a. URL https://openreview.net/forum?id=xMJWUKJnFSw.
  12. Inductive logical query answering in knowledge graphs. In Advances in Neural Information Processing Systems, 2022b.
  13. Towards foundation models for knowledge graph reasoning. arXiv preprint 2310.04562, 2023.
  14. Double equivariance for inductive link prediction for both new nodes and new relation types. In NeurIPS 2023 Workshop: New Frontiers in Graph Learning, 2023.
  15. Knowledge sheaves: A sheaf-theoretic framework for knowledge graph embedding. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics. PMLR, 2023. URL https://proceedings.mlr.press/v206/gebhart23a.html.
  16. Relational message passing for fully inductive knowledge graph completion. In 2023 IEEE 39th International Conference on Data Engineering (ICDE), pp.  1221–1233. IEEE, 2023.
  17. Embedding logical queries on knowledge graphs. In Advances in Neural Information Processing Systems, volume 31, 2018.
  18. A theory of link prediction via relational weisfeiler-leman on knowledge graphs. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=7hLlZNrkt5.
  19. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=SJU4ayYgl.
  20. InGram: Inductive knowledge graph embedding via relation graphs. In Proceedings of the 40th International Conference on Machine Learning, pp.  18796–18809. PMLR, 2023. URL https://proceedings.mlr.press/v202/lee23c.html.
  21. TFLEX: Temporal feature-logic embedding framework for complex reasoning over temporal knowledge graph. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=oaGdsgB18L.
  22. Beta embeddings for multi-hop logical reasoning in knowledge graphs. In Advances in Neural Information Processing Systems, volume 33, 2020.
  23. Query2box: Reasoning over knowledge graphs in vector space using box embeddings. In International Conference on Learning Representations, 2020.
  24. Neural graph reasoning: Complex logical query answering meets graph databases. arXiv preprint arXiv:2303.14617, 2023.
  25. Faithful embeddings for knowledge base queries. In Advances in Neural Information Processing Systems, 2020.
  26. Inductive relation prediction by subgraph reasoning. In International Conference on Machine Learning, pp.  9448–9457. PMLR, 2020.
  27. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd workshop on continuous vector space models and their compositionality, pp.  57–66, 2015.
  28. Complex embeddings for simple link prediction. In International conference on machine learning, pp.  2071–2080. PMLR, 2016.
  29. Analyzing differentiable fuzzy logic operators. Artificial Intelligence, 302, 2022. URL https://www.sciencedirect.com/science/article/pii/S0004370221001533.
  30. Composition-based multi-relational graph convolutional networks. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=BylA_C4tPr.
  31. Graph attention networks. In International Conference on Learning Representations, 2018.
  32. Logical message passing networks with one-hop inference on atomic formulas. In The Eleventh International Conference on Learning Representations, ICLR, 2023. URL https://openreview.net/forum?id=SoyOsp7i_l.
  33. EFOksubscriptEFO𝑘\text{EFO}_{k}EFO start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT-cqa: Towards knowledge graph complex query answering beyond set operation. arXiv preprint arXiv:2307.13701, 2023a.
  34. On existential first order queries inference on knowledge graphs. arXiv preprint arXiv:2304.07063, 2023b.
  35. Labeling trick: A theory of using graph neural networks for multi-node representation learning. In Advances in Neural Information Processing Systems, volume 34, pp.  9061–9073, 2021a.
  36. Knowledge graph reasoning with relational digraph. In Proceedings of the ACM Web Conference 2022, pp.  912–924, 2022.
  37. Adaprop: Learning adaptive propagation for graph neural network based knowledge graph reasoning. In KDD, 2023.
  38. Cone: Cone embeddings for multi-hop reasoning over knowledge graphs. In Advances in Neural Information Processing Systems, 2021b.
  39. An ood multi-task perspective for link prediction with new relation types and nodes. arXiv preprint arXiv:2307.06046, 2023.
  40. Neural bellman-ford networks: A general graph neural network framework for link prediction. Advances in Neural Information Processing Systems, 34:29476–29490, 2021.
  41. Neural-symbolic models for logical queries on knowledge graphs. In International Conference on Machine Learning. PMLR, 2022.
  42. A*net: A scalable path-based reasoning approach for knowledge graphs. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=dAJrxQz1lk.
Citations (1)
View on Semantic Scholar

Summary

  • The paper presents UltraQuery, a foundation model enabling zero-shot logical query reasoning on any knowledge graph without retraining.
  • It leverages an inductive relation projection with transferable meta-graph representations and employs non-parametric fuzzy logic for complex operations.
  • Experimental results show UltraQuery outperforming baselines on 14 of 23 datasets, demonstrating robust adaptability and enhanced efficiency in KG reasoning.

Zero-shot Logical Query Reasoning on Any Knowledge Graph

Introduction to UltraQuery: A Paradigm Shift in Knowledge Graph Reasoning

Knowledge Graphs (KGs) are pivotal in a myriad of applications, from question answering systems to semantic web services, owing to their structured representation of knowledge. Complex Logical Query Answering (CLQA) extends the utility of KGs by enabling the resolution of intricate queries composed of multiple logical operations and projections. However, the static nature of existing approaches, which rely on pre-learned embeddings specific to given entity or relation vocabularies, severely limits their applicability to novel KGs without substantial retraining. This paper introduces UltraQuery, a novel inductive reasoning model capable of zero-shot logical query reasoning on any KG, effectively addressing the adaptability challenge that plagues current CLQA systems.

Theoretical Background and Related Works

Extant CLQA techniques are predominantly transductive, tailored to specific entity-relation vocabularies within training graphs, thereby undermining their utility on unseen graphs. Few attempts towards inductive generalization exist but are constrained by fixed relation vocabularies, failing to extend to KGs with entirely novel entities and relations. In contrast, UltraQuery stands as the first model to tackle the inductive (e,r)(e,r) scenario, where both entities and relations are unfamiliar at inference time, through innovative employment of transferable meta-graph representations and fuzzy logic for logical operations, circumventing the need for external node or edge features.

Methodological Advancements: The UltraQuery Approach

UltraQuery's innovative architecture is grounded on two core components:

  1. Inductive Relation Projection: Leveraging Ultra, a pre-trained inductive KG reasoning model, UltraQuery dynamically constructs relation representations from a meta-graph of relation interactions, enabling the model to generalize across different KGs irrespective of their specific vocabularies.
  2. Inductive Logical Operations: Utilizing non-parametric fuzzy logics, UltraQuery effectively models logical operations (conjunction, disjunction, negation) across fuzzy sets, facilitating adaptable and scalable reasoning over any KG.

This architecture enables UltraQuery to reliably perform CLQA on entirely new KGs, showcasing remarkable zero-shot inference capabilities across 23 diverse datasets, and outperforming existing baselines in 14 of those in terms of both Mean Reciprocal Rank (MRR) and Hits@10 metrics for complex query answering.

Experimental Insights and Practical Implications

Experimental evaluations underscore UltraQuery's superior performance, particularly in the most challenging inductive (e,r)(e,r) scenarios, suggesting a significant leap towards KG-agnostic complex query answering systems. This development not only advances the frontier of KG reasoning but also encourages the adoption of KGs in domains where rapid adaptability to novel knowledge structures is crucial. Moreover, the dual benefits of reducing computational expense and environmental impact, by negating the need for extensive retraining on new datasets, further amplify UltraQuery's utility in real-world applications.

Future Directions and Conclusion

While UltraQuery pioneers in adapting to new KGs without retraining, future explorations might delve into refining the methodology to mitigate challenges like the multi-source propagation issue and exploring the incorporation of temporal dynamics or numerical literals into the CLQA framework. The innovative stride made by UltraQuery in enabling zero-shot logical reasoning on any KG opens new avenues for research and applications, promising a more versatile and responsive landscape for KG-driven technologies.