Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Less is More: One-shot Subgraph Reasoning on Large-scale Knowledge Graphs (2403.10231v1)

Published 15 Mar 2024 in cs.LG, cs.AI, and cs.SI

Abstract: To deduce new facts on a knowledge graph (KG), a link predictor learns from the graph structure and collects local evidence to find the answer to a given query. However, existing methods suffer from a severe scalability problem due to the utilization of the whole KG for prediction, which hinders their promise on large scale KGs and cannot be directly addressed by vanilla sampling methods. In this work, we propose the one-shot-subgraph link prediction to achieve efficient and adaptive prediction. The design principle is that, instead of directly acting on the whole KG, the prediction procedure is decoupled into two steps, i.e., (i) extracting only one subgraph according to the query and (ii) predicting on this single, query dependent subgraph. We reveal that the non-parametric and computation-efficient heuristics Personalized PageRank (PPR) can effectively identify the potential answers and supporting evidence. With efficient subgraph-based prediction, we further introduce the automated searching of the optimal configurations in both data and model spaces. Empirically, we achieve promoted efficiency and leading performances on five large-scale benchmarks. The code is publicly available at: https://github.com/tmlr-group/one-shot-subgraph.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261, 2018.
  2. Hyperopt: A python library for optimizing the hyperparameters of machine learning algorithms. In Proceedings of the 12th Python in science conference, 2013.
  3. Translating embeddings for modeling multi-relational data. In NeurIPS, 2013.
  4. L. Breiman. Random forests. ML, 45(1):5–32, 2001.
  5. Unifying knowledge graph learning and recommendation: Towards a better understanding of user preferences. In TheWebConf, 2019.
  6. Fastgcn: Fast learning with graph convolutional networks via importance sampling. In ICLR, 2018.
  7. A review: Knowledge reasoning over knowledge graph. Expert Systems with Applications, 141:112948, 2020.
  8. Understanding and improving graph injection attack by promoting unnoticeability. In ICLR, 2022.
  9. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems, 2016.
  10. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems, 2016.
  11. Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. In ICLR, 2017.
  12. Convolutional 2D knowledge graph embeddings. In AAAI, 2017.
  13. P. Diaconis and S. Janson. Graph limits and exchangeable random graphs. arXiv preprint arXiv:0712.2749, 2007.
  14. Nodepiece: Compositional and parameter-efficient representations of large knowledge graphs. In ICLR, 2022.
  15. Towards foundation models for knowledge graph reasoning. arXiv preprint arXiv:2310.04562, 2023.
  16. M. Gardner and S. Dorling. Artificial neural networks (the multilayer perceptron) – a review of applications in the atmospheric sciences. Atmospheric environment, 32(14-15):2627–2636, 1998.
  17. Why do tree-based models still outperform deep learning on tabular data? arXiv preprint arXiv:2207.08815, 2022.
  18. Inductive representation learning on large graphs. In NeurIPS, 2017.
  19. Open graph benchmark: Datasets for machine learning on graphs. In NeurIPS, 2020.
  20. Knowledge graph embedding based question answering. In WSDM, 2019.
  21. Sequential model-based optimization for general algorithm configuration. In ICLIO, 2011.
  22. G. Jeh and J. Widom. Scaling personalized web search. In WWW, 2003.
  23. A survey on knowledge graphs: Representation, acquisition and applications. arxiv preprint arXiv:2002.00388, 2020.
  24. Learning defense transformations for counterattacking adversarial examples. Neural Networks, 164:177–185, 2023.
  25. Communicative message passing for inductive relation reasoning. In AAAI, 2021.
  26. Generalization analysis of message passing neural networks on large random graphs. In NeurIPS, 2022.
  27. Interpretable and generalizable graph learning via stochastic attention mechanism. In ICML, 2022.
  28. Locality-aware subgraphs for inductive link prediction in knowledge graphs. Pattern Recognition Letters, 2023.
  29. K. Oono and T. Suzuki. Graph neural networks exponentially lose expressive power for node classification. In ICLR, 2019.
  30. The pagerank citation ranking: Bringing order to the web. Technical report, Stanford infolab, 1999.
  31. Automatic differentiation in PyTorch. In ICLR, 2017.
  32. Rnnlogic: Learning logic rules for reasoning on knowledge graphs. In ICLR, 2021.
  33. Drum: End-to-end differentiable rule mining on knowledge graphs. In NeurIPS, 2019.
  34. Modeling relational data with graph convolutional networks. In ESWC, 2018.
  35. Lmc: Fast training of gnns via subgraph sampling with provable convergence. arXiv preprint arXiv:2302.00924, 2023.
  36. Yago: A core of semantic knowledge. In TheWebConf, pp.  697–706, 2007.
  37. Rotate: Knowledge graph embedding by relational rotation in complex space. In ICLR, 2019.
  38. Gossipfl: A decentralized federated learning framework with sparsified and adaptive communication. IEEE Transactions on Parallel and Distributed Systems, 2022a.
  39. Virtual homogeneity learning: Defending against data heterogeneity in federated learning. In ICML, 2022b.
  40. Inductive relation prediction by subgraph reasoning. In ICML, 2020.
  41. Composition-based multi-relational graph convolutional networks. In ICLR, 2019.
  42. Knowledge graph embedding: A survey of approaches and applications. TKDE, 2017.
  43. Kgat: Knowledge graph attention network for recommendation. In SIGKDD, 2019.
  44. C. Williams and C. Rasmussen. Gaussian processes for regression. In NIPS, 1995.
  45. Deeppath: A reinforcement learning method for knowledge graph reasoning. In EMNLP, 2017.
  46. Dynamically pruned message passing networks for large-scale knowledge graph reasoning. In ICLR, 2019.
  47. Qa-gnn: Reasoning with language models and knowledge graphs for question answering. In NAACL, 2021.
  48. Sumgnn: multi-typed drug interaction prediction via efficient knowledge graph summarization. Bioinformatics, 37(18):2988–2995, 2021.
  49. Quaternion knowledge graph embeddings. In NeurIPS, 2019.
  50. Detecting adversarial data by probing multiple perturbations using expected perturbation score. In ICML, 2023a.
  51. Y. Zhang and Q. Yao. Knowledge graph reasoning with relational directed graph. In TheWebConf, 2022.
  52. Adversarial robustness through the lens of causality. In ICLR, 2022a.
  53. Efficient hyper-parameter search for knowledge graph embedding. In ACL, 2022b.
  54. Emerging drug interaction prediction enabled by a flow-based graph neural network with biomedical network. Nature Computational Science, 3(12):1023–1033, 2023b.
  55. Adaprop: Learning adaptive propagation for knowledge graph reasoning. In SIGKDD, 2023c.
  56. Ood link prediction generalization capabilities of message-passing gnns in larger test graphs. In NeurIPS, 2022.
  57. Combating bilateral edge noise for robust link prediction. In NeurIPS, 2023a.
  58. On strengthening and defending graph reconstruction attack with markov chain approximation. In ICML, 2023b.
  59. Neural bellman-ford networks: A general graph neural network framework for link prediction. In NeurIPS, 2021.
  60. A*net: A scalable path-based reasoning approach for knowledge graphs. In NeurIPS, 2023.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets