Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Chain-of-History Reasoning for Temporal Knowledge Graph Forecasting (2402.14382v2)

Published 22 Feb 2024 in cs.CL

Abstract: Temporal Knowledge Graph (TKG) forecasting aims to predict future facts based on given histories. Most recent graph-based models excel at capturing structural information within TKGs but lack semantic comprehension abilities. Nowadays, with the surge of LLMs, the LLM-based TKG prediction model has emerged. However, the existing LLM-based model exhibits three shortcomings: (1) It only focuses on the first-order history for prediction while ignoring high-order historical information, resulting in the provided information for LLMs being extremely limited. (2) LLMs struggle with optimal reasoning performance under heavy historical information loads. (3) For TKG prediction, the temporal reasoning capability of LLM alone is limited. To address the first two challenges, we propose Chain-of-History (CoH) reasoning which explores high-order histories step-by-step, achieving effective utilization of high-order historical information for LLMs on TKG prediction. To address the third issue, we design CoH as a plug-and-play module to enhance the performance of graph-based models for TKG prediction. Extensive experiments on three datasets and backbones demonstrate the effectiveness of CoH.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Can large language models be good path planners? a benchmark and investigation on spatial-temporal reasoning. arXiv preprint arXiv:2310.03249.
  2. Icews coded event data. Harvard Dataverse, 12.
  3. Multi-granularity temporal question answering over knowledge graphs. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11378–11392.
  4. Zero-shot relational learning on temporal knowledge graphs with large language models. arXiv preprint arXiv:2311.10112.
  5. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323.
  6. Learning joint structural and temporal contextualized knowledge embeddings for temporal knowledge graph completion. In Findings of the Association for Computational Linguistics: ACL 2023, pages 417–430.
  7. Learning sequence encoders for temporal knowledge graph completion. arXiv preprint arXiv:1809.03202.
  8. Explainable subgraph reasoning for forecasting on temporal knowledge graphs. In ICLR.
  9. Learning neural ordinary equations for forecasting future links on temporal knowledge graphs. In EMNLP, pages 8352–8364.
  10. Enhanced temporal knowledge embeddings with contextualized language representations.
  11. Graph hawkes neural network for forecasting on temporal knowledge graphs. In AKBC.
  12. Do language models have a common sense regarding time? revisiting temporal commonsense reasoning in the era of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6750–6774.
  13. Mixtral of experts. arXiv preprint arXiv:2401.04088.
  14. Structgpt: A general framework for large language model to reason over structured data. arXiv e-prints, pages arXiv–2305.
  15. Recurrent event network: Autoregressive structure inferenceover temporal knowledge graphs. In EMNLP, pages 6669–6683.
  16. Recurrent event network: Autoregressive structure inference over temporal knowledge graphs. arXiv preprint arXiv:1904.05530.
  17. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  18. Temporal knowledge graph forecasting without knowledge using in-context learning. arXiv preprint arXiv:2305.10613.
  19. Tirgn: Time-guided recurrent graph network with local-global historical patterns for temporal knowledge graph reasoning. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI 2022, Vienna, Austria, 23-29 July 2022, pages 2152–2158.
  20. Search from history and reason for future: Two-stage reasoning on temporal knowledge graphs. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4732–4743.
  21. Temporal knowledge graph reasoning based on evolutional representation learning. In SIGIR, pages 408–417.
  22. Learn from relational correlations and periodic events for temporal knowledge graph reasoning. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1559–1568.
  23. Tlogic: Temporal logical rules for explainable link forecasting on temporal knowledge graphs. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 4120–4127.
  24. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32.
  25. Language models can improve event prediction by few-shot abductive reasoning. arXiv preprint arXiv:2305.16646.
  26. TimeTraveler: Reinforcement learning for temporal knowledge graph forecasting. In EMNLP, pages 8306–8319.
  27. Towards benchmarking and improving the temporal reasoning capability of large language models. arXiv preprint arXiv:2306.08952.
  28. Know-evolve: Deep temporal reasoning for dynamic knowledge graphs. In ICML, pages 3462–3471.
  29. Yuqing Wang and Yun Zhao. 2023. Tram: Benchmarking temporal reasoning for large language models. arXiv preprint arXiv:2310.00835.
  30. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771.
  31. Metatkg: Learning evolutionary meta-knowledge for temporal knowledge graph reasoning. In EMNLP, pages 7230–7240.
  32. Temporal and heterogeneous graph neural network for financial time series prediction. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 3584–3593.
  33. Pre-trained language model with prompts for temporal knowledge graph completion. arXiv preprint arXiv:2305.07912.
  34. Temporal knowledge graph reasoning with historical contrastive learning. In AAAI.
  35. Back to the future: Towards explainable temporal reasoning with large language models. arXiv preprint arXiv:2310.01074.
  36. Learning latent relations for temporal knowledge graph reasoning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12617–12631.
  37. Learning from history: Modeling temporal knowledge graphs with sequential copy-generation networks. In AAAI, pages 4732–4740.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Yuwei Xia (2 papers)
  2. Ding Wang (71 papers)
  3. Qiang Liu (405 papers)
  4. Liang Wang (512 papers)
  5. Shu Wu (109 papers)
  6. Xiaoyu Zhang (144 papers)
Citations (2)
View on Semantic Scholar