Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Question Calibration and Multi-Hop Modeling for Temporal Question Answering (2402.13188v1)

Published 20 Feb 2024 in cs.CL

Abstract: Many models that leverage knowledge graphs (KGs) have recently demonstrated remarkable success in question answering (QA) tasks. In the real world, many facts contained in KGs are time-constrained thus temporal KGQA has received increasing attention. Despite the fruitful efforts of previous models in temporal KGQA, they still have several limitations. (I) They adopt pre-trained LLMs (PLMs) to obtain question representations, while PLMs tend to focus on entity information and ignore entity transfer caused by temporal constraints, and finally fail to learn specific temporal representations of entities. (II) They neither emphasize the graph structure between entities nor explicitly model the multi-hop relationship in the graph, which will make it difficult to solve complex multi-hop question answering. To alleviate this problem, we propose a novel Question Calibration and Multi-Hop Modeling (QC-MHM) approach. Specifically, We first calibrate the question representation by fusing the question and the time-constrained concepts in KG. Then, we construct the GNN layer to complete multi-hop message passing. Finally, the question representation is combined with the embedding output by the GNN to generate the final prediction. Empirical results verify that the proposed model achieves better performance than the state-of-the-art models in the benchmark dataset. Notably, the Hits@1 and Hits@10 results of QC-MHM on the CronQuestions dataset's complex questions are absolutely improved by 5.1% and 1.2% compared to the best-performing baseline. Moreover, QC-MHM can generate interpretable and trustworthy predictions.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Analyzing the expressive power of graph neural networks in a spectral perspective. In ICLR.
  2. Translating embeddings for modeling multi-relational data. In NeurIPS.
  3. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  4. Time-aware Multiway Adaptive Fusion Network for Temporal KGQA. In ISWC.
  5. CQG: A simple and effective controlled generation framework for multi-hop question generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 6896–6906.
  6. Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering. In EMNLP.
  7. Entities as Experts: Sparse Memory Access with Entity Supervision. In EMNLP.
  8. Learning Sequence Encoders for Temporal Knowledge Graph Completion. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 4816–4821.
  9. Vpalg: Paper-publication prediction with graph neural networks. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 617–626.
  10. Transferring from formal newswire domain with hypernet for twitter pos tagging. In Proceedings of the 2018 conference on empirical methods in natural language processing, 2540–2549.
  11. Inductive representation learning on large graphs. In NeurIPS.
  12. Strategic explanations for a diagnostic consultation system. International Journal of Man-Machine Studies, 20(1): 3–19.
  13. Gpt-gnn: Generative pre-training of graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 1857–1867.
  14. Knowledge graph embedding based question answering. In WSDM.
  15. Tequila: Temporal question answering over knowledge bases. In CIKM.
  16. Complex temporal question answering on knowledge graphs. In Proceedings of the 30th ACM international conference on information & knowledge management, 792–802.
  17. Adam: A method for stochastic optimization. In ICLR.
  18. Semi-supervised classification with graph convolutional networks. In ICLR.
  19. Tensor decompositions for temporal knowledge base completion. In ICLR.
  20. Asynchronous deep interaction network for natural language inference. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2692–2700.
  21. Adaptive multi-attention network incorporating answer information for duplicate question detection. In Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, 95–104.
  22. Joint knowledge graph completion and question answering. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 1098–1108.
  23. Few-shot Node Classification on Attributed Networks with Graph Meta-learning. In SIGIR.
  24. Time-aware multiway adaptive fusion network for temporal knowledge graph question answering. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1–5. IEEE.
  25. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  26. Neural network-based question answering over knowledge graphs on word and character level. In Proceedings of the 26th international conference on World Wide Web, 1211–1220.
  27. Searching for optimal subword tokenization in cross-domain ner. arXiv preprint arXiv:2206.03352.
  28. TempoQR: Temporal Question Reasoning over Knowledge Graphs. In AAAI.
  29. Revisiting graph neural networks: All we have is low-pass filters. arXiv preprint arXiv:1905.09550.
  30. Knowledge Enhanced Contextual Word Representations. In EMNLP-IJCNLP.
  31. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR.
  32. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of Conference on Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing, 3982–3992.
  33. Reasoning about entailment with neural attention. arXiv preprint arXiv:1509.06664.
  34. Question Answering Over Temporal Knowledge Graphs. In Proceedings of the Annual Meeting of the ACL and IJCNLP, 6663–6676.
  35. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, 4498–4507.
  36. Improving Time Sensitivity for Question Answering over Temporal Knowledge Graphs. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 8017–8026. Dublin, Ireland: Association for Computational Linguistics.
  37. Improving Time Sensitivity for Question Answering over Temporal Knowledge Graphs. In ACL.
  38. TwiRGCN: Temporally Weighted Graph Convolution for Question Answering over Temporal Knowledge Graphs. arXiv preprint arXiv:2210.06281.
  39. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Magazine, 30(3): 83–98.
  40. Improving Semantic Matching through Dependency-Enhanced Pre-trained Model with Adaptive Fusion. In Findings of the Association for Computational Linguistics: EMNLP 2022, 45–57. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics.
  41. Multiway Attention Networks for Modeling Sentence Pairs. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18. International Joint Conferences on Artificial Intelligence Organization.
  42. Knowledge graph completion via complex tensor factorization. arXiv preprint arXiv:1702.06879.
  43. Attention is all you need. In Advances in Neural Information Processing Systems.
  44. Graph attention networks. In ICLR.
  45. DABERT: Dual Attention Enhanced BERT for Semantic Matching. In Proceedings of the 29th International Conference on Computational Linguistics, 1645–1654. Gyeongju, Republic of Korea: International Committee on Computational Linguistics.
  46. Reinforcement knowledge graph reasoning for explainable recommendation. In Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval, 285–294.
  47. Dual Path Modeling for Semantic Matching by Perceiving Subtle Conflicts. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1–5. IEEE.
  48. Embedding entities and relations for learning and inference in knowledge bases. In ICLR.
  49. Deep bidirectional language-knowledge graph pretraining. Advances in Neural Information Processing Systems, 35: 37309–37323.
  50. QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 535–546. Online: Association for Computational Linguistics.
  51. Variational reasoning for question answering with knowledge graph. In Proceedings of the AAAI conference on artificial intelligence, volume 32.
  52. Robust Lottery Tickets for Pre-trained Language Models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2211–2224. Dublin, Ireland: Association for Computational Linguistics.
  53. Neural-symbolic models for logical queries on knowledge graphs. In International Conference on Machine Learning, 27454–27478. PMLR.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Chao Xue (16 papers)
  2. Di Liang (21 papers)
  3. Pengfei Wang (176 papers)
  4. Jing Zhang (730 papers)
Citations (3)
View on Semantic Scholar