Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

EntailE: Introducing Textual Entailment in Commonsense Knowledge Graph Completion (2402.09666v1)

Published 15 Feb 2024 in cs.CL

Abstract: Commonsense knowledge graph completion is a new challenge for commonsense knowledge graph construction and application. In contrast to factual knowledge graphs such as Freebase and YAGO, commonsense knowledge graphs (CSKGs; e.g., ConceptNet) utilize free-form text to represent named entities, short phrases, and events as their nodes. Such a loose structure results in large and sparse CSKGs, which makes the semantic understanding of these nodes more critical for learning rich commonsense knowledge graph embedding. While current methods leverage semantic similarities to increase the graph density, the semantic plausibility of the nodes and their relations are under-explored. Previous works adopt conceptual abstraction to improve the consistency of modeling (event) plausibility, but they are not scalable enough and still suffer from data sparsity. In this paper, we propose to adopt textual entailment to find implicit entailment relations between CSKG nodes, to effectively densify the subgraph connecting nodes within the same conceptual class, which indicates a similar level of plausibility. Each node in CSKG finds its top entailed nodes using a finetuned transformer over natural language inference (NLI) tasks, which sufficiently capture textual entailment signals. The entailment relation between these nodes are further utilized to: 1) build new connections between source triplets and entailed nodes to densify the sparse CSKGs; 2) enrich the generalization ability of node representations by comparing the node embeddings with a contrastive loss. Experiments on two standard CSKGs demonstrate that our proposed framework EntailE can improve the performance of CSKG completion tasks under both transductive and inductive settings.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. E. Davis and G. Marcus, “Commonsense reasoning and commonsense knowledge in artificial intelligence,” Communications of the ACM, vol. 58, no. 9, pp. 92–103, 2015.
  2. M. Sap, H. Rashkin, D. Chen, R. Le Bras, and Y. Choi, “Social IQa: Commonsense reasoning about social interactions,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).   Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 4463–4473. [Online]. Available: https://aclanthology.org/D19-1454
  3. T. Young, E. Cambria, I. Chaturvedi, H. Zhou, S. Biswas, and M. Huang, “Augmenting end-to-end dialogue systems with commonsense knowledge,” in Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, S. A. McIlraith and K. Q. Weinberger, Eds.   AAAI Press, 2018, pp. 4970–4977. [Online]. Available: https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16573
  4. S. Gao, J. D. Hwang, S. Kanno, H. Wakaki, Y. Mitsufuji, and A. Bosselut, “Comfact: A benchmark for linking contextual commonsense knowledge,” arXiv preprint arXiv:2210.12678, 2022.
  5. R. Speer, J. Chin, and C. Havasi, “Conceptnet 5.5: An open multilingual graph of general knowledge,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, no. 1, 2017.
  6. M. Sap, R. Le Bras, E. Allaway, C. Bhagavatula, N. Lourie, H. Rashkin, B. Roof, N. A. Smith, and Y. Choi, “Atomic: An atlas of machine commonsense for if-then reasoning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp. 3027–3035.
  7. C. Malaviya, C. Bhagavatula, A. Bosselut, and Y. Choi, “Commonsense knowledge base completion with structural and semantic context,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 03, 2020, pp. 2925–2933.
  8. C. Shang, Y. Tang, J. Huang, J. Bi, X. He, and B. Zhou, “End-to-end structure-aware convolutional networks for knowledge base completion,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp. 3060–3067.
  9. G. A. Miller, “Wordnet: A lexical database for english,” Commun. ACM, vol. 38, no. 11, pp. 39–41, 1995. [Online]. Available: http://doi.acm.org/10.1145/219717.219748
  10. K. D. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor, “Freebase: a collaboratively created graph database for structuring human knowledge,” in Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2008, Vancouver, BC, Canada, June 10-12, 2008, J. T. Wang, Ed.   ACM, 2008, pp. 1247–1250. [Online]. Available: https://doi.org/10.1145/1376616.1376746
  11. B. Wang, G. Wang, J. Huang, J. You, J. Leskovec, and C.-C. J. Kuo, “Inductive learning on commonsense knowledge graph completion,” arXiv preprint arXiv:2009.09263, 2020.
  12. T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in International Conference on Learning Representations.
  13. M. Schlichtkrull, T. N. Kipf, P. Bloem, R. Van Den Berg, I. Titov, and M. Welling, “Modeling relational data with graph convolutional networks,” in The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings 15.   Springer, 2018, pp. 593–607.
  14. Y. Wilks, “A preferential, pattern-seeking, semantics for natural language inference,” Artificial intelligence, vol. 6, no. 1, pp. 53–74, 1975.
  15. P. Pantel, R. Bhagat, B. Coppola, T. Chklovski, and E. Hovy, “Isp: Learning inferential selectional preferences,” in Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, 2007, pp. 564–571.
  16. H. Zhang, Y. Song, Y. Song, and D. Yu, “Knowledge-aware pronoun coreference resolution,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 867–876.
  17. W. Wu, H. Li, H. Wang, and K. Zhu, “Probase: a probabilistic taxonomy for text understanding,” in International Conference on Management of Data, 2012.
  18. C. Yu, H. Zhang, Y. Song, W. Ng, and L. Shang, “Enriching large-scale eventuality knowledge graph with entailment relations,” in Automated Knowledge Base Construction.
  19. I. Porada, K. Suleman, A. Trischler, and J. C. K. Cheung, “Modeling event plausibility with consistent conceptual abstraction,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, pp. 1732–1743.
  20. J. M. Zacks and B. Tversky, “Event structure in perception and conception.” Psychological bulletin, vol. 127, no. 1, p. 3, 2001.
  21. Z. Chen, Y. Feng, and D. Zhao, “Entailment graph learning with textual entailment and soft transitivity,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 5899–5910.
  22. B. MacCartney and C. D. Manning, “Modeling semantic containment and exclusion in natural language inference,” in Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), 2008, pp. 521–528.
  23. R. Rudinger, V. Shwartz, J. D. Hwang, C. Bhagavatula, M. Forbes, R. Le Bras, N. A. Smith, and Y. Choi, “Thinking like a skeptic: Defeasible inference in natural language,” in Findings of the Association for Computational Linguistics: EMNLP 2020, 2020, pp. 4661–4675.
  24. Suchanek, M. Fabian, Kasneci, Gjergji, and Gerhard, “Yago: a core of semantic knowledge,” in Proceedings of the 16th International Conference on World Wide Web, WWW 2007, Banff, Alberta, Canada, May 8-12, 2007, 2007.
  25. S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. G. Ives, “Dbpedia: A nucleus for a web of open data,” in The Semantic Web, 6th International Semantic Web Conference, 2nd Asian Semantic Web Conference, ISWC 2007 + ASWC 2007, Busan, Korea, November 11-15, 2007, ser. Lecture Notes in Computer Science, K. Aberer, K. Choi, N. F. Noy, D. Allemang, K. Lee, L. J. B. Nixon, J. Golbeck, P. Mika, D. Maynard, R. Mizoguchi, G. Schreiber, and P. Cudré-Mauroux, Eds., vol. 4825.   Springer, 2007, pp. 722–735. [Online]. Available: https://doi.org/10.1007/978-3-540-76298-0\_52
  26. H. Zhang, D. Khashabi, Y. Song, and D. Roth, “Transomcs: From linguistic graphs to commonsense knowledge,” 2020.
  27. H. Zhang, X. Liu, H. Pan, Y. Song, and C. W.-K. Leung, “Aser: A large-scale eventuality knowledge graph,” in Proceedings of The Web Conference 2020, 2020, pp. 201–211.
  28. A. Bosselut, H. Rashkin, M. Sap, C. Malaviya, A. Celikyilmaz, and Y. Choi, “Comet: Commonsense transformers for automatic knowledge graph construction,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 4762–4779.
  29. Y. Su, Z. Wang, T. Fang, H. Zhang, Y. Song, and T. Zhang, “Mico: A multi-alternative contrastive learning framework for commonsense knowledge representation,” arXiv preprint arXiv:2210.07570, 2022.
  30. J. WIEBE, T. WILSON, and C. CARDIE, “Annotating expressions of opinions and emotions in language,” Language resources and evaluation, vol. 39, no. 2-3, pp. 165–210, 2005.
  31. M. Marelli, S. Menini, M. Baroni, L. Bentivogli, R. Bernardi, and R. Zamparelli, “A sick cure for the evaluation of compositional distributional semantic models,” in Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), 2014, pp. 216–223.
  32. D. Cer, M. Diab, E. Agirre, I. Lopez-Gazpio, and L. Specia, “Semeval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation,” in Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), 2017, pp. 1–14.
  33. Z. Sun, C. Fan, Q. Han, X. Sun, Y. Meng, F. Wu, and J. Li, “Self-explaining structures improve nlp models,” arXiv preprint arXiv:2012.01786, 2020.
  34. S. Bowman, G. Angeli, C. Potts, and C. D. Manning, “A large annotated corpus for learning natural language inference,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015, pp. 632–642.
  35. A. Williams, N. Nangia, and S. Bowman, “A broad-coverage challenge corpus for sentence understanding through inference,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 2018, pp. 1112–1122.
  36. Z. Zhang, Y. Wu, H. Zhao, Z. Li, S. Zhang, X. Zhou, and X. Zhou, “Semantics-aware bert for language understanding,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, 2020, pp. 9628–9635.
  37. S. Wang, H. Fang, M. Khabsa, H. Mao, and H. Ma, “Entailment as few-shot learner,” arXiv preprint arXiv:2104.14690, 2021.
  38. T. Gao, X. Yao, and D. Chen, “Simcse: Simple contrastive learning of sentence embeddings,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 6894–6910.
  39. C. Bhagavatula, R. L. Bras, C. Malaviya, K. Sakaguchi, A. Holtzman, H. Rashkin, D. Downey, S. W.-t. Yih, and Y. Choi, “Abductive commonsense reasoning,” arXiv preprint arXiv:1908.05739, 2019.
  40. W. Wang, T. Fang, B. Xu, C. Y. L. Bo, Y. Song, and L. Chen, “Cat: A contextualized conceptualization and instantiation framework for commonsense reasoning,” arXiv preprint arXiv:2305.04808, 2023.
  41. B. Van Durme, P. Michalak, and L. Schubert, “Deriving generalized knowledge from corpora using wordnet abstraction,” in Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009), 2009, pp. 808–816.
  42. Y. Gong, K. Zhao, and K. Q. Zhu, “Representing verbs as argument concepts,” in Thirtieth AAAI Conference on Artificial Intelligence, 2016.
  43. M. He, Y. Song, K. Xu, and D. Yu, “On the role of conceptualization in commonsense knowledge graph construction,” arXiv preprint arXiv:2003.03239, 2020.
  44. H. Yanaka, K. Mineshima, D. Bekki, K. Inui, S. Sekine, L. Abzianidze, and J. Bos, “Can neural networks understand monotonicity reasoning?” ACL 2019, p. 31, 2019.
  45. E. Goodwin, K. Sinha, and T. J. O’Donnell, “Probing linguistic systematicity,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault, Eds.   Online: Association for Computational Linguistics, Jul. 2020, pp. 1958–1969. [Online]. Available: https://aclanthology.org/2020.acl-main.177
  46. A. Geiger, K. Richardson, and C. Potts, “Neural natural language inference models partially embed theories of lexical entailment and negation,” in Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, A. Alishahi, Y. Belinkov, G. Chrupała, D. Hupkes, Y. Pinter, and H. Sajjad, Eds.   Online: Association for Computational Linguistics, Nov. 2020, pp. 163–173. [Online]. Available: https://aclanthology.org/2020.blackboxnlp-1.16
  47. T. Dettmers, P. Minervini, P. Stenetorp, and S. Riedel, “Convolutional 2d knowledge graph embeddings,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018.
  48. B. Yang, S. W.-t. Yih, X. He, J. Gao, and L. Deng, “Embedding entities and relations for learning and inference in knowledge bases,” in Proceedings of the International Conference on Learning Representations (ICLR) 2015, 2015.
  49. T. Trouillon, J. Welbl, S. Riedel, É. Gaussier, and G. Bouchard, “Complex embeddings for simple link prediction,” in International conference on machine learning.   PMLR, 2016, pp. 2071–2080.
  50. Z. Sun, Z.-H. Deng, J.-Y. Nie, and J. Tang, “Rotate: Knowledge graph embedding by relational rotation in complex space,” in International Conference on Learning Representations, 2018.
  51. A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, and O. Yakhnenko, “Translating embeddings for modeling multi-relational data,” in Neural Information Processing Systems (NIPS), 2013, pp. 1–9.
  52. J. D. M.-W. C. Kenton and L. K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of NAACL-HLT, 2019, pp. 4171–4186.
  53. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in ICLR (Poster), 2015.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Ying Su (42 papers)
  2. Tianqing Fang (43 papers)
  3. Huiru Xiao (4 papers)
  4. Weiqi Wang (58 papers)
  5. Yangqiu Song (196 papers)
  6. Tong Zhang (569 papers)
  7. Lei Chen (484 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets