Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

KETM:A Knowledge-Enhanced Text Matching method (2308.06235v1)

Published 11 Aug 2023 in cs.CL

Abstract: Text matching is the task of matching two texts and determining the relationship between them, which has extensive applications in natural language processing tasks such as reading comprehension, and Question-Answering systems. The mainstream approach is to compute text representations or to interact with the text through attention mechanism, which is effective in text matching tasks. However, the performance of these models is insufficient for texts that require commonsense knowledge-based reasoning. To this end, in this paper, We introduce a new model for text matching called the Knowledge Enhanced Text Matching model (KETM), to enrich contextual representations with real-world common-sense knowledge from external knowledge sources to enhance our model understanding and reasoning. First, we use Wiktionary to retrieve the text word definitions as our external knowledge. Secondly, we feed text and knowledge to the text matching module to extract their feature vectors. The text matching module is used as an interaction module by integrating the encoder layer, the co-attention layer, and the aggregation layer. Specifically, the interaction process is iterated several times to obtain in-depth interaction information and extract the feature vectors of text and knowledge by multi-angle pooling. Then, we fuse text and knowledge using a gating mechanism to learn the ratio of text and knowledge fusion by a neural network that prevents noise generated by knowledge. After that, experimental validation on four datasets are carried out, and the experimental results show that our proposed model performs well on all four datasets, and the performance of our method is improved compared to the base model without adding external knowledge, which validates the effectiveness of our proposed method. The code is available at https://github.com/1094701018/KETM

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. K. Jiang, Y. Zhao, R. Cui, and Z. Zhang, “Deim: An effective deep encoding and interaction model for sentence matching,” arXiv preprint arXiv:2203.10482, 2022.
  2. S. Sugawara, P. Stenetorp, K. Inui, and A. Aizawa, “Assessing the benchmarking capacity of machine reading comprehension datasets,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, 2020, pp. 8918–8927.
  3. B. Liu, H. Wei, D. Niu, H. Chen, and Y. He, “Asking questions the human way: Scalable question-answer generation from text corpus,” in Proceedings of The Web Conference 2020, 2020, pp. 2032–2043.
  4. F. Li, Y. Zhao, F. Yang, and R. Cui, “Incorporating translation quality estimation into chinese-korean neural machine translation,” in China National Conference on Chinese Computational Linguistics.   Springer, 2021, pp. 45–57.
  5. S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning, “A large annotated corpus for learning natural language inference,” in Conference on Empirical Methods in Natural Language Processing, EMNLP 2015.   Association for Computational Linguistics (ACL), 2015, pp. 632–642.
  6. A. Williams, N. Nangia, and S. R. Bowman, “A broad-coverage challenge corpus for sentence understanding through inference,” in NAACL-HLT, 2018.
  7. G. Shen, Z.-H. Deng, T. Huang, and X. Chen, “Learning to compose over tree structures via pos tags for sentence representation,” Expert Systems with Applications, vol. 141, p. 112917, 2020.
  8. G. Zhou, Y. Zhou, T. He, and W. Wu, “Learning semantic representation with neural networks for community question answering retrieval,” Knowledge-Based Systems, vol. 93, pp. 75–83, 2016.
  9. R. Yang, J. Zhang, X. Gao, F. Ji, and H. Chen, “Simple and effective text matching with richer alignment features,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 4699–4709.
  10. S. Kim, I. Kang, and N. Kwak, “Semantic sentence matching with densely-connected recurrent and co-attentive information,” in Proceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 6586–6593.
  11. K. Jiang, Y. Zhao, and R. Cui, “Difm: An effective deep interaction and fusion model for sentence matching,” in Chinese Computational Linguistics: 21st China National Conference, CCL 2022, Nanchang, China, October 14–16, 2022, Proceedings.   Springer, 2022, pp. 19–30.
  12. M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer, “Deep contextualized word representations,” in Proceedings of NAACL-HLT, 2018, pp. 2227–2237.
  13. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in NAACL-HLT (1), 2019.
  14. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692, 2019.
  15. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, pp. 6000–6010.
  16. H. Ren, Y. Sheng, and W. Feng, “Recognizing textualentailmentbasedonknowledgetopicmodels,” Journal of Chinese Information Processing, vol. 29, no. 6, pp. 119–127, 2015.
  17. C. Hu, C. Wu, and Y. Yang, “Extended s-lstm based textual entailment recognition,” Journal of Computer Research and Development, vol. 57, no. 7, pp. 1481–1489, 2020.
  18. M. A. Sultan, S. Bethard, and T. Sumner, “Feature-rich two-stage logistic regression for monolingual alignment,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015, pp. 949–959.
  19. A. Conneau, D. Kiela, H. Schwenk, L. Barrault, and A. Bordes, “Supervised learning of universal sentence representations from natural language inference data,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.   Association for Computational Linguistics, 2017, pp. 670–680.
  20. N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 3982–3992.
  21. T. Gao, X. Yao, and D. Chen, “Simcse: Simple contrastive learning of sentence embeddings,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 6894–6910.
  22. A. Parikh, O. Täckström, D. Das, and J. Uszkoreit, “A decomposable attention model for natural language inference,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016, pp. 2249–2255.
  23. H. He and J. Lin, “Pairwise word interaction modeling with deep neural networks for semantic similarity measurement,” in Proceedings of the 2016 conference of the north American chapter of the Association for Computational Linguistics: human language technologies, 2016, pp. 937–948.
  24. Q. Chen, X. Zhu, Z.-H. Ling, S. Wei, H. Jiang, and D. Inkpen, “Enhanced lstm for natural language inference,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017, pp. 1657–1668.
  25. Z. Wang, W. Hamza, and R. Florian, “Bilateral multi-perspective matching for natural language sentences,” in Proceedings of the 26th International Joint Conference on Artificial Intelligence, 2017, pp. 4144–4150.
  26. Y. Tay, A. T. Luu, and S. C. Hui, “Compare, compress and propagate: Enhancing neural architectures with alignment factorization for natural language inference,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, pp. 1565–1575.
  27. M. Seo, A. Kembhavi, A. Farhadi, and H. Hajishirzi, “Bidirectional attention flow for machine comprehension,” arXiv preprint arXiv:1611.01603, 2016.
  28. D. Shen, G. Wang, W. Wang, M. R. Min, Q. Su, Y. Zhang, C. Li, R. Henao, and L. Carin, “Baseline needs more love: On simple word-embedding-based models and associated pooling mechanisms,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018, pp. 440–450.
  29. A. Talman, A. Yli-Jyrä, and J. Tiedemann, “Natural language inference with hierarchical bilstm max pooling architecture,” arXiv preprint arXiv:1808.08762, 2018.
  30. C. Yu, H. Xue, Y. Jiang, L. An, and G. Li, “A simple and efficient text matching model based on deep interaction,” Information Processing & Management, vol. 58, no. 6, p. 102738, 2021.
  31. K. Zhang, G. Lv, L. Wang, L. Wu, E. Chen, F. Wu, and X. Xie, “Drr-net: Dynamic re-read network for sentence semantic matching,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp. 7442–7449.
  32. R. Zhang, Q. Zhou, B. Wu, W. Li, and T. Mo, “What do questions exactly ask? mfae: Duplicate question identification with multi-fusion asking emphasis,” in Proceedings of the 2020 SIAM International Conference on Data Mining.   SIAM, 2020, pp. 226–234.
  33. M. Glockner, V. Shwartz, and Y. Goldberg, “Breaking nli systems with sentences that require simple lexical inferences,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2018, pp. 650–655.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Kexin Jiang (4 papers)
  2. Yahui Zhao (4 papers)
  3. Guozhe Jin (1 paper)
  4. Zhenguo Zhang (5 papers)
  5. Rongyi Cui (3 papers)
Citations (6)
View on Semantic Scholar