Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A General and Flexible Multi-concept Parsing Framework for Multilingual Semantic Matching (2403.02975v2)

Published 5 Mar 2024 in cs.CL and cs.AI

Abstract: Sentence semantic matching is a research hotspot in natural language processing, which is considerably significant in various key scenarios, such as community question answering, searching, chatbot, and recommendation. Since most of the advanced models directly model the semantic relevance among words between two sentences while neglecting the \textit{keywords} and \textit{intents} concepts of them, DC-Match is proposed to disentangle keywords from intents and utilizes them to optimize the matching performance. Although DC-Match is a simple yet effective method for semantic matching, it highly depends on the external NER techniques to identify the keywords of sentences, which limits the performance of semantic matching for minor languages since satisfactory NER tools are usually hard to obtain. In this paper, we propose to generally and flexibly resolve the text into multi concepts for multilingual semantic matching to liberate the model from the reliance on NER models. To this end, we devise a \underline{M}ulti-\underline{C}oncept \underline{P}arsed \underline{S}emantic \underline{M}atching framework based on the pre-trained LLMs, abbreviated as \textbf{MCP-SM}, to extract various concepts and infuse them into the classification tokens. We conduct comprehensive experiments on English datasets QQP and MRPC, and Chinese dataset Medical-SM. Besides, we experiment on Arabic datasets MQ2Q and XNLI, the outstanding performance further prove MCP-SM's applicability in low-resource languages.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic. (arXiv:2101.01785).
  2. AraBERT: Transformer-based Model for Arabic Language Understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association.
  3. An Approach for Bengali Automatic Question Answering System Using Attention Mechanism.
  4. End-to-End Object Detection with Transformers. (arXiv:2005.12872).
  5. XNLI: Evaluating Cross-lingual Sentence Representations. (arXiv:1809.05053).
  6. Pre-Training with Whole Word Masking for Chinese BERT. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3504–3514.
  7. Convolutional Neural Networks for Soft-Matching N-Grams in Ad-hoc Search. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pages 126–134, Marina Del Rey CA USA. ACM.
  8. Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing. (arXiv:2006.03236).
  9. Enhanced distance-aware self-attention and multi-level match for sentence semantic matching. Neurocomputing, 501:174–187.
  10. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. (arXiv:1810.04805).
  11. Automatically Constructing a Corpus of Sentential Paraphrases. page 8.
  12. Structural Representations for Learning Relations between Pairs of Texts. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1003–1013, Beijing, China. Association for Computational Linguistics.
  13. JABER and SABER: Junior and Senior Arabic BERt. (arXiv:2112.04329).
  14. Natural Language Inference over Interaction Space. (arXiv:1709.04348).
  15. A Deep Relevance Matching Model for Ad-hoc Retrieval. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pages 55–64.
  16. DeBERTa: Decoding-enhanced BERT with Disentangled Attention. (arXiv:2006.03654).
  17. Using Deep Learning and an External Knowledge Base to Develop Human-Robot Dialogues. In 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 3709–3714.
  18. The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models. (arXiv:2103.06678).
  19. First quora dataset release: Question pairs. data. quora. com.
  20. SpanBERT: Improving Pre-training by Representing and Predicting Spans. Transactions of the Association for Computational Linguistics, 8:64–77.
  21. Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information. (arXiv:1805.11360).
  22. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. (arXiv:1909.11942).
  23. Towards Deep Conversational Recommendations. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc.
  24. Deep bi-directional interaction network for sentence matching. Applied Intelligence, 51(7):4305–4329.
  25. RoBERTa: A Robustly Optimized BERT Pretraining Approach. (arXiv:1907.11692).
  26. Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. (arXiv:1711.05101).
  27. MKPM: Multi keyword-pair matching for natural language sentences. Applied Intelligence, 52(2):1878–1892.
  28. RevCore: Review-augmented Conversational Recommendation. (arXiv:2106.00957).
  29. Bhaskar Mitra and Nick Craswell. 2019. An Updated Duet Model for Passage Re-ranking. (arXiv:1903.07666).
  30. Learning to Match Using Local and Distributed Representations of Text for Web Search. (arXiv:1610.08136).
  31. Jonas Mueller and Aditya Thyagarajan. 2016. Siamese Recurrent Architectures for Learning Sentence Similarity. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1).
  32. Learning Text Similarity with Siamese Recurrent Networks. In Proceedings of the 1st Workshop on Representation Learning for NLP, pages 148–157, Berlin, Germany. Association for Computational Linguistics.
  33. Enhanced-RCNN: An Efficient Method for Learning Sentence Similarity. In Proceedings of The Web Conference 2020, pages 2500–2506, Taipei Taiwan. ACM.
  34. KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 2054–2059, Barcelona (online). International Committee for Computational Linguistics.
  35. NSURL-2019 Task 8: Semantic Question Similarity in Arabic. In Proceedings of the First International Workshop on NLP Solutions for Under Resourced Languages (NSURL 2019) Co-Located with ICNLSP 2019 - Short Papers, pages 1–8, Trento, Italy. Association for Computational Linguistics.
  36. Sequence to Sequence Learning with Neural Networks. (arXiv:1409.3215).
  37. SANTM: Efficient Self-attention-driven Network for Text Matching. ACM Transactions on Internet Technology, 22(3):1–21.
  38. Neural Paraphrase Identification of Questions with Noisy Pretraining. (arXiv:1704.04565).
  39. Attention Is All You Need. (arXiv:1706.03762).
  40. Attention is All you Need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  41. Bilateral Multi-Perspective Matching for Natural Language Sentences. (arXiv:1702.03814).
  42. Sentence Similarity Learning by Lexical Decomposition and Composition. (arXiv:1602.07019).
  43. End-to-End Neural Ad-hoc Ranking with Kernel Pooling. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 55–64.
  44. XLNet: Generalized Autoregressive Pretraining for Language Understanding. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
  45. Retriever: Learning Content-Style Representation as a Token-Level Bipartite Graph. (arXiv:2202.12307).
  46. A Context-Enriched Neural Network Method for Recognizing Lexical Entailment. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1).
  47. DRr-Net: Dynamic Re-Read Network for Sentence Semantic Matching. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):7442–7449.
  48. Making the Relation Matters: Relation of Relation Learning Network for Sentence Semantic Matching. Proceedings of the AAAI Conference on Artificial Intelligence, 35(16):14411–14419.
  49. KECRS: Towards Knowledge-Enriched Conversational Recommendation System. (arXiv:2105.08261).
  50. Divide and conquer: Text semantic matching with disentangled keywords and intents. (arXiv:2203.02898).

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now