Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Hyperbolic Relevance Matching for Neural Keyphrase Extraction (2205.02047v2)

Published 4 May 2022 in cs.CL and cs.IR

Abstract: Keyphrase extraction is a fundamental task in natural language processing and information retrieval that aims to extract a set of phrases with important information from a source document. Identifying important keyphrase is the central component of the keyphrase extraction task, and its main challenge is how to represent information comprehensively and discriminate importance accurately. In this paper, to address these issues, we design a new hyperbolic matching model (HyperMatch) to represent phrases and documents in the same hyperbolic space and explicitly estimate the phrase-document relevance via the Poincar\'e distance as the important score of each phrase. Specifically, to capture the hierarchical syntactic and semantic structure information, HyperMatch takes advantage of the hidden representations in multiple layers of RoBERTa and integrates them as the word embeddings via an adaptive mixing layer. Meanwhile, considering the hierarchical structure hidden in the document, HyperMatch embeds both phrases and documents in the same hyperbolic space via a hyperbolic phrase encoder and a hyperbolic document encoder. This strategy can further enhance the estimation of phrase-document relevance due to the good properties of hyperbolic space. In this setting, the keyphrase extraction can be taken as a matching problem and effectively implemented by minimizing a hyperbolic margin-based triplet loss. Extensive experiments are conducted on six benchmarks and demonstrate that HyperMatch outperforms the state-of-the-art baselines.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Select, extract and generate: Neural keyphrase generation with layer-wise coverage attention. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing., pages 1389–1404.
  2. Probing bert in hyperbolic spaces. In International Conference on Learning Representations.
  3. APo-VAE: Text generation in hyperbolic space. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 416–431.
  4. What’s so special about bert’s layers? a closer look at the nlp pipeline in monolingual and multilingual models. In EMNLP (Findings), pages 4339–4350. Association for Computational Linguistics.
  5. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, pages 4171–4186. Association for Computational Linguistics.
  6. Embedding text in hyperbolic spaces. In TextGraphs@NAACL-HLT, pages 59–69. Association for Computational Linguistics.
  7. Hyperbolic neural networks. In NeurIPS, pages 5350–5360.
  8. Extracting key terms from noisy and multitheme documents. In WWW, pages 661–670. ACM.
  9. Hyperbolic attention networks. In ICLR (Poster). OpenReview.net.
  10. Matthias Hamann. 2018. On the tree-likeness of hyperbolic spaces. In Mathematical Proceedings of the Cambridge Philosophical Society, volume 164, pages 345–361.
  11. Kazi Saidul Hasan and Vincent Ng. 2014. Automatic keyphrase extraction: A survey of the state of the art. In ACL (1), pages 1262–1273. The Association for Computer Linguistics.
  12. C. Hopper and B. Andrews. 2011. The Ricci Flow in Riemannian Geometry.
  13. Anette Hulth. 2003. Improved automatic keyword extraction given more linguistic knowledge. In EMNLP.
  14. Anette Hulth. 2004. Enhancing linguistically oriented automatic keyword extraction. In HLT-NAACL (Short Papers). The Association for Computational Linguistics.
  15. What does bert learn about the structure of language? In ACL (1), pages 3651–3657. Association for Computational Linguistics.
  16. Karen Spärck Jones. 2004. A statistical interpretation of term specificity and its application in retrieval. J. Documentation, 60(5):493–502.
  17. Hyperbolic image embeddings. In CVPR, pages 6417–6427. IEEE.
  18. Semeval-2010 task 5 : Automatic keyphrase extraction from scientific articles. In SemEval@ACL, pages 21–26. The Association for Computer Linguistics.
  19. Applying graph-based keyword extraction to document retrieval. In IJCNLP, pages 864–868. Asian Federation of Natural Language Processing / ACL.
  20. M. Krapivin and M. Marchese. 2009. Large dataset for keyphrase extraction.
  21. On the sentence embeddings from pre-trained language models. In EMNLP (1), pages 9119–9130. Association for Computational Linguistics.
  22. The geometry of graphs and some of its algorithmic applications. Combinatorica, 15(2):215–245.
  23. Unsupervised approaches for automatic keyword extraction using meeting transcripts. In HLT-NAACL, pages 620–628. The Association for Computational Linguistics.
  24. Keyphrase prediction with pre-trained language model.
  25. Roberta: A robustly optimized bert pretraining approach. CoRR, abs/1907.11692.
  26. Clustering to find exemplar terms for keyphrase extraction. In EMNLP, pages 257–266. ACL.
  27. Continuous hierarchical representations with poincaré variational auto-encoders. In NeurIPS, pages 12544–12555.
  28. Human-competitive tagging using automatic keyphrase extraction. In Internat. Conference of Empirical Methods in Natural Language Processing, EMNLP-2009,.
  29. Deep keyphrase generation. In ACL, pages 582–592. Association for Computational Linguistics.
  30. Rada Mihalcea and Paul Tarau. 2004. Textrank: Bringing order into text. In EMNLP, pages 404–411. ACL.
  31. Keyphrase extraction with span-based feature representations. CoRR, abs/2002.05407.
  32. Chau Q. Nguyen and Tuoi T. Phan. 2009. An ontology-based approach for key phrase extraction. In ACL/IJCNLP (Short Papers), pages 181–184. The Association for Computer Linguistics.
  33. Thuy Dung Nguyen and Min-Yen Kan. 2007. Keyphrase extraction in scientific publications. In ICADL, volume 4822 of Lecture Notes in Computer Science, pages 317–326.
  34. Maximilian Nickel and Douwe Kiela. 2017. Poincaré embeddings for learning hierarchical representations. In NIPS, pages 6338–6347.
  35. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS, pages 8024–8035.
  36. Deep contextualized word representations. In NAACL-HLT, pages 2227–2237. Association for Computational Linguistics.
  37. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. Cite arxiv:1908.10084Comment: Published at EMNLP 2019.
  38. Representation tradeoffs for hyperbolic embeddings. CoRR, abs/1804.03329.
  39. Rik Sarkar. 2011. Low distortion delaunay embedding of trees in hyperbolic plane. In Graph Drawing, volume 7034 of Lecture Notes in Computer Science, pages 355–366.
  40. Importance Estimation from Multiple Perspectives for Keyphrase Extraction. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  41. Joint keyphrase chunking and salience ranking with bert. CoRR, abs/2004.13639.
  42. Divgraphpointer: A graph pointer network for extracting diverse keyphrases. In SIGIR, pages 755–764.
  43. Poincare glove: Hyperbolic word embeddings. In ICLR (Poster). OpenReview.net.
  44. Xiaojun Wan and Jianguo Xiao. 2008. Collabrank: Towards a collaborative approach to single-document keyphrase extraction. In COLING, pages 969–976.
  45. Incorporating multimodal information in open-domain web keyphrase extraction. In EMNLP (1), pages 1790–1800. Association for Computational Linguistics.
  46. Bilateral multi-perspective matching for natural language sentences. In IJCAI, pages 4144–4150.
  47. Huggingface’s transformers: State-of-the-art natural language processing. CoRR, abs/1910.03771.
  48. Unikeyphrase: A unified extraction and generation framework for keyphrase prediction. CoRR, abs/2106.04847.
  49. Open domain web keyphrase extraction beyond language modeling. In EMNLP/IJCNLP (1), pages 5174–5183. Association for Computational Linguistics.
  50. Keyphrase extraction with dynamic graph convolutional networks and diversified inference. CoRR, abs/2010.12828.
  51. Extractive summarization as text matching. In ACL, pages 6197–6208. Association for Computational Linguistics.
  52. Parsing all: Syntax and semantics, dependencies and spans. In EMNLP (Findings), pages 4438–4449. Association for Computational Linguistics.
  53. Hypertext: Endowing fasttext with hyperbolic geometry. In EMNLP (Findings), pages 1166–1171. Association for Computational Linguistics.
Citations (16)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now