Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PromptNER: Prompting For Named Entity Recognition (2305.15444v2)

Published 24 May 2023 in cs.CL, cs.AI, and cs.LG

Abstract: In a surprising turn, LLMs together with a growing arsenal of prompt-based heuristics now offer powerful off-the-shelf approaches providing few-shot solutions to myriad classic NLP problems. However, despite promising early results, these LLM-based few-shot methods remain far from the state of the art in Named Entity Recognition (NER), where prevailing methods include learning representations via end-to-end structural understanding and fine-tuning on standard labeled corpora. In this paper, we introduce PromptNER, a new state-of-the-art algorithm for few-Shot and cross-domain NER. To adapt to any new NER task PromptNER requires a set of entity definitions in addition to the standard few-shot examples. Given a sentence, PromptNER prompts an LLM to produce a list of potential entities along with corresponding explanations justifying their compatibility with the provided entity type definitions. Remarkably, PromptNER achieves state-of-the-art performance on few-shot NER, achieving a 4% (absolute) improvement in F1 score on the ConLL dataset, a 9% (absolute) improvement on the GENIA dataset, and a 4% (absolute) improvement on the FewNERD dataset. PromptNER also moves the state of the art on Cross Domain NER, outperforming prior methods (including those not limited to the few-shot setting), setting a new mark on 3/5 CrossNER target domains, with an average F1 gain of 3%, despite using less than 2% of the available data.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. Named entity recognition using deep learning: A review. In 2022 International Conference on Business Analytics for Technology and Security (ICBATS), pages 1–7. IEEE.
  2. Exploiting diverse knowledge sources via maximum entropy in named entity recognition. In Sixth Workshop on Very Large Corpora.
  3. Andrew Eliot Borthwick. 1999. A maximum entropy approach to named entity recognition. New York University.
  4. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  5. One model for all domains: Collaborative domain-prefix tuning for cross-domain ner. arXiv preprint arXiv:2301.10410.
  6. Prompt-based metric learning for few-shot ner. arXiv preprint arXiv:2211.04337.
  7. Nancy Chinchor. 1995. Muc-6 named entity task definition (version 2.1). In 6th Message Understanding Conference, Columbia, Maryland.
  8. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.
  9. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416.
  10. Emerging trends: A gentle introduction to fine-tuning. Natural Language Engineering, 27(6):763–778.
  11. Template-based named entity recognition using bart. arXiv preprint arXiv:2106.01760.
  12. Container: Few-shot named entity recognition via contrastive learning. In ACL.
  13. A survey on the application of recurrent neural networks to statistical language modeling. Computer Speech & Language, 30(1):61–98.
  14. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, pages 1–16.
  15. Few-nerd: A few-shot named entity recognition dataset. arXiv preprint arXiv:2105.07464.
  16. A rule-based named-entity recognition method for knowledge extraction of evidence-based dietary recommendations. PloS one, 12(6):e0179488.
  17. Rule-based named entity recognition for greek financial texts. In Proceedings of the Workshop on Computational lexicography and Multimedia Dictionaries (COMLEX 2000), pages 75–78.
  18. A label-aware autoregressive framework for cross-domain ner. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2222–2232.
  19. Entda: Entity-to-text based data augmentation approach for named entity recognition tasks. arXiv preprint arXiv:2210.10343.
  20. Copner: Contrastive learning with prompt guiding for few-shot named entity recognition. In Proceedings of the 29th International conference on computational linguistics, pages 2515–2527.
  21. COPNER: Contrastive learning with prompt guiding for few-shot named entity recognition. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2515–2527, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  22. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991.
  23. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361.
  24. Genia corpus—a semantically annotated corpus for bio-textmining. Bioinformatics, 19(suppl_1):i180–i182.
  25. Good examples make a faster learner: Simple demonstration-based learning for low-resource ner. arXiv preprint arXiv:2110.08454.
  26. Unified named entity recognition as word-word relation classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10965–10973.
  27. Nut Limsopatham and Nigel Henry Collier. 2016. Bidirectional lstm for named entity recognition in twitter messages.
  28. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1–35.
  29. Crossner: Evaluating cross-domain named entity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13452–13460.
  30. Punifiedner: a prompting-based unified ner system for diverse datasets. arXiv preprint arXiv:2211.14838.
  31. Unified structure generation for universal information extraction. arXiv preprint arXiv:2203.12277.
  32. Template-free prompt tuning for few-shot ner. arXiv preprint arXiv:2109.13532.
  33. Decomposed meta-learning for few-shot named entity recognition. arXiv preprint arXiv:2204.05751.
  34. Tuning support vector machines for biomedical named entity recognition. In Proceedings of the ACL-02 workshop on Natural language processing in the biomedical domain, pages 1–8.
  35. Named entity recognition using hundreds of thousands of features. In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003, pages 184–187.
  36. Named entity recognition without gazetteers. In Ninth Conference of the European Chapter of the Association for Computational Linguistics, pages 1–8.
  37. Few-shot nested named entity recognition. arXiv preprint arXiv:2212.00953.
  38. OpenAI. 2023. Gpt-4 technical report.
  39. Erik F Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. arXiv preprint cs/0306050.
  40. Named entity recognition in natural language processing: A systematic review. In Proceedings of Second Doctoral Symposium on Computational Intelligence: DoSCI 2021, pages 817–828. Springer.
  41. Locate and label: A two-stage identifier for nested named entity recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2782–2794, Online. Association for Computational Linguistics.
  42. Improving biomedical named entity recognition with a unified multi-task mrc framework. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8332–8336. IEEE.
  43. Towards understanding chain-of-thought prompting: An empirical study of what matters. arXiv preprint arXiv:2212.10001.
  44. Deepstruct: Pretraining of language models for structure prediction. arXiv preprint arXiv:2205.10475.
  45. Automated concatenation of embeddings for structured prediction. arXiv preprint arXiv:2010.05006.
  46. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652.
  47. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903.
  48. From clozing to comprehending: Retrofitting pre-trained language model to pre-trained machine reader. arXiv preprint arXiv:2212.04755.
  49. Factmix: Using a few labeled in-domain examples to generalize to cross-domain named entity recognition. arXiv preprint arXiv:2208.11464.
  50. Named entity recognition as dependency parsing. arXiv preprint arXiv:2005.07150.
  51. GuoDong Zhou and Jian Su. 2002. Named entity recognition using an hmm-based chunk tagger. In Proceedings of the 40th annual meeting of the association for computational linguistics, pages 473–480.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Dhananjay Ashok (8 papers)
  2. Zachary C. Lipton (137 papers)
Citations (28)
View on Semantic Scholar