2000 character limit reached
Discourse-Aware In-Context Learning for Temporal Expression Normalization (2404.07775v1)
Published 11 Apr 2024 in cs.CL, cs.AI, and cs.LG
Abstract: Temporal expression (TE) normalization is a well-studied problem. However, the predominately used rule-based systems are highly restricted to specific settings, and upcoming machine learning approaches suffer from a lack of labeled data. In this work, we explore the feasibility of proprietary and open-source LLMs for TE normalization using in-context learning to inject task, document, and example information into the model. We explore various sample selection strategies to retrieve the most relevant set of examples. By using a window-based prompt design approach, we can perform TE normalization across sentences, while leveraging the LLM knowledge without training the model. Our experiments show competitive results to models designed for this task. In particular, our method achieves large performance improvements for non-standard settings by dynamically including relevant examples during inference.
- Mikel Artetxe and Holger Schwenk. 2019. Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond. Transactions of the Association for Computational Linguistics, 7:597–610.
- Steven Bethard. 2013. A synchronous context free grammar for time normalization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, volume 2013, page 821. NIH Public Access.
- Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
- Angel X Chang and Christopher D Manning. 2012. Sutime: A library for recognizing and normalizing time expressions. In Lrec, volume 12, pages 3735–3740.
- Timebench: A comprehensive evaluation of temporal reasoning abilities in large language models.
- Exploring the feasibility of chatgpt for event extraction.
- Mixtral of experts.
- Adversarial alignment of multilingual models for extracting temporal expressions from text. In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 103–109.
- Multilingual normalization of temporal expressions with masked language models. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1174–1186.
- From characters to time intervals: New paradigms for evaluation and neural parsing of time normalizations. Transactions of the Association for Computational Linguistics, 6:343–356.
- Context-dependent semantic parsing for time expressions. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1437–1447.
- A bert-based universal model for both within-and cross-sentence clinical temporal relation extraction. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pages 65–71.
- Multi-document summarization with maximal marginal relevance-guided reinforcement learning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1737–1751.
- Metaicl: Learning to learn in context. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2791–2809.
- Tempcourt: evaluation of temporal taggers on a new corpus of court decisions. The Knowledge Engineering Review, 34:e24.
- The specification language timeml.
- Scaling language models: Methods, analysis & insights from training gopher.
- Massively multilingual transfer for ner. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 151–164.
- Learning to retrieve prompts for in-context learning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2655–2671.
- Extending HeidelTime for temporal expressions referring to historic dates. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 2390–2397, Reykjavik, Iceland. European Language Resources Association (ELRA).
- Jannik Strötgen and Michael Gertz. 2013. Multilingual and cross-domain temporal tagging. Language Resources and Evaluation, 47:269–298.
- Jannik Strötgen and Michael Gertz. 2016. Domain-sensitive temporal tagging. In Domain-Sensitive Temporal Tagging, pages 47–83. Springer.
- Evaluating temporal relations in clinical text: 2012 i2b2 challenge. Journal of the American Medical Informatics Association, 20(5):806–813.
- Does synthetic data generation of llms help clinical text mining?
- Llama 2: Open foundation and fine-tuned chat models.
- Zephyr: Direct distillation of lm alignment.
- SemEval-2013 task 1: TempEval-3: Evaluating time expressions, events, and temporal relations. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 1–9, Atlanta, Georgia, USA. Association for Computational Linguistics.
- Zero-shot information extraction via chatting with chatgpt.