Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

BioDEX: Large-Scale Biomedical Adverse Drug Event Extraction for Real-World Pharmacovigilance (2305.13395v2)

Published 22 May 2023 in cs.CL

Abstract: Timely and accurate extraction of Adverse Drug Events (ADE) from biomedical literature is paramount for public safety, but involves slow and costly manual labor. We set out to improve drug safety monitoring (pharmacovigilance, PV) through the use of NLP. We introduce BioDEX, a large-scale resource for Biomedical adverse Drug Event Extraction, rooted in the historical output of drug safety reporting in the U.S. BioDEX consists of 65k abstracts and 19k full-text biomedical papers with 256k associated document-level safety reports created by medical experts. The core features of these reports include the reported weight, age, and biological sex of a patient, a set of drugs taken by the patient, the drug dosages, the reactions experienced, and whether the reaction was life threatening. In this work, we consider the task of predicting the core information of the report given its originating paper. We estimate human performance to be 72.0% F1, whereas our best model achieves 62.3% F1, indicating significant headroom on this task. We also begin to explore ways in which these models could help professional PV reviewers. Our code and data are available: https://github.com/KarelDO/BioDEX.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Pubmed parser: A python parser for pubmed open-access xml subset and medline xml dataset xml dataset. Journal of Open Source Software, 5(46):1979.
  2. Yasser M Alatawi and Richard A Hansen. 2017. Empirical estimation of under-reporting in the us food and drug administration adverse event reporting system (faers). Expert opinion on drug safety, 16(7):761–767.
  3. A curated and standardized adverse drug event resource to accelerate drug safety research. Scientific data, 3(1):1–11.
  4. Artificial intelligence for drug toxicity and safety. Trends in pharmacological sciences, 40(9):624–635.
  5. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623.
  6. The medical dictionary for regulatory activities (meddra). Drug safety, 20(2):109–117.
  7. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  8. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416.
  9. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  10. How do others cope? extracting coping strategies for adverse drug events from social media. Journal of Biomedical Informatics, page 104228.
  11. FDA AEs reporting system (FAERS) public dashboard. https://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/AdverseDrugEffects/ucm070093.htm. Accessed May 16, 2023.
  12. An historical overview over Pharmacovigilance. International journal of clinical pharmacy, 40:744–747.
  13. More extreme duplication in the us fda faers database and a suggested check point for disproportionality analysis. Pharmacoepidemiology and drug safety, 30(8):1140–1141.
  14. More extreme duplication in FDA adverse event reporting system detected by literature reference normalization and fuzzy string matching. Pharmacoepidemiology and Drug Safety.
  15. Pubmedqa: A dataset for biomedical research question answering. arXiv preprint arXiv:1909.06146.
  16. Pretraining to recognize pico elements from randomized controlled trial literature. Studies in health technology and informatics, 264:188.
  17. Openfda: an innovative platform providing access to a wealth of fda’s publicly available data. Journal of the American Medical Informatics Association, 23(3):596–600.
  18. A standardized dataset of a spontaneous adverse event reporting system. Healthcare, 10(3):420.
  19. Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive nlp. arXiv preprint arXiv:2212.14024.
  20. Increased confidence in deduplication of drug safety reports with natural language processing of narratives at the us food and drug administration. Frontiers in Drug Safety and Regulation, 2.
  21. Carolyn E Lipscomb. 2000. Medical subject headings (mesh). Bulletin of the Medical Library Association, 88(3):265.
  22. Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in Bioinformatics, 23(6).
  23. NLM. 2003. PMC Open Access Subset. Internet. Cited 2023 May 23.
  24. NLM. 2021. Medline. Internet. Cited 2023 May 23.
  25. Capabilities of gpt-4 on medical challenge problems. arXiv preprint arXiv:2303.13375.
  26. A corpus with multi-level annotations of patients, interventions and outcomes to support language processing for medical literature. In Proceedings of the conference. Association for Computational Linguistics. Meeting, volume 2018, page 197. NIH Public Access.
  27. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  28. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551.
  29. Overview of the tac 2017 adverse reaction extraction from drug labels track. In TAC.
  30. David L Sackett. 1997. Evidence-based medicine. In Seminars in Perinatology, volume 21, pages 3–5. Elsevier.
  31. Noam Shazeer and Mitchell Stern. 2018. Adafactor: Adaptive learning rates with sublinear memory cost. In International Conference on Machine Learning, pages 4596–4604. PMLR.
  32. Large language models encode clinical knowledge. arXiv preprint arXiv:2212.13138.
  33. Attention is all you need. Advances in neural information processing systems, 30.
  34. Extracting pico sentences from clinical trial reports using supervised distant supervision. The Journal of Machine Learning Research, 17(1):4572–4596.
  35. Rong Xu and QuanQiu Wang. 2014. Large-scale combining signals from both biomedical literature and the fda adverse event reporting system (faers) to improve post-marketing drug safety signal detection. BMC bioinformatics, 15(1):1–10.
  36. Linkbert: Pretraining language models with document links. arXiv preprint arXiv:2203.15827.
  37. Adversarial attacks on deep-learning models in natural language processing: A survey. ACM Transactions on Intelligent Systems and Technology (TIST), 11(3):1–41.
  38. Pmc-patients: A large-scale dataset of patient notes and relations extracted from case reports in pubmed central. arXiv preprint arXiv:2202.13876.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (10)
  1. Karel D'Oosterlinck (11 papers)
  2. François Remy (10 papers)
  3. Johannes Deleu (29 papers)
  4. Thomas Demeester (76 papers)
  5. Chris Develder (59 papers)
  6. Klim Zaporojets (14 papers)
  7. Aneiss Ghodsi (1 paper)
  8. Simon Ellershaw (2 papers)
  9. Jack Collins (19 papers)
  10. Christopher Potts (113 papers)
Citations (5)
View on Semantic Scholar