Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
91 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
o3 Pro
5 tokens/sec
GPT-4.1 Pro
15 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
Gemini 2.5 Flash Deprecated
12 tokens/sec
2000 character limit reached

emrQA-msquad: A Medical Dataset Structured with the SQuAD V2.0 Framework, Enriched with emrQA Medical Information (2404.12050v1)

Published 18 Apr 2024 in cs.CL

Abstract: Machine Reading Comprehension (MRC) holds a pivotal role in shaping Medical Question Answering Systems (QAS) and transforming the landscape of accessing and applying medical information. However, the inherent challenges in the medical field, such as complex terminology and question ambiguity, necessitate innovative solutions. One key solution involves integrating specialized medical datasets and creating dedicated datasets. This strategic approach enhances the accuracy of QAS, contributing to advancements in clinical decision-making and medical research. To address the intricacies of medical terminology, a specialized dataset was integrated, exemplified by a novel Span extraction dataset derived from emrQA but restructured into 163,695 questions and 4,136 manually obtained answers, this new dataset was called emrQA-msquad dataset. Additionally, for ambiguous questions, a dedicated medical dataset for the Span extraction task was introduced, reinforcing the system's robustness. The fine-tuning of models such as BERT, RoBERTa, and Tiny RoBERTa for medical contexts significantly improved response accuracy within the F1-score range of 0.75 to 1.00 from 10.1% to 37.4%, 18.7% to 44.7% and 16.0% to 46.8%, respectively. Finally, emrQA-msquad dataset is publicy available at https://huggingface.co/datasets/Eladio/emrqa-msquad.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (29)
  1. Y. Chang, X. Wang, J. Wang, Y. Wu, L. Yang, K. Zhu, H. Chen, X. Yi, C. Wang, Y. Wang, W. Ye, Y. Zhang, Y. Chang, P. S. Yu, Q. Yang, and X. Xie, “A survey on evaluation of large language models,” 2023.
  2. S. Wang, Z. Zhao, X. Ouyang, Q. Wang, and D. Shen, “Chatcad: Interactive computer-aided diagnosis on medical image using large language models,” 2023.
  3. H. B. Duygu Gok and B. Bozoglan, “Effects of online flipped classroom on foreign language classroom anxiety and reading anxiety,” Computer Assisted Language Learning, vol. 36, no. 4, pp. 840–860, 2023.
  4. R. Chengoden, N. Victor, T. Huynh-The, G. Yenduri, R. H. Jhaveri, M. Alazab, S. Bhattacharya, P. Hegde, P. K. R. Maddikunta, and T. R. Gadekallu, “Metaverse for healthcare: A survey on potential applications, challenges and future directions,” IEEE Access, vol. 11, pp. 12765–12795, 2023.
  5. H. Zhang, H. Song, S. Li, M. Zhou, and D. Song, “A survey of controllable text generation using transformer-based pre-trained language models,” ACM Comput. Surv., vol. 56, 10 2023.
  6. K. Van Nguyen, T. Van Huynh, D.-V. Nguyen, A. G.-T. Nguyen, and N. L.-T. Nguyen, “New vietnamese corpus for machine reading comprehension of health news articles,” ACM Trans. Asian Low-Resour. Lang. Inf. Process., vol. 21, 9 2022.
  7. C. J. Haug and J. M. Drazen, “Artificial intelligence and machine learning in clinical medicine, 2023,” New England Journal of Medicine, vol. 388, no. 13, pp. 1201–1208, 2023. PMID: 36988595.
  8. C. Zhou, Q. Li, C. Li, J. Yu, Y. Liu, G. Wang, K. Zhang, C. Ji, Q. Yan, L. He, H. Peng, J. Li, J. Wu, Z. Liu, P. Xie, C. Xiong, J. Pei, P. S. Yu, and L. Sun, “A comprehensive survey on pretrained foundation models: A history from bert to chatgpt,” 2023.
  9. E. Mutabazi, J. Ni, G. Tang, and W. Cao, “A review on medical textual question answering systems based on deep learning approaches,” Applied Sciences, vol. 11, no. 12, 2021.
  10. A. Thirunavukarasu, D. Ting, K. Elangovan, et al., “Large language models in medicine,” Nat Med, vol. 29, pp. 1930–1940, 2023.
  11. X.-Y. Li, W.-J. Lei, and Y.-B. Yang, “From easy to hard: Two-stage selector and reader for multi-hop question answering,” 2022.
  12. Y. Guan, Z. Li, J. Leng, Z. Lin, M. Guo, and Y. Zhu, “Block-skim: Efficient question answering for transformer,” 2022.
  13. X. Zhang, A. Bosselut, M. Yasunaga, H. Ren, P. Liang, C. D. Manning, and J. Leskovec, “Greaselm: Graph reasoning enhanced language models for question answering,” 2022.
  14. X. Cao and Y. Liu, “Coarse-grained decomposition and fine-grained interaction for multi-hop question answering,” Journal of Intelligent Information Systems, vol. 58, pp. 21–41, 2022.
  15. A. Rogers, M. Gardner, and I. Augenstein, “Qa dataset explosion: A taxonomy of nlp resources for question answering and reading comprehension,” ACM Comput. Surv., vol. 55, 2 2023.
  16. D. Chen, Neural reading comprehension and beyond. PhD thesis, Stanford University, 2018.
  17. C. Zeng, S. Li, Q. Li, J. Hu, and J. Hu, “(2020),” A Survey on Machine Reading Comprehension—Tasks, Evaluation Metrics and Benchmark Datasets. Applied Sciences. 10. 7640, vol. 10., no. 7640.
  18. S. . Z. Liu, X. . Zhang, S. . Wang, H. . Zhang, and W., “Neural machine reading comprehension: Methods and trends,” Applied Sciences 2019, vol. 9, no. 3698., 2019.
  19. ArXiv 2019, 1906.
  20. Z. Liu, X. Yu, L. Zhang, Z. Wu, C. Cao, H. Dai, L. Zhao, W. Liu, D. Shen, Q. Li, T. Liu, D. Zhu, and X. Li, “Deid-gpt: Zero-shot medical text de-identification by gpt-4,” 2023.
  21. H. Nori, N. King, S. M. McKinney, D. Carignan, and E. Horvitz, “Capabilities of gpt-4 on medical challenge problems,” 2023.
  22. A. Sharma, I. Lin, A. Miner, et al., “Human–ai collaboration enables more empathic conversations in text-based peer-to-peer mental health support,” Nat Mach Intell, vol. 5, pp. 46–57, 2023.
  23. L. Hu, Z. Liu, Z. Zhao, L. Hou, L. Nie, and J. Li, “A survey of knowledge enhanced pre-trained language models,” IEEE Transactions on Knowledge and Data Engineering, pp. 1–19, 2023.
  24. A. Tapeh and M. Naser, “Artificial intelligence, machine learning, and deep learning in structural engineering: A scientometrics review of trends and best practices,” Arch Computat Methods Eng, vol. 30, pp. 115–159, 2023.
  25. V. D. Lai, N. T. Ngo, A. P. B. Veyseh, H. Man, F. Dernoncourt, T. Bui, and T. H. Nguyen, “Chatgpt beyond english: Towards a comprehensive evaluation of large language models in multilingual learning,” 2023.
  26. emrqa: A large corpus for question answering on electronic medical records. EMNLP.
  27. P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “SQuAD: 100,000+ Questions for Machine Comprehension of Text,” arXiv e-prints, p. arXiv:1606.05250, 2016.
  28. NAACL, 2019.
  29. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” 2019.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets