Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DKEC: Domain Knowledge Enhanced Multi-Label Classification for Diagnosis Prediction (2310.07059v2)

Published 10 Oct 2023 in cs.CL, cs.AI, and cs.LG

Abstract: Multi-label text classification (MLTC) tasks in the medical domain often face the long-tail label distribution problem. Prior works have explored hierarchical label structures to find relevant information for few-shot classes, but mostly neglected to incorporate external knowledge from medical guidelines. This paper presents DKEC, Domain Knowledge Enhanced Classification for diagnosis prediction with two innovations: (1) automated construction of heterogeneous knowledge graphs from external sources to capture semantic relations among diverse medical entities, (2) incorporating the heterogeneous knowledge graphs in few-shot classification using a label-wise attention mechanism. We construct DKEC using three online medical knowledge sources and evaluate it on a real-world Emergency Medical Services (EMS) dataset and a public electronic health record (EHR) dataset. Results show that DKEC outperforms the state-of-the-art label-wise attention networks and transformer models of different sizes, particularly for the few-shot classes. More importantly, it helps the smaller LLMs achieve comparable performance to LLMs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (28)
  1. Multi-label classification of patient notes a case study on ICD code assignment. arXiv preprint arXiv:1709.09587.
  2. Deep short text classification with knowledge powered attention. In Proceedings of the AAAI conference on artificial intelligence, volume 33, 6252–6259.
  3. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  4. Kan: Knowledge-aware attention network for fake news detection. In Proceedings of the AAAI conference on artificial intelligence, volume 35, 81–89.
  5. BioMedLM.
  6. Heterogeneous graph transformer. In Proceedings of the web conference 2020, 2704–2710.
  7. EMSAssist: An End-to-End Mobile Voice Assistant at the Edge for Emergency Medical Services. In Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services, 275–288.
  8. MIMIC-III, a freely accessible critical care database. Scientific data, 3(1): 1–9.
  9. Information Extraction from Patient Care Reports for Intelligent Emergency Medical Services. In 2021 IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), 58–69. IEEE.
  10. Kim, Y. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882.
  11. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4): 1234–1240.
  12. Multi-label few/zero-shot learning with knowledge aggregated from multiple label graphs. arXiv preprint arXiv:2010.07459.
  13. BioGPT: generative pre-trained transformer for biomedical text generation and mining. Briefings in Bioinformatics, 23(6).
  14. Explainable prediction of medical codes from clinical text. arXiv preprint arXiv:1802.05695.
  15. Few-shot and zero-shot multi-label learning for structured label spaces. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, volume 2018, 3132. NIH Public Access.
  16. Lightweight Transformers for Clinical Natural Language Processing. arXiv preprint arXiv:2302.04725.
  17. On the effectiveness of compact biomedical transformers. Bioinformatics, 39(3): btad103.
  18. Statistical topic models for multi-label document classification. Machine learning, 88: 157–208.
  19. Mobilebert: a compact task-agnostic bert for resource-limited devices. arXiv preprint arXiv:2004.02984.
  20. A scikit-based Python environment for performing multi-label classification. arXiv preprint arXiv:1702.01460.
  21. Clinical outcome prediction from admission notes using self-supervised knowledge integration. arXiv preprint arXiv:2102.04110.
  22. KenMeSH: Knowledge-enhanced end-to-end biomedical text labelling. arXiv preprint arXiv:2203.06835.
  23. Counterfactual supporting facts extraction for explainable medical record based diagnosis with graph network. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1942–1955.
  24. KerPrint: Local-Global Knowledge Graph Enhanced Diagnosis Prediction for Retrospective and Prospective Interpretations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 5357–5365.
  25. SGM: sequence generation model for multi-label classification. arXiv preprint arXiv:1806.04822.
  26. GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records. arXiv preprint arXiv:2203.03540.
  27. Medpath: Augmenting health risk prediction via medical knowledge paths. In Proceedings of the Web Conference 2021, 1397–1409.
  28. BioWordVec, improving biomedical word embeddings with subword information and MeSH. Scientific data, 6(1): 52.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now