Automated Fusion of Multimodal Electronic Health Records for Better Medical Predictions (2401.11252v1)
Abstract: The widespread adoption of Electronic Health Record (EHR) systems in healthcare institutes has generated vast amounts of medical data, offering significant opportunities for improving healthcare services through deep learning techniques. However, the complex and diverse modalities and feature structures in real-world EHR data pose great challenges for deep learning model design. To address the multi-modality challenge in EHR data, current approaches primarily rely on hand-crafted model architectures based on intuition and empirical experiences, leading to sub-optimal model architectures and limited performance. Therefore, to automate the process of model design for mining EHR data, we propose a novel neural architecture search (NAS) framework named AutoFM, which can automatically search for the optimal model architectures for encoding diverse input modalities and fusion strategies. We conduct thorough experiments on real-world multi-modal EHR data and prediction tasks, and the results demonstrate that our framework not only achieves significant performance improvement over existing state-of-the-art methods but also discovers meaningful network architectures effectively.
- Doctor ai: Predicting clinical events via recurrent neural networks. In Machine learning for healthcare conference, pages 301–318. PMLR, 2016.
- Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. In Advances in Neural Information Processing Systems, pages 3504–3512, 2016.
- Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
- Automed: Automated medical risk predictive modeling on electronic health records. In 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 948–953. IEEE, 2022.
- Neural architecture search: A survey. The Journal of Machine Learning Research, 20(1):1997–2017, 2019.
- Dcmn: Double core memory network for patient outcome prediction with multimodal data. In 2019 IEEE International Conference on Data Mining (ICDM), pages 200–209. IEEE, 2019.
- Recent advances in convolutional neural networks. Pattern recognition, 77:354–377, 2018.
- Star-transformer. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1315–1325, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
- Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
- Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv preprint arXiv:1904.05342, 2019.
- Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1–9, 2016.
- Neural architecture search with bayesian optimisation and optimal transport. Advances in neural information processing systems, 31, 2018.
- Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
- Progressive neural architecture search. In Proceedings of the European conference on computer vision (ECCV), pages 19–34, 2018.
- Darts: Differentiable architecture search. In International Conference on Learning Representations, 2019.
- Dipole: Diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1903–1911, 2017.
- Risk prediction on electronic health records with prior medical knowledge. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1910–1919, 2018.
- Efficient neural architecture search via parameters sharing. In International conference on machine learning, pages 4095–4104. PMLR, 2018.
- Mnn: multimodal attentional neural networks for diagnosis prediction. Extraction, 1:A1, 2019.
- Integrating multimodal information in large pretrained transformers. In Proceedings of the conference. Association for Computational Linguistics. Meeting, volume 2020, page 2359. NIH Public Access, 2020.
- Regularized evolution for image classifier architecture search. In Proceedings of the aaai conference on artificial intelligence, volume 33, pages 4780–4789, 2019.
- Gamenet: Graph augmented memory networks for recommending medication combination. In proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1126–1133, 2019.
- Democratizing ehr analyses with fiddle: a flexible data-driven preprocessing pipeline for structured clinical data. Journal of the American Medical Informatics Association, 27(12):1921–1934, 2020.
- Attention is all you need. In Advances in neural information processing systems, pages 5998–6008, 2017.
- Rethinking architecture selection in differentiable nas. arXiv preprint arXiv:2108.04392, 2021.
- Raim: Recurrent attentive and intensive model of multimodal patient monitoring data. In Proceedings of the 24th ACM SIGKDD international conference on Knowledge Discovery & Data Mining, pages 2565–2573, 2018.
- Mufasa: Multimodal fusion architecture search for electronic health records. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10532–10540, 2021.
- Bo Yang and Lijun Wu. How to leverage multimodal ehr data for better medical predictions? arXiv preprint arXiv:2110.15763, 2021.
- Understanding and robustifying differentiable architecture search. arXiv preprint arXiv:1909.09656, 2019.
- Improving medical predictions by irregular multimodal electronic health records modeling. arXiv preprint arXiv:2210.12156, 2022.
- Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578, 2016.
Sponsored by Paperpile, the PDF & BibTeX manager trusted by top AI labs.
Get 30 days freePaper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.