2000 character limit reached
Multi-Modal Contrastive Learning for Online Clinical Time-Series Applications (2403.18316v1)
Published 27 Mar 2024 in cs.LG
Abstract: Electronic Health Record (EHR) datasets from Intensive Care Units (ICU) contain a diverse set of data modalities. While prior works have successfully leveraged multiple modalities in supervised settings, we apply advanced self-supervised multi-modal contrastive learning techniques to ICU data, specifically focusing on clinical notes and time-series for clinically relevant online prediction tasks. We introduce a loss function Multi-Modal Neighborhood Contrastive Loss (MM-NCL), a soft neighborhood function, and showcase the excellent linear probe and zero-shot performance of our approach.
- Understanding intermediate layers using linear classifier probes, 2017. URL https://openreview.net/forum?id=ryF7rTqgl.
- Altclip: Altering the language encoder in clip for extended language capabilities, 2022.
- Learning phrase representations using rnn encoder-decoder for statistical machine translation, 2014.
- Multitask learning and benchmarking with clinical time series data. Scientific Data, 6(1):96, Jun 2019. ISSN 2052-4463. doi: 10.1038/s41597-019-0103-9. URL https://doi.org/10.1038/s41597-019-0103-9.
- Medfuse: Multi-modal fusion with clinical time-series data and chest x-ray images, 2023.
- Clinicalbert: Modeling clinical notes and predicting hospital readmission, 2020.
- A comprehensive ml-based respiratory monitoring system for physiological monitoring & resource planning in the icu. medRxiv, 2024. doi: 10.1101/2024.01.23.24301516. URL https://www.medrxiv.org/content/early/2024/01/23/2024.01.23.24301516.
- On the importance of clinical notes in multi-modal learning for ehr data. In NeurIPS 2022 Workshop on Learning from Time Series for Health, 2022.
- Early prediction of circulatory failure in the intensive care unit using machine learning. Nature medicine, 26(3):364–373, 2020.
- Knowledge graph representations to enhance intensive care time-series predictions, 2023. URL https://arxiv.org/abs/2311.07180.
- Mimic-iii, a freely accessible critical care database. Scientific Data, 3(1):160035, May 2016. ISSN 2052-4463. doi: 10.1038/sdata.2016.35. URL https://doi.org/10.1038/sdata.2016.35.
- Using clinical notes with time series data for icu management. arXiv preprint arXiv:1909.09702, 2019.
- Multimodal pretraining of medical time series and notes, 2023.
- Adam: A method for stochastic optimization, 2017.
- On the importance of step-wise embeddings for heterogeneous clinical time-series. In Stefan Hegselmann, Antonio Parziale, Divya Shanmugam, Shengpu Tang, Mercy Nyamewaa Asiedu, Serina Chang, Tom Hartvigsen, and Harvineet Singh (eds.), Proceedings of the 3rd Machine Learning for Health Symposium, volume 225 of Proceedings of Machine Learning Research, pp. 268–291. PMLR, 10 Dec 2023. URL https://proceedings.mlr.press/v225/kuznetsova23a.html.
- Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023.
- Multi-modal contrastive learning for healthcare data analytics. In 2022 IEEE 10th International Conference on Healthcare Informatics (ICHI), pp. 120–127, 2022. doi: 10.1109/ICHI54592.2022.00029.
- Revisiting temporal modeling for clip-based image-to-video knowledge transferring, 2023a.
- Self-supervised contrastive learning for medical time series: A systematic review. Sensors, 23(9), 2023b. ISSN 1424-8220. doi: 10.3390/s23094221. URL https://www.mdpi.com/1424-8220/23/9/4221.
- POETREE: Interpretable policy learning with adaptive decision trees. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=AJsI-ymaKn_.
- Pytorch: An imperative style, high-performance deep learning library, 2019.
- Learning transferable visual models from natural language supervision, 2021.
- High-resolution image synthesis with latent diffusion models, 2022.
- Lidar: Sensing linear probing performance in joint embedding ssl architectures, 2023.
- Yet another ICU benchmark: A flexible multi-center framework for clinical ML. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=ox2ATRM90I.
- Representation learning with contrastive predictive coding, 2019.
- Medclip: Contrastive learning from unpaired medical images and text, 2022.
- Learning unsupervised representations for icu timeseries. In Gerardo Flores, George H Chen, Tom Pollard, Joyce C Ho, and Tristan Naumann (eds.), Proceedings of the Conference on Health, Inference, and Learning, volume 174 of Proceedings of Machine Learning Research, pp. 152–168. PMLR, 07–08 Apr 2022. URL https://proceedings.mlr.press/v174/weatherhead22a.html.
- Neighborhood contrastive learning applied to online patient monitoring. In International Conference on Machine Learning, pp. 11964–11974. PMLR, 2021.
- Hirid-icu-benchmark – a comprehensive machine learning benchmark on high-resolution icu data, 2022.
- Self-supervised contrastive pre-training for time series via time-frequency consistency, 2022.