Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SMART: Towards Pre-trained Missing-Aware Model for Patient Health Status Prediction (2405.09039v1)

Published 15 May 2024 in cs.LG

Abstract: Electronic health record (EHR) data has emerged as a valuable resource for analyzing patient health status. However, the prevalence of missing data in EHR poses significant challenges to existing methods, leading to spurious correlations and suboptimal predictions. While various imputation techniques have been developed to address this issue, they often obsess unnecessary details and may introduce additional noise when making clinical predictions. To tackle this problem, we propose SMART, a Self-Supervised Missing-Aware RepresenTation Learning approach for patient health status prediction, which encodes missing information via elaborated attentions and learns to impute missing values through a novel self-supervised pre-training approach that reconstructs missing data representations in the latent space. By adopting missing-aware attentions and focusing on learning higher-order representations, SMART promotes better generalization and robustness to missing data. We validate the effectiveness of SMART through extensive experiments on six EHR tasks, demonstrating its superiority over state-of-the-art methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. Attend and diagnose: Clinical time series analysis using attention models. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  2. Hitanet: Hierarchical time-aware attention networks for risk prediction on electronic health records. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 647–656, 2020.
  3. Distilling knowledge from publicly available online emr data to emerging epidemic for prognosis. In Proceedings of the Web Conference 2021, pages 3558–3568, 2021.
  4. Vecocare: visit sequences-clinical notes joint learning for diagnosis prediction in healthcare data. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pages 4921–4929, 2023.
  5. Seqcare: Sequential training with external medical knowledge graph for diagnosis prediction in healthcare data. In Proceedings of the ACM Web Conference 2023, pages 2819–2830, 2023.
  6. Graphcare: Enhancing healthcare predictions with personalized knowledge graphs. In The Twelfth International Conference on Learning Representations, 2024.
  7. Deep representation learning of electronic health records to unlock patient stratification at scale. NPJ digital medicine, 3(1):96, 2020.
  8. Deep patient: an unsupervised representation to predict the future of patients from the electronic health records. Scientific reports, 6(1):1–10, 2016.
  9. Stagenet: Stage-aware neural networks for health risk prediction. In Proceedings of The Web Conference 2020, pages 530–540, 2020.
  10. Adacare: Explainable clinical health status representation learning via scale-adaptive feature extraction and recalibration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 825–832, 2020.
  11. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. Advances in neural information processing systems, 29, 2016.
  12. Patient subtyping via time-aware lstm networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 65–74, 2017.
  13. Concare: Personalized clinical feature embedding via capturing the healthcare context. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 833–840, 2020.
  14. Interpretable representation learning for healthcare via capturing disease progression through time. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 43–51, 2018.
  15. Mortality prediction with adaptive feature importance recalibration for peritoneal dialysis patients. Patterns, 4(12), 2023.
  16. Grasp: generic framework for health status representation learning based on incorporating knowledge from similar patients. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 715–723, 2021.
  17. Patient health representation learning via correlational sparse prior of medical features. IEEE Transactions on Knowledge and Data Engineering, 2022.
  18. Multi-time attention networks for irregularly sampled time series. In International Conference on Learning Representations, 2021.
  19. Latent ordinary differential equations for irregularly-sampled time series. Advances in neural information processing systems, 32, 2019.
  20. Probabilistic imputation for time-series classification with missing data. In International Conference on Machine Learning, pages 16654–16667. PMLR, 2023.
  21. Graph-guided network for irregularly sampled multivariate time series. In International Conference on Learning Representations, 2022.
  22. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  23. Interpolation-prediction networks for irregularly sampled time series. In International Conference on Learning Representations, 2019.
  24. Generative semi-supervised learning for multivariate time series imputation. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 8983–8991, 2021.
  25. Primenet: Pre-training for irregular multivariate time series. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 7184–7192, 2023.
  26. Real-time prediction of mortality, readmission, and length of stay using electronic health record data. Journal of the American Medical Informatics Association, 23(3):553–561, 2016.
  27. Health-atm: A deep architecture for multifaceted patient health record representation and risk prediction. In Proceedings of the 2018 SIAM International Conference on Data Mining, pages 261–269. SIAM, 2018.
  28. Deep representation learning of patient data from electronic health records (ehr): A systematic review. Journal of biomedical informatics, 115:103671, 2021.
  29. M3care: Learning with missing modalities in multimodal healthcare data. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2418–2428, 2022.
  30. Camp: Co-attention memory networks for diagnosis prediction in healthcare. In 2019 IEEE international conference on data mining (ICDM), pages 1036–1041. IEEE, 2019.
  31. Improving medical code prediction from clinical text via incorporating online knowledge sources. In The World Wide Web Conference, pages 72–82, 2019.
  32. Blood pressure prediction via recurrent models with contextual layer. In Proceedings of the 26th International Conference on World Wide Web, pages 685–693, 2017.
  33. Raim: Recurrent attentive and intensive model of multimodal patient monitoring data. In Proceedings of the 24th ACM SIGKDD international conference on Knowledge Discovery & Data Mining, pages 2565–2573, 2018.
  34. Dipole: Diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1903–1911, 2017.
  35. Scehr: Supervised contrastive learning for clinical risk prediction using electronic health records. In Proceedings. IEEE International Conference on Data Mining, volume 2021, page 857. NIH Public Access, 2021.
  36. Completing missing prevalence rates for multiple chronic diseases by jointly leveraging both intra-and inter-disease population health data correlations. In Proceedings of the Web Conference 2021, pages 183–193, 2021.
  37. Context-aware health event prediction via transition functions on dynamic disease graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 4567–4574, 2022.
  38. Patient2vec: A personalized interpretable deep representation of the longitudinal electronic health record. IEEE Access, 6:65333–65346, 2018.
  39. Gram: graph-based attention model for healthcare representation learning. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 787–795, 2017.
  40. Kame: Knowledge-based attention model for diagnosis prediction in healthcare. In Proceedings of the 27th ACM international conference on information and knowledge management, pages 743–752, 2018.
  41. Medml: fusing medical knowledge and machine learning models for early pediatric covid-19 hospitalization and severity prediction. Iscience, 25(9), 2022.
  42. Predict and interpret health risk using ehr through typical patients. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1506–1510. IEEE, 2024.
  43. Collaborative graph learning with auxiliary text for temporal event prediction in healthcare. 2021.
  44. Medpath: Augmenting health risk prediction via medical knowledge paths. In Proceedings of the Web Conference 2021, pages 1397–1409, 2021.
  45. Adversarial joint-learning recurrent neural network for incomplete time series classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(4):1765–1776, 2020.
  46. Data-gru: Dual-attention time-aware gated recurrent unit for irregular multivariate time series. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 930–937, 2020.
  47. Phased lstm: Accelerating recurrent network training for long or event-based sequences. Advances in neural information processing systems, 29, 2016.
  48. Recurrent neural networks for multivariate time series with missing values. Scientific reports, 8(1):6085, 2018.
  49. Set functions for time series. In International Conference on Machine Learning, pages 4353–4363. PMLR, 2020.
  50. Warpformer: A multi-scale modeling approach for irregular clinical time series. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3273–3285, 2023.
  51. Missing value imputation in multivariate time series with end-to-end generative adversarial networks. Information Sciences, 551:67–82, 2021.
  52. Contiformer: Continuous-time transformer for irregular time series modeling. Advances in Neural Information Processing Systems, 36, 2024.
  53. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations, 2023.
  54. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171–4186, 2019.
  55. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.
  56. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  57. Self-supervised learning from images with a joint-embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15619–15629, 2023.
  58. J Stuart Hunter. The exponentially weighted moving average. Journal of quality technology, 18(4):203–210, 1986.
  59. Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals. circulation, 101(23):e215–e220, 2000.
  60. Predicting in-hospital mortality of icu patients: The physionet/computing in cardiology challenge 2012. In 2012 Computing in Cardiology, pages 245–248. IEEE, 2012.
  61. Early prediction of sepsis from clinical data: the physionet/computing in cardiology challenge 2019. Critical care medicine, 48(2):210–217, 2020.
  62. Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1–9, 2016.
  63. The relationship between precision-recall and roc curves. In Proceedings of the 23rd international conference on Machine learning, pages 233–240, 2006.
  64. Multitask learning and benchmarking with clinical time series data. Scientific Data, 6(1):96, 2019.
  65. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets