Robust Survival Analysis with Adversarial Regularization (2312.16019v4)
Abstract: Survival Analysis (SA) models the time until an event occurs, with applications in fields like medicine, defense, finance, and aerospace. Recent research indicates that Neural Networks (NNs) can effectively capture complex data patterns in SA, whereas simple generalized linear models often fall short in this regard. However, dataset uncertainties (e.g., noisy measurements, human error) can degrade NN model performance. To address this, we leverage advances in NN verification to develop training objectives for robust, fully-parametric SA models. Specifically, we propose an adversarially robust loss function based on a Min-Max optimization problem. We employ CROWN-Interval Bound Propagation (CROWN-IBP) to tackle the computational challenges inherent in solving this Min-Max problem. Evaluated over 10 SurvSet datasets, our method, Survival Analysis with Adversarial Regularization (SAWAR), consistently outperforms baseline adversarial training methods and state-of-the-art (SOTA) deep SA models across various covariate perturbations with respect to Negative Log Likelihood (NegLL), Integrated Brier Score (IBS), and Concordance Index (CI) metrics. Thus, we demonstrate that adversarial robustness enhances SA predictive performance and calibration, mitigating data uncertainty and improving generalization across diverse datasets by up to 150% compared to baselines.
- P. Wang, Y. Li, and C. K. Reddy, “Machine Learning for Survival Analysis: A Survey,” ACM Comput. Surv., vol. 51, no. 6, feb 2019. [Online]. Available: https://doi.org/10.1145/3214306
- S. Salerno and Y. Li, “High-dimensional survival analysis: Methods and applications,” Annual review of statistics and its application, vol. 10, pp. 25–49, 2023.
- J. B. Greenhouse, D. Stangl, and J. Bromberg, “An introduction to survival analysis: statistical methods for analysis of clinical trial data.” Journal of Consulting and Clinical Psychology, vol. 57, no. 4, p. 536, 1989.
- C. Brard, G. Le Teuff, M.-C. Le Deley, and L. V. Hampson, “Bayesian survival analysis in clinical trials: What methods are used in practice?” Clinical Trials, vol. 14, no. 1, pp. 78–87, 2017.
- D. Altman, B. De Stavola, S. Love, and K. Stepniewska, “Review of survival analyses published in cancer journals,” British journal of cancer, vol. 72, no. 2, pp. 511–518, 1995.
- D. R. Cox, “The regression analysis of binary sequences,” Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 20, no. 2, pp. 215–232, 1958.
- B. Cheng and M. Potter, “Bayesian Weapon System Reliability Modeling with Cox-Weibull Neural Network,” in 2023 Annual Reliability and Maintainability Symposium (RAMS), 2023, pp. 1–6.
- R. C. Deo, “Machine learning in medicine,” Circulation, vol. 132, no. 20, pp. 1920–1930, 2015.
- G. Handelman, H. Kok, R. Chandra, A. Razavi, M. Lee, and H. Asadi, “ed octor: machine learning and the future of medicine,” Journal of internal medicine, vol. 284, no. 6, pp. 603–619, 2018.
- A. Qayyum, J. Qadir, M. Bilal, and A. Al-Fuqaha, “Secure and robust machine learning for healthcare: A survey,” IEEE Reviews in Biomedical Engineering, vol. 14, pp. 156–180, 2020.
- F. J. Rodríguez-Vera, Y. Marín, A. Sánchez, C. Borrachero, and E. Pujol, “Illegible handwriting in medical records,” J R Soc Med, vol. 95, no. 11, pp. 545–546, Nov 2002.
- J. Li, Y. Mao, and J. Zhang, “Maintenance and quality control of medical equipment based on information fusion technology,” Comput Intell Neurosci, vol. 2022, p. 9333328, Oct 2022.
- L. V. Utkin, V. S. Zaborovsky, M. S. Kovalev, A. V. Konstantinov, N. A. Politaeva, and A. A. Lukashin, “Uncertainty interpretation of the machine learning survival model predictions,” IEEE Access, vol. 9, pp. 120 158–120 175, 2021.
- E. Liu and K. Lim, “Using the weibull accelerated failure time regression model to predict time to health events,” bioRxiv, 2018. [Online]. Available: https://www.biorxiv.org/content/early/2018/08/27/362186
- A. Schwarzschild, M. Goldblum, A. Gupta, J. P. Dickerson, and T. Goldstein, “Just how toxic is data poisoning? a unified benchmark for backdoor and data poisoning attacks,” in International Conference on Machine Learning. PMLR, 2021, pp. 9389–9398.
- P. Chapfuwa, C. Tao, C. Li, C. Page, B. Goldstein, L. C. Duke, and R. Henao, “Adversarial time-to-event modeling,” in Proceedings of the 35th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, J. Dy and A. Krause, Eds., vol. 80. PMLR, 10–15 Jul 2018, pp. 735–744. [Online]. Available: https://proceedings.mlr.press/v80/chapfuwa18a.html
- S. Wang, H. Zhang, K. Xu, X. Lin, S. Jana, C.-J. Hsieh, and J. Z. Kolter, “Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification,” Advances in Neural Information Processing Systems, vol. 34, pp. 29 909–29 921, 2021.
- H. Zhang, H. Chen, C. Xiao, S. Gowal, R. Stanforth, B. Li, D. Boning, and C.-J. Hsieh, “Towards stable and efficient training of verifiably robust neural networks,” 2019.
- S. Wiegrebe, P. Kopper, R. Sonabend, B. Bischl, and A. Bender, “Deep Learning for Survival Analysis: A Review,” 2023.
- D. Faraggi and R. Simon, “A neural network model for survival data,” Statistics in Medicine, vol. 14, no. 1, pp. 73–82, 1995. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.4780140108
- T. Ching, X. Zhu, and L. X. Garmire, “Cox-nnet: An artificial neural network method for prognosis prediction of high-throughput omics data,” PLOS Computational Biology, vol. 14, no. 4, pp. 1–18, 04 2018. [Online]. Available: https://doi.org/10.1371/journal.pcbi.1006076
- J. P. Klein and P. Goel, “Survival analysis: state of the art,” 1992.
- J. L. Katzman, U. Shaham, A. Cloninger, J. Bates, T. Jiang, and Y. Kluger, “Deepsurv: personalized treatment recommender system using a cox proportional hazards deep neural network,” BMC Medical Research Methodology, vol. 18, no. 1, p. 24, Feb 26 2018. [Online]. Available: https://doi.org/10.1186/s12874-018-0482-1
- A. Bennis, S. Mouysset, and M. Serrurier, “Estimation of conditional mixture weibull distribution with right censored data using neural network for time-to-event analysis,” in Advances in Knowledge Discovery and Data Mining: 24th Pacific-Asia Conference, PAKDD 2020, Singapore, May 11–14, 2020, Proceedings, Part I 24. Springer, 2020, pp. 687–698.
- H. Haider, B. Hoehn, S. Davis, and R. Greiner, “Effective ways to build and evaluate individual survival distributions,” The Journal of Machine Learning Research, vol. 21, no. 1, pp. 3289–3351, 2020.
- M. Goldstein, X. Han, A. Puli, A. Perotte, and R. Ranganath, “X-cal: Explicit calibration for survival analysis,” Advances in neural information processing systems, vol. 33, pp. 18 296–18 307, 2020.
- F. Kamran and J. Wiens, “Estimating calibrated individualized survival curves with deep learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 1, 2021, pp. 240–248.
- C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” in International conference on machine learning. PMLR, 2017, pp. 1321–1330.
- I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” arXiv preprint arXiv:1412.6572, 2014.
- R. Vasilev and A. D’yakonov, “Calibration of neural networks,” 2023.
- Y. Qin, X. Wang, A. Beutel, and E. Chi, “Improving calibration through the relationship with adversarial robustness,” Advances in Neural Information Processing Systems, vol. 34, pp. 14 358–14 369, 2021.
- T. Uemura, J. J. Näppi, C. Watari, T. Hironaka, T. Kamiya, and H. Yoshida, “Weakly unsupervised conditional generative adversarial network for image-based prognostic prediction for covid-19 patients based on chest ct,” Medical Image Analysis, vol. 73, p. 102159, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S136184152100205X
- P. Liu, L. Ji, F. Ye, and B. Fu, “Advmil: Adversarial multiple instance learning for the survival analysis on whole-slide images,” Medical Image Analysis, vol. 91, p. 103020, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1361841523002803
- I. Goodfellow, “Nips 2016 tutorial: Generative adversarial networks,” 2017.
- R. Bender, T. Augustin, and M. Blettner, “Generating survival times to simulate cox proportional hazards models,” Statistics in medicine, vol. 24, no. 11, pp. 1713–1723, 2005.
- C. Stanley, E. Molyneux, and M. Mukaka, “Comparison of performance of exponential, cox proportional hazards, weibull and frailty survival models for analysis of small sample size data,” Journal of Medical Statistics and Informatics, vol. 4, no. 1, 2016.
- P. Royston, “Flexible parametric alternatives to the cox model, and more,” The Stata Journal, vol. 1, no. 1, pp. 1–28, 2001.
- R. B. Cooper, “Queueing theory,” in Proceedings of the ACM’81 conference, 1981, pp. 119–122.
- K. Das, “A comparative study of exponential distribution vs weibull distribution in machine reliability analysis in a cms design,” Computers & Industrial Engineering, vol. 54, no. 1, pp. 12–33, 2008.
- S. P. Jenkins, “Survival analysis,” Unpublished manuscript, Institute for Social and Economic Research, University of Essex, Colchester, UK, vol. 42, pp. 54–56, 2005.
- G. Rodrıguez, “Parametric survival models,” Rapport technique, Princeton: Princeton University, 2010.
- D. R. Cox, “Regression Models and Life-Tables,” Journal of the Royal Statistical Society. Series B (Methodological), vol. 34, no. 2, pp. 187–220, 1972. [Online]. Available: http://www.jstor.org/stable/2985181
- W. Lowe, “Rare events research,” in Encyclopedia of Social Measurement, K. Kempf-Leonard, Ed. New York: Elsevier, 2005, pp. 293–297. [Online]. Available: https://www.sciencedirect.com/science/article/pii/B0123693985005570
- C. Lee, J. Yoon, and M. Van Der Schaar, “Dynamic-deephit: A deep learning approach for dynamic survival analysis with competing risks based on longitudinal data,” IEEE Transactions on Biomedical Engineering, vol. 67, no. 1, pp. 122–133, 2019.
- C. Lee, W. Zame, J. Yoon, and M. Van Der Schaar, “Deephit: A deep learning approach to survival analysis with competing risks,” in Proceedings of the AAAI conference on artificial intelligence, vol. 32 Issued 1, 2018.
- S. Pölsterl, “scikit-survival: A Library for Time-to-Event Analysis Built on Top of scikit-learn,” Journal of Machine Learning Research, vol. 21, no. 212, pp. 1–6, 2020. [Online]. Available: http://jmlr.org/papers/v21/20-729.html
- P. Chapfuwa, C. Tao, C. Li, I. Khan, K. J. Chandross, M. J. Pencina, L. Carin, and R. Henao, “Calibration and Uncertainty in Neural Time-to-Event Modeling,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 4, pp. 1666–1680, 2023.
- A. D. Palma, R. Bunel, K. Dvijotham, M. P. Kumar, and R. Stanforth, “IBP Regularization for Verified Adversarial Robustness via Branch-and-Bound,” 2023.
- A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards Deep Learning Models Resistant to Adversarial Attacks,” 2019.
- S. Gowal, K. Dvijotham, R. Stanforth, R. Bunel, C. Qin, J. Uesato, R. Arandjelovic, T. A. Mann, and P. Kohli, “On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models,” CoRR, vol. abs/1810.12715, 2018. [Online]. Available: http://arxiv.org/abs/1810.12715
- K. Xu, Z. Shi, H. Zhang, Y. Wang, K.-W. Chang, M. Huang, B. Kailkhura, X. Lin, and C.-J. Hsieh, “Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond,” 2020.
- E. Rusak, L. Schott, R. S. Zimmermann, J. Bitterwolf, O. Bringmann, M. Bethge, and W. Brendel, “A simple way to make neural networks robust against diverse image corruptions,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16. Springer, 2020, pp. 53–69.
- Zhang, Guojun, “Understanding Minimax Optimization in Modern Machine Learning,” Ph.D. dissertation, University of Waterloo, 2021. [Online]. Available: http://hdl.handle.net/10012/17157
- D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2017.
- Y. Yao, L. Rosasco, and A. Caponnetto, “On Early Stopping in Gradient Descent Learning,” Constructive Approximation, vol. 26, no. 2, pp. 289–315, August 1 2007. [Online]. Available: https://doi.org/10.1007/s00365-006-0663-2
- J. Harrell, Frank E., R. M. Califf, D. B. Pryor, K. L. Lee, and R. A. Rosati, “Evaluating the Yield of Medical Tests,” JAMA, vol. 247, no. 18, pp. 2543–2546, 05 1982. [Online]. Available: https://doi.org/10.1001/jama.1982.03320430047030
- E. Graf, C. Schmoor, W. Sauerbrei, and M. Schumacher, “Assessment and comparison of prognostic classification schemes for survival data,” Statistics in Medicine, vol. 18, no. 17-18, pp. 2529–2545, 1999.
- C. Davidson-Pilon, “lifelines: survival analysis in python,” Journal of Open Source Software, vol. 4, no. 40, p. 1317, 2019. [Online]. Available: https://doi.org/10.21105/joss.01317
- E. Drysdale, “SurvSet: An open-source time-to-event dataset repository,” 2022.
- G. V. H. Jensen, C. Torp-Pedersen, P. Hildebrandt, L. Kober, F. Nielsen, T. Melchior, T. Joen, and P. Andersen, “Does in-hospital ventricular fibrillation affect prognosis after myocardial infarction?” European heart journal, vol. 18, no. 6, pp. 919–924, 1997.
- “Stage C Prostate Cancer,” {https://rdrr.io/cran/rpart/man/stagec.html}.
- R. A. Kyle, T. M. Therneau, S. V. Rajkumar, D. R. Larson, M. F. Plevak, J. R. Offord, A. Dispenzieri, J. A. Katzmann, and L. J. Melton III, “Prevalence of monoclonal gammopathy of undetermined significance,” New England Journal of Medicine, vol. 354, no. 13, pp. 1362–1369, 2006.
- T. J. Wang, J. M. Massaro, D. Levy, R. S. Vasan, P. A. Wolf, R. B. D’Agostino, M. G. Larson, W. B. Kannel, and E. J. Benjamin, “A risk score for predicting stroke or death in individuals with new-onset atrial fibrillation in the community: the framingham heart study,” Jama, vol. 290, no. 8, pp. 1049–1056, 2003.
- “dataDIVAT1,” {https://rdrr.io/cran/RISCA/man/dataDIVAT1.html}.
- “Prostate dataset,” {https://hbiostat.org/data/repo/cprostate}.
- C. C. Abnet, B. Lai, Y.-L. Qiao, S. Vogt, X.-M. Luo, P. R. Taylor, Z.-W. Dong, S. D. Mark, and S. M. Dawsey, “Zinc concentration in esophageal biopsy specimens measured by x-ray fluorescence and esophageal cancer risk,” Journal of the National Cancer Institute, vol. 97, no. 4, pp. 301–306, 2005.
- A. Blair, D. Hadden, J. Weaver, D. Archer, P. Johnston, and C. Maguire, “The 5-year prognosis for vision in diabetes.” The Ulster medical journal, vol. 49, no. 2, p. 139, 1980.
- R. Henderson, S. Shimakura, and D. Gorst, “Modeling spatial variation in leukemia survival data,” Journal of the American Statistical Association, vol. 97, no. 460, pp. 965–972, 2002.