Deep Copula-Based Survival Analysis for Dependent Censoring with Identifiability Guarantees (2312.15566v4)
Abstract: Censoring is the central problem in survival analysis where either the time-to-event (for instance, death), or the time-tocensoring (such as loss of follow-up) is observed for each sample. The majority of existing machine learning-based survival analysis methods assume that survival is conditionally independent of censoring given a set of covariates; an assumption that cannot be verified since only marginal distributions is available from the data. The existence of dependent censoring, along with the inherent bias in current estimators has been demonstrated in a variety of applications, accentuating the need for a more nuanced approach. However, existing methods that adjust for dependent censoring require practitioners to specify the ground truth copula. This requirement poses a significant challenge for practical applications, as model misspecification can lead to substantial bias. In this work, we propose a flexible deep learning-based survival analysis method that simultaneously accommodate for dependent censoring and eliminates the requirement for specifying the ground truth copula. We theoretically prove the identifiability of our model under a broad family of copulas and survival distributions. Experiments results from a wide range of datasets demonstrate that our approach successfully discerns the underlying dependency structure and significantly reduces survival estimation bias when compared to existing methods.
- Generating survival times to simulate Cox proportional hazards models. Statistics in medicine, 24(11): 1713–1723.
- Brier, G. W. 1950. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1): 1–3.
- Calibration: the Achilles heel of predictive analytics. BMC Medicine, 17(1).
- Neural likelihoods via cumulative distribution functions. In Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), volume 124 of Proceedings of Machine Learning Research, 420–429. PMLR.
- Cox, D. R. 1972. Regression models and life-tables. Journal of the Royal Statistical Society: Series B (Methodological), 34(2): 187–202.
- Dependent censoring based on parametric copulas. Biometrika, asac06.
- Davidson-Pilon, C. 2023. lifelines, survival analysis in Python.
- On semiparametric modelling, estimation and inference for survival data subject to dependent censoring. Biometrika, 108(4): 965–979.
- Copula based cox proportional hazards models for dependent censoring. Journal of the American Statistical Association, 1–11.
- Automated versus do-it-yourself methods for causal inference: lessons learned from a data analysis competition. Statistical Science, 34(1): 43–68.
- Understanding survival analysis in clinical trials. Clinical Oncology, 33(1): 12–14.
- Analysis of survival data with dependent censoring: copula-based approaches. Springer.
- A neural network model for survival data. Statistics in Medicine, 14(1): 73–82.
- Copula-based deep survival models for dependent censoring. In Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI), volume 216 of Proceedings of Machine Learning Research, 669–680. PMLR.
- Harrell, F. E. 1982. Evaluating the yield of medical tests. JAMA: The Journal of the American Medical Association, 247(18): 2543.
- Hill, J. L. 2011. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1): 217–240.
- Enrollment, retention, and visit attendance in the university of north carolina center for AIDS research HIV clinical cohort, 2001–2007. AIDS Research and Human Retroviruses, 26(8): 875–881.
- Improved Estimates of Cancer-Specific Survival Rates From Population-Based Data. JNCI: Journal of the National Cancer Institute, 102(20): 1584–1598.
- Random survival forests. The Annals of Applied Statistics, 2(3).
- Relaxing the independent censoring assumption in the Cox proportional hazards model using multiple imputation. Statistics in Medicine, 33(27): 4681–4694.
- Nonparametric estimation from incomplete observations. Journal of the American Statistical Association, 53(282): 457–481.
- DeepSurv: personalized treatment recommender system using a cox proportional hazards deep neural network. BMC Medical Research Methodology, 18(1).
- Kimberling, C. H. 1974. A probabilistic interpretation of complete monotonicity. Aequationes Mathematicae, 10(2-3): 152–164.
- Kleinbaum, D. G. 2012. Survival analysis a self-learning text. Springer.
- Koyama, Y. M. 2023. Exponential sum approximations of finite completely monotonic functions. arXiv:2301.08931.
- Continuous and discrete-time survival prediction with neural networks. Lifetime Data Analysis, 27(4): 710–736.
- Time-to-event prediction with neural networks and cox regression. Journal of Machine Learning Research, 20(129): 1–30.
- Lagakos, S. W. 1979. General right censoring and its impact on the analysis of survival data. Biometrics, 35: 139–156.
- Deephit: a deep learning approach to survival analysis with competing risks. Proceedings of the AAAI Conference on Artificial Intelligence, 32: 2314–2321.
- Censoring issues in survival analysis. Annual Review of Public Health, 18(1): 83–104.
- Deep archimedean copulas. In Advances in Neural Information Processing Systems, volume 33, 1535–1545.
- Decoupled weight decay regularization. In International Conference on Learning Representations.
- Multivariate archimedean copulas, d-monotone functions and ℓℓ\ellroman_ℓ1-norm symmetric distributions. The Annals of Statistics, 37(5B).
- A copula-based boosting model for time-to-event prediction with dependent censoring. arXiv:2210.04869.
- Nelsen, R. B. 1998. An introduction to copulas. Springer. ISBN 9780387986234.
- Predicting good probabilities with supervised learning. In Proceedings of the 22nd international conference on Machine learning - ICML '05, 625–632.
- Empirical examination of the ability of children to consent to clinical research. Journal of Medical Ethics, 24(3): 158–165.
- Validating causal inference methods. In Chaudhuri, K.; Jegelka, S.; Song, L.; Szepesvari, C.; Niu, G.; and Sabato, S., eds., Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, 17346–17358. PMLR.
- Automatic Differentiation in PyTorch. In NIPS 2017 Workshop on Autodiff.
- Pearl, J. 2009. Causality. Cambridge, England: Cambridge University Press.
- Pölsterl, S. 2020. Scikit-survival: a library for time-to-event analysis built on top of scikit-learn. Journal of Machine Learning Research, 21(212): 1–6.
- Survival regression with proper scoring rules and monotonic neural networks. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, 1190–1205. PMLR.
- Rubin, D. B. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5): 688–701.
- Adjusting for nonignorable drop-out using semiparametric nonresponse models. Journal of the American Statistical Association, 94(448): 1096–1120.
- Simulating copulas: stochastic models, sampling algorithms, and applications. Imperial College Press.
- Randomized 2 x 2 trial evaluating hormonal treatment and the duration of chemotherapy in node-positive breast cancer patients. German breast cancer study group. Journal of Clinical Oncology, 12(10): 2086–2093.
- Selvin, S. 2008. Survival analysis for epidemiologic and medical research. Cambridge University Press.
- Sklar, A. 1959. Fonctions de répartition à n dimensions et leurs marges. Publications de l’Institut Statistique de l’Université de Paris, 8: 229–231.
- Soden: a scalable continuous-time survival model through ordinary differential equation networks. Journal of Machine Learning Research, 23(34): 1–29.
- Informative censoring — a neglected cause of bias in oncology trials. Nature Reviews Clinical Oncology, 17(6): 327–328.
- Thomas Brooks, D. P. 1989. Airfoil self-noise.
- Tsiatis, A. 1975. A nonidentifiability aspect of the problem of competing risks. Proceedings of the National Academy of Sciences, 72(1): 20–22.
- Efficient energy consumption prediction model for a data analytic-enabled industry building in a smart city. Building Research & Information, 49(1): 127–143.
- Data-driven battery lifetime prediction and confidence estimation for heavy-duty trucks. IEEE Transactions on Reliability, 67(2): 623–639.
- Machine learning for survival analysis. ACM Computing Surveys, 51(6): 1–36.
- Widder, D. V. 2010. The laplace transform. Dover Publications.
- Models for Censored Survival Analysis: Constant-Sum and Variable-Sum Model. Biometrika, 64(2): 215.
- Mining heterogeneous causal effects for personalized cancer treatment. Bioinformatics, 33(15): 2372–2378.
- A unified survey of treatment effect heterogeneity modelling and uplift modelling. ACM Computing Surveys, 54(8): 1–36.
- Weijia Zhang (52 papers)
- Chun Kai Ling (22 papers)
- Xuanhui Zhang (2 papers)