Efficient Algorithms and Implementation of a Semiparametric Joint Model for Longitudinal and Competing Risks Data: With Applications to Massive Biobank Data (2110.14822v3)

Abstract: Semiparametric joint models of longitudinal and competing risks data are computationally costly and their current implementations do not scale well to massive biobank data. This paper identifies and addresses some key computational barriers in a semiparametric joint model for longitudinal and competing risks survival data. By developing and implementing customized linear scan algorithms, we reduce the computational complexities from $O(n^2)$ or $O(n^3)$ to $O(n)$ in various components including numerical integration, risk set calculation, and standard error estimation, where $n$ is the number of subjects. Using both simulated and real world biobank data, we demonstrate that these linear scan algorithms generate drastic speed-up of up to hundreds of thousands fold when $n>10^4$, sometimes reducing the run-time from days to minutes. We have developed an R-package, FastJM, based on the proposed algorithms for joint modeling of longitudinal and time-to-event data with and without competing risks, and made it publicly available on the Comprehensive R Archive Network (CRAN).