--- title: Time-dependent ROC Analysis url: https://www.emergentmind.com/topics/time-dependent-receiver-operating-characteristics-roc-analysis type: topic --- # Time-dependent ROC Analysis Time-dependent receiver operating characteristic (ROC) analysis is a framework for evaluating the discriminative performance of a biomarker, marker, or risk score when the outcome is time to event rather than a static binary endpoint. Its central premise is that case/control status evolves with time \(t\): a subject may be event-free at one horizon and become a case later. Accordingly, sensitivity, specificity, ROC curves, and area under the curve (AUC) are indexed by time, and their estimation must accommodate censoring, delayed entry, and, in some settings, time-dependent covariates or study-specific thresholds. Across the literature, two main formulations recur—the cumulative/dynamic definition and the incident/dynamic definition—and recent work has extended the framework to partial AUC, interval-censored data, covariate-specific ROC curves, left-truncated right-censored data, and meta-analytic summary ROC analysis [1103.1963] [1806.01760] [1809.05627] [2506.13604] [2305.19741] [2509.05693]. ## 1. Conceptual basis and principal case/control formulations In survival settings, ordinary ROC analysis is inadequate because it is designed for a binary outcome observed at the same time as the marker, whereas in time-to-event studies all subjects are event-free at baseline, event status evolves over time, and some subjects are censored before their eventual status is known [2506.13604]. Time-dependent ROC analysis therefore evaluates discrimination at a specified time \(t\), with cases and controls defined relative to survival status by that horizon or instant. Under the **cumulative/dynamic** formulation, used in several of the cited works, cases at time \(t\) are subjects with \(T \le t\), and controls are subjects with \(T > t\) [1806.01760] [1103.1963] [2506.13604] [2509.05693]. For a threshold \(m\) or \(y\), one representation is \[ TP_t(m)=P(M>m\mid T\le t), \qquad FP_t(m)=P(M>m\mid T>t), \] and the time-dependent ROC curve and AUC are \[ ROC_t(p)=TP_t\{FP_t^{-1}(p)\}, \qquad AUC_t=\int_0^1 ROC_t(p)\,dp. \] Equivalent survivor-function formulations are also used: \[ TPR_t(y)=P(Y>y\mid T\le t), \qquad FPR_t(y)=P(Y>y\mid T>t). \] This definition evaluates how well the marker separates subjects who fail by \(t\) from those who survive beyond \(t\) [1103.1963]. Under the **incident/dynamic** formulation, the classification problem is posed on the survivor population at each time \(t\): a case is a subject with event at time \(t\), \(T=t\), and a control is a subject surviving beyond \(t\), \(T>t\) [1809.05627]. For a scalar score \(g(Z(t))\), \[ FPR_t(c)=P\{g(Z(t))>c\mid T>t\}, \qquad TPR_t(c)=P\{g(Z(t))>c\mid T=t\}, \] and \[ ROC_t(q)=TPR_t\!\big(FPR_t^{-1}(q)\big). \] This version is explicitly dynamic because the relevant covariate space is the survivor population at time \(t\), and in the tree-based survival framework the natural risk score is the hazard \(\lambda(t\mid Z(t))\) [1809.05627]. These formulations are not interchangeable. The cumulative/dynamic definition compares failure by time \(t\) against survival beyond \(t\), whereas the incident/dynamic definition compares failure at time \(t\) against survival beyond \(t\). A plausible implication is that the choice depends on whether the scientific target is cumulative prognosis up to a horizon or instantaneous discrimination among subjects still at risk. ## 2. Core functionals: ROC, AUC, partial AUC, and dynamic concordance The pointwise time-dependent AUC, \[ AUC_t=\int_0^1 ROC_t(p)\,dp, \] is the standard scalar summary of discrimination at time \(t\) under cumulative/dynamic definitions [1806.01760] [2506.13604]. In interval-censored settings, the same target is retained, but estimation proceeds through the joint distribution \(F(t,m)\) of event time and marker and the marker marginal \(F_2(m)\), because exact failure times are unavailable [1806.01760]. A major refinement is the **time-dependent partial area under the ROC curve (pAUC)**, developed for right-censored survival data under cumulative/dynamic case/control definitions [1103.1963]. For \(\alpha\in(0,1]\), let \[ q_{\alpha t}=FPR_t^{-1}(\alpha)=\inf\{y: FPR_t(y)\le \alpha\}. \] The target pAUC over the region \(FPR_t(y)\le \alpha\) is written as \[ \theta_t(q_{\alpha t}) = \Theta_\alpha(S), \] with \[ \Theta_{\alpha}(S)=\frac{-\int (S(0,u)-S(t,u))\,I(u\ge q_{\alpha t})\, d_u S(t,u)} {S_T(t)(1-S_T(t))}. \] If \(\alpha=1\), the time-dependent pAUC reduces to the full time-dependent AUC [1103.1963]. The rescaled quantity \(\theta_t(q_{\alpha t})/\alpha\) has a probability interpretation: \[ P(Y_i > Y_j \mid T_i \le t,\; T_j > t,\; Y_j > q_{\alpha t}), \qquad i\ne j. \] The same paper notes the benchmark values \(\theta_t(q_{\alpha t})=\alpha\) for a perfect biomarker and \(\theta_t(q_{\alpha t})=0.5\alpha^2\) for a useless biomarker [1103.1963]. For discrete-valued prognostic scores, especially tree-based scores, ordinary time-dependent ROC curves may degenerate to finitely many points. To address this, a generalized ROC curve \(ROC_t^*\) can be defined by linear interpolation, interpreted as a randomized classification rule at ties [1809.05627]. For \[ q\in \big(FPR_t(c),\,FPR_t(c-)\big), \] the rule predicts \(T=t\) if \(g(Z(t))>c\), predicts \(T=t\) with probability \[ \frac{q-FPR_t(c)}{FPR_t(c-)-FPR_t(c)} \] if \(g(Z(t))=c\), and predicts \(T>t\) if \(g(Z(t))g(Z_2(t))\mid T_2>T_1=t\} +\frac12 P\{g(Z_1(t))=g(Z_2(t))\mid T_2>T_1=t\}. \] An integrated functional, \[ ICON(\widetilde g)=\int_0^s \omega(t)\,CON_t(\widetilde g(t,\cdot))\,dt, \] serves as a dynamic concordance index over time [1809.05627]. This turns time-dependent ROC analysis from a purely evaluative device into an optimization criterion for learning algorithms. ## 3. Estimation under right censoring and marker-dependent censoring For right-censored survival data with a baseline continuous marker, a nonparametric inferential framework for time-dependent pAUC under marker-dependent censoring assumes \[ T \perp C \mid Y \] and estimates the joint survivor function \[ S(t,y)=P(T>t, Y>y) \] using Akritas’ nearest-neighbor estimator [1103.1963]. The estimator is \[ \widehat{S}(t,y)=\frac{1}{n}\sum_{i=1}^n \widehat{S}_T(t\mid Y_i)\, I(Y_i>y), \] where \[ \widehat{S}_{T}(t\mid y)= \prod_{\{i:X_i\le t,\delta_i=1\} \left\{1-\frac{K_{\lambda}(\widehat{S}_Y(Y_i)-\widehat{S}_Y(y))} {n\widehat{S}_X(X_i\mid y)}\right\}, \] with \[ K_\lambda(u)=\frac{1}{2\lambda}I(|u|<\lambda). \] Plugging \(\widehat S\) into \(\Theta_\alpha(S)\) yields a closed-form estimator of time-dependent pAUC that avoids trapezoidal numerical integration [1103.1963]. This line of work also provides explicit asymptotic theory. Under regularity assumptions including \[ \inf_t f_t(q_{\alpha t})>0, \] the estimator admits a uniform asymptotic linear representation and converges weakly to a Gaussian process, with estimated variance-covariance \[ \widehat\Sigma_\alpha(s,t)=\frac{1}{n}\sum_{i=1}^n \widehat\Psi_{\alpha i}(s)\widehat\Psi_{\alpha i}(t). \] Pointwise confidence intervals and simultaneous confidence bands are then constructed from this influence-function representation [1103.1963]. A distinct recent approach estimates **covariate-specific cumulative/dynamic ROC curves and AUCs** by combining a flexible hazard model for \(T\mid (Y,\boldsymbol X)\) with a flexible model for the biomarker distribution \(Y\mid \boldsymbol X\) [2506.13604]. The conditional survival is represented by \[ S_T\left(t \mid y, \boldsymbol{x}\right) = \exp\left(-\int_{0}^{t}\lambda\left(u \mid y, \boldsymbol x\right)\text{d}u\right), \] with a flexible additive hazard specification \[ \lambda\left(t \mid y, \boldsymbol x\right) = \exp\left(\eta(t \mid y, \boldsymbol{x})\right), \] where \(\eta\) includes smooth main effects and interactions such as \(f_{Y,T}(y,t)\), \(f_{b,T}(\boldsymbol{x}_b,t)\), and \(f_{d,Y}(\boldsymbol{x}_d,y)\). The biomarker model is a location-scale model \[ Y = \mu\left(\boldsymbol{x}\right) + \sigma\left(\boldsymbol{x}\right)\varepsilon, \] with \[ F_Y\left(y \mid \boldsymbol{x} \right) = F_\varepsilon\left(\frac{y - \mu\left(\boldsymbol{x}\right)}{\sigma\left(\boldsymbol{x}\right)}\right). \] Plug-in estimators of cumulative sensitivity and dynamic specificity are then obtained from finite sums over empirical residuals, with ROC computed by linear interpolation and AUC by the composite Simpson’s rule [2506.13604]. This model-based approach uses the survival likelihood and piecewise exponential additive model formulation to handle censoring, assumes non-informative censoring, and relies on penalised splines with smoothing selection by REML; the reported software framework uses **mgcv** and **pammtools**, and the accompanying R package is **CondTimeROC** [2506.13604]. This suggests that conditional time-dependent ROC analysis is increasingly treated as a semiparametric regression problem rather than solely a nonparametric smoothing problem. ## 4. Extensions to interval censoring and left truncation When the event time is **interval censored**, standard right-censored ROC methods cannot be applied directly because exact event times are unavailable, and naive imputation by the midpoint or right endpoint of the censoring interval is biased [1806.01760]. For subject \(i\), the observed data are \[ (u_i,v_i,m_i,\delta_i^{(1)},\delta_i^{(2)},\delta_i^{(3)}), \] where \(\delta_i^{(1)}\), \(\delta_i^{(2)}\), and \(\delta_i^{(3)}\) encode left, interval, and right censoring, and the observation interval \((U,V)\) is assumed independent of \((T,M)\): \[ (U,V)\perp (T,M). \] A fully nonparametric solution estimates the joint distribution \(F(t,m)\) of \((T,M)\) and the marginal \(F_2(m)\) using a spline-based sieve maximum likelihood estimator [1806.01760]. With B-spline basis functions, \[ F_n(t,m)=\sum_{j=1}^{p_n}\sum_{k=1}^{q_n}\alpha_{j,k}B_j^{(1),l}(t)B_k^{(2),l}(m), \qquad F_{n,2}(m)=\sum_{k=1}^{q_n}\beta_k B_k^{(2),l}(m), \] subject to monotonicity and compatibility constraints. For computation, the method is reparameterized using I-splines and M-splines: \[ F_n(t,m)=\sum_{j=1}^{p_n-1}\sum_{k=1}^{q_n-1}\gamma_{j,k}I_j^{(1),l}(t)I_k^{(2),l}(m), \] \[ F_{n,2}(m)=\sum_{k=1}^{q_n-1}\left\{\sum_{j=1}^{p_n-1}\gamma_{j,k}+\omega_k\right\}I_k^{(2),l}(m), \] with constraints \[ \gamma_{j,k}\ge 0,\quad \omega_k\ge 0,\quad \sum_{j,k}\gamma_{j,k}+\sum_k\omega_k\le 1. \] Optimization is performed using the generalized gradient projection algorithm [1806.01760]. Plug-in estimators \(\widehat{ROC}_{n,t}(p)\) and \(\widehat{AUC}_{n,t}\) follow, and under assumptions C1–C4 the paper proves \[ \sup_{p\in[0,1]} \left| \widehat{ROC}_{n,t}(p)-ROC_{0,t}(p) \right| \to_P 0, \qquad \widehat{AUC}_{n,t}\to_P AUC_{0,t}. \] Bootstrap confidence intervals using the BCa method are recommended for inference, and the associated CRAN package is `intcensROC` [1806.01760]. A different extension concerns **left-truncated and right-censored (LTRC)** data, where subjects are observed only if \[ L < \min(T,C). \] In this setting, ignoring left truncation can cause serious bias because the observed sample is conditioned on surviving long enough to enter the study [2509.05693]. Under the cumulative/dynamic definition, \[ Se(c,t)=P(X>c\mid T\leq t), \qquad Sp(c,t)=P(X\leq c\mid T> t), \] and \[ AUC(t)=P(X_i>X_j\mid T_i\leq t,\; T_j>t). \] For independent truncation/censoring, a nonparametric regression estimator uses the left-truncation-adjusted risk set \[ \widehat R(t)=\dfrac{1}{n}\sum_{i=1}^n\mathbbm 1(L_i< t\leq \widetilde T_i) \] and \[ \widehat F_{T,X}(t,c)=\dfrac{1}{n}\sum_{i=1}^n \dfrac{\mathbbm 1(\widetilde T_i\leq t, X_i\leq c, \Delta_i=1)\widehat S_T(\widetilde T_i-)} {\widehat R(\widetilde T_i)}. \] New inverse probability weighting estimators instead weight subjects by the inverse probability of being both untruncated and sufficiently uncensored to contribute as a case or control. For example, \[ \widehat{Se}_{\text{IPW}(c,t) = \dfrac{\sum_{i=1}^n\dfrac{\Delta_i}{\widehat K_1(\widetilde T_i)}\mathbbm 1(X_i> c, \widetilde T_iu\mid T=u)\), and covariate-adjusted conditional IPW versions replace \(K_1\) and \(K_2\) by \(K_{C1}(u,z)\) and \(K_{C2}(t,u,z)\) [2509.05693]. The paper states that right-censored-only estimators ignoring left truncation were often severely biased, whereas conditional IPW estimators performed well when truncation dependence was explained by measured covariates and the nuisance models were correctly specified [2509.05693]. ## 5. Dynamic risk, time-dependent covariates, and learning algorithms Time-dependent ROC analysis is not limited to baseline markers. In ROC-guided survival trees and ensembles, the relevant prediction target is explicitly dynamic: \[ \lambda(t\mid Z(t))\,dt = P\{T\in [t,t+dt)\mid Z(t),\,T\ge t\}, \] and the survivor population at time \(t\) is \[ \mathcal Z_t = \text{support of } Z(t)\text{ among } \{T\ge t\}. \] A time-invariant partition \(T=\{\tau_1,\dots,\tau_M\}\) induces the tree-based hazard model \[ \lambda_T(t\mid Z(t)) = \sum_{\tau\in T} I\{Z(t)\in \tau\}\,\lambda(t\mid \tau), \] so the same subject may move across nodes over time as \(Z(t)\) changes, while the partition remains a fixed decision rule on the current covariates [1809.05627]. A key decision-theoretic result is that among all scalar functions \(g:\mathcal Z_t\to \mathbb R\), \[ g(Z(t))=\lambda(t\mid Z(t)) \] yields the highest \(ROC_t^*\) [1809.05627]. This establishes the hazard as the ROC-optimal time-\(t\) discriminator between incident failures at \(t\) and survivors beyond \(t\). The tree-growing algorithm is then guided by a local increment in dynamic concordance: \[ \Delta ICON_\tau = \int_0^s \frac{|f(t,\tau^L)S(t,\tau^R)-f(t,\tau^R)S(t,\tau^L)|} {f(t,\tau)S(t,\tau)} \omega(t)\,dt. \] Pruning is based on \[ ICON_\alpha(T)=\widehat{ICON}(\widehat\lambda_T)-\alpha |T|, \] which is the ROC-guided analogue of CART cost-complexity pruning [1809.05627]. Under right censoring, node-specific hazards are estimated from counting-process quantities and kernel smoothing. For a node \(\tau\), \[ \widehat\lambda(t\mid\tau) = \frac{\int_0^\infty K_h(t-u)\,d\widehat F^*(u,\tau)} {\widehat S^*(t,\tau)}, \] with \[ d\widehat F^*(u,\tau) = \frac1n\sum_{i=1}^n I\{Z_i(Y_i)\in \tau\}\,dN_i(u), \qquad \widehat S^*(t,\tau) = \frac1n\sum_{i=1}^n I\{Z_i(t)\in \tau,\ Y_i\ge t\}. \] The resulting framework accommodates time-dependent covariates natively, without the “pseudo-subject” device used in prior work, and yields dynamic survival prediction through the subject’s covariate history \(Z_0^H(t)\) [1809.05627]. The same paper proposes an ensemble estimator that averages martingale estimating equations rather than predicted survival curves or cumulative hazard functions. With bootstrap-grown trees \(\mathbb T=\{T_b\}_{b=1}^B\), adaptive nearest-neighbor weights \[ w_i(t,z)=\frac1B\sum_{b=1}^B w_{bi}\,I\{Z_i(t)\in l_{T_b}\{z\}\} \] lead to \[ \widehat\lambda_{\mathbb T}(t\mid z) = \int_0^\infty K_h(t-u)\, \frac{\sum_{i=1}^n w_i(u,z)\,dN_i(u)} {\sum_{i=1}^n w_i(u,z)\,I(Y_i\ge u)}. \] This places time-dependent ROC analysis at the center of model construction, not merely post hoc evaluation [1809.05627]. ## 6. Meta-analysis, applications, limitations, and interpretation Time-dependent ROC methods also appear in **meta-analysis of prognosis studies**, where study-specific cutoffs induce heterogeneity in reported sensitivities, specificities, and hazard ratios [2305.19741]. In time-dependent summary ROC (SROC) analysis, the cumulative/dynamic quantities at study-specific cutoff \(v^{(i)}\) are \[ \se\left(v^{(i)}, t \right) = \dfrac{\left\{1- S_1^{(i)}\left(t\right)\right\}q_1^{(i)}} {\left\{1- S_0^{(i)}\left(t\right)\right\}q_0^{(i)} + \left\{1- S_1^{(i)}\left(t\right)\right\}q_1^{(i)}}, \] \[ \sp\left(v^{(i)}, t\right) = \dfrac{ S_0^{(i)}\left(t\right)\cdot q_0^{(i)}} { S_0^{(i)}\left(t\right)\cdot q_0^{(i)} + S_1^{(i)}\left(t\right)\cdot q_1^{(i)}}. \] A recent extension introduces a trivariate normal hierarchical model for \[ \bigl(\logit\{\text{time-dependent sensitivity}\},\; \logit\{\text{time-dependent specificity}\},\; \log(\text{HR})\bigr), \] then models selective publication through \[ a\left(t^{(i)}\right)=\Phi\left(\alpha + \beta\cdot t^{(i)}\right), \qquad t^{(i)}=\frac{\hat\mu_\lnHR^{(i)}}{\hat s_\lnHR^{(i)}}. \] Because the expected publication proportion \(p=P(\select)\) is not identifiable from observed studies alone, the method is explicitly a sensitivity analysis rather than a full correction [2305.19741]. Applications across the cited papers illustrate distinct use cases. Interval-censored ROC/AUC estimation was applied to **CALGB 30801**, where progression-free survival was interval censored and the markers **COX-2** and **pgem1** had estimated AUCs at \(t=12\) weeks of \(0.50\) and \(0.55\), respectively; neither was concluded to be very helpful for predicting event time [1806.01760]. Time-dependent pAUC methods were applied to **ACTG 175**, where CD4 cell counts showed discriminatory value in broader low-FPR regions for \(\alpha=0.2\) and \(0.3\), but for \(\alpha=0.1\) the simultaneous bands suggested CD4 was essentially useless as a classifier over the considered period in both therapy groups [1103.1963]. Covariate-specific ROC analysis was applied to **3488 acute coronary syndrome survivors**, showing that GRACE discrimination varies with left ventricular ejection fraction and that marginal AUCs from Uno et al. (2007) were generally larger than the LVEF-specific AUCs across much of the LVEF range [2506.13604]. ROC-guided survival trees were illustrated on an AIDS study, where the final pruned tree used current Karnofsky score and cumulative opportunistic infections and yielded a clinically interpretable dynamic discrimination rule [1809.05627]. LTRC ROC analysis was applied to the **St. Jude Lifetime Cohort Study**, where most AUC estimators gave similar results except that the regression-type semiparametric estimator produced smaller AUC estimates, consistent with its negative bias in simulation [2509.05693]. Time-dependent SROC sensitivity analysis was applied to **Ki67** in breast cancer, where publication-bias adjustment decreased \(\mathrm{SAUC}(t)\) but the substantive conclusion of statistically significant, though not especially high, prognostic discrimination remained fairly robust [2305.19741]. Several recurrent limitations appear across the literature. Interval-censored ROC estimation establishes consistency but not asymptotic normality or analytic standard errors, so inference relies on bootstrap BCa intervals [1806.01760]. Time-dependent pAUC estimation is sensitive when \(\alpha\) is very small and when censoring is heavy, requiring adequate sample sizes and careful bandwidth selection [1103.1963]. Conditional ROC estimation via penalised splines does not derive asymptotic theory in the provided text and uses bootstrap percentile intervals that were often conservative in simulation [2506.13604]. In LTRC settings, neither the regression-based nor the conditional IPW estimators are claimed to be doubly robust, and model misspecification in \(S_{T\mid Z}\), \(F_{L\mid Z}\), \(S_{D\mid Z}\), or \(S_{C\mid Z}\) can induce bias [2509.05693]. In meta-analysis, publication-bias adjustment depends on asymptotic approximations, estimated within-study covariance matrices, and user-specified values of \(p\), and numerical instability may occur [2305.19741]. Taken together, these developments show that time-dependent ROC analysis has evolved from a direct survival analogue of ordinary ROC curves into a broad methodological domain. It now includes cumulative/dynamic and incident/dynamic discrimination targets, full and partial AUC summaries, estimators for right-censored, interval-censored, and left-truncated data, covariate-specific and time-dependent-covariate formulations, algorithmic learning criteria for trees and ensembles, and SROC methodology for evidence synthesis [1809.05627] [1806.01760] [1103.1963] [2506.13604] [2305.19741] [2509.05693]. A plausible implication is that, in contemporary practice, the principal methodological question is no longer whether a time-dependent ROC curve can be defined, but which definition, estimand, and observation-model assumptions best match the survival prediction problem at hand. Source: https://www.emergentmind.com/topics/time-dependent-receiver-operating-characteristics-roc-analysis