CNN-LSTM with Copula Activation

• The paper introduces a CNN-LSTM framework integrating copula-based activations to jointly model and predict correlated, right-censored survival responses.
• It employs a 1D CNN for local feature extraction, LSTM layers for sequential dependencies, and copula functions to capture tail dependencies effectively.
• Empirical evaluations using simulations and breast cancer data demonstrate improved prediction accuracy, stability, and robustness over baseline models.

CNN-LSTM with copula-based activation functions denotes a deep learning formulation for multivariate multi-types of survival responses in which a convolutional feature extractor, an LSTM stack, and copula-driven nonlinearities are combined to model highly correlated and right-censored multivariate survival data. In the formulation reported in "Deep Learning-Based Survival Analysis with Copula-Based Activation Functions for Multivariate Response Prediction" (Kim et al., 19 Jul 2025), copula functions are integrated into activation layers to capture nonlinear dependencies and tail-dependence structure, while survival-analysis components handle censoring through a likelihood built from marginal densities, survival functions, and a copula density. The reported empirical setting includes simulation studies and real breast cancer data, with performance evaluated by Shewhart control charts through the average run length (ARL) (Kim et al., 19 Jul 2025).

1. Conceptual setting

The model is designed for multivariate survival prediction when responses are both correlated and right-censored. The stated objective is to enhance prediction accuracy by explicitly addressing right-censored data and capturing complex patterns in multivariate survival outcomes (Kim et al., 19 Jul 2025).

Its core architectural idea is to combine three technical elements. First, a 1D CNN acts as a convolutional feature extractor over a multivariate time series input. Second, stacked LSTM layers process the extracted temporal representation. Third, copula-based activation functions, specifically Clayton, Gumbel, and combinations of these, are inserted into the network so that dependency structure is not left implicit but is instead represented through differentiable transforms tied to copula formulations.

The survival-analysis component is equally central. For each response margin, the formulation includes a predicted conditional density $f_j(t \mid x)$ and survival function $S_j(t \mid x)$, while the copula activation couples the margins through transformed variables $u_j = F_j(t)$. This places dependence modeling directly alongside censoring-aware likelihood construction rather than treating correlation as a downstream residual phenomenon.

A plausible implication is that the approach is intended not merely as a generic sequence model with custom nonlinearities, but as a joint model in which architecture, activation design, and likelihood all target multivariate survival structure simultaneously.

2. Network architecture

The input is a multivariate time series $X \in \mathbb{R}^{T \times d}$, where $T$ denotes time steps and $d$ denotes features (Kim et al., 19 Jul 2025). The network begins with a 1D CNN:

• Conv layer 1 uses a kernel of length $k_1$ and $F_1$ filters, producing $Z^{(1)} \in \mathbb{R}^{(T-k_1+1) \times F_1}$.
• Max-pool 1 with pool size $p$ yields $S_j(t \mid x)$0.
• Conv layer 2 uses kernel length $S_j(t \mid x)$1 and $S_j(t \mid x)$2 filters, producing $S_j(t \mid x)$3.
• Max-pool 2 with the same pool size yields $S_j(t \mid x)$4.

The pooled representation is then passed into an LSTM stack. The description allows either flattening the time dimension or feeding the sequence $S_j(t \mid x)$5 over $S_j(t \mid x)$6. LSTM layer 1 with $S_j(t \mid x)$7 units outputs $S_j(t \mid x)$8 and cell state $S_j(t \mid x)$9, and LSTM layer 2 with $u_j = F_j(t)$0 units outputs $u_j = F_j(t)$1 and $u_j = F_j(t)$2. At the final time step, the model takes $u_j = F_j(t)$3.

The output projection is a dense layer,

$u_j = F_j(t)$4

with $u_j = F_j(t)$5 and $u_j = F_j(t)$6, where $u_j = F_j(t)$7 is the number of survival responses. A copula-based activation $u_j = F_j(t)$8 then produces the joint predictions $u_j = F_j(t)$9 (Kim et al., 19 Jul 2025).

Within this architecture, the CNN is explicitly associated with local temporal pattern extraction, whereas the LSTM stack models sequential dependence over the transformed representation. The reported comparison with a standard LSTM indicates that the CNN-LSTM is better at capturing local temporal patterns, especially under high censoring, which situates the convolutional front-end as more than a preprocessing convenience (Kim et al., 19 Jul 2025).

3. Copula-based activation functions

All copulas are defined on $X \in \mathbb{R}^{T \times d}$0 after mapping a pre-activation $X \in \mathbb{R}^{T \times d}$1 to $X \in \mathbb{R}^{T \times d}$2 via the Gaussian CDF $X \in \mathbb{R}^{T \times d}$3 (Kim et al., 19 Jul 2025). This mapping is the basis on which standard real-valued pre-activations are turned into copula-compatible arguments.

For the Clayton copula,

$X \in \mathbb{R}^{T \times d}$4

with density

$X \in \mathbb{R}^{T \times d}$5

The univariate Clayton-activation used as the paper’s surrogate is

$X \in \mathbb{R}^{T \times d}$6

For the Gumbel copula,

$X \in \mathbb{R}^{T \times d}$7

with density

$X \in \mathbb{R}^{T \times d}$8

The univariate Gumbel-activation is

$X \in \mathbb{R}^{T \times d}$9

The reported hybrid activations are

$T$0

and

$T$1

The paper’s conclusion attributes distinct dependence emphases to the copula families: Clayton copulas capture early joint failures, Gumbel capture late failures, and hybrids combine both (Kim et al., 19 Jul 2025). This suggests that the activation choice is not only a numerical design decision but also a structural assumption about the dependence regime most relevant to the target survival process.

4. LSTM gate dynamics and censoring-aware objective

The LSTM formulation replaces the usual $T$2 or $T$3 with a chosen copula-based activation $T$4 (Kim et al., 19 Jul 2025). For time step $T$5,

$T$6

$T$7

$T$8

$T$9

The gates and candidate cell state become

$d$0

$d$1

$d$2

$d$3

The cell and hidden updates remain

$d$4

and

$d$5

For right-censored multivariate survival data, the per-subject log-likelihood is defined using observed times $d$6, event indicators $d$7, marginal density $d$8, survival function $d$9, and copula density $k_1$0 on $k_1$1 with $k_1$2:

$k_1$3

where $k_1$4 indicates fully observed joint density. The total negative log-likelihood is

$k_1$5

Backpropagation proceeds through the copula activations by the chain rule, exemplified by

$k_1$6

Censoring is handled by plugging $k_1$7 into the partial likelihood and zeroing density terms when $k_1$8, that is, using $k_1$9 instead of $F_1$0 (Kim et al., 19 Jul 2025). In this construction, the dependence model and the censoring mechanism enter the same objective rather than being treated as separate modeling stages.

5. Forward pass, optimization, and parameterization

The forward and backward passes are specified procedurally (Kim et al., 19 Jul 2025). The forward pass is:

$p$4

The backward pass is:

$p$5

The training procedure specifies Adam as the optimizer, with learning rate $F_1$1, tuned, and betas $F_1$2; batch size $F_1$3-$F_1$4; and $F_1$5-$F_1$6 epochs with early stopping on validation loss (Kim et al., 19 Jul 2025). Dropout is $F_1$7 in LSTM layers, and batch-normalization is applied after the CNN. Copula parameters $F_1$8 are unconstrained through a softplus reparameterization,

$F_1$9

The paper also states that gradients flow through the copula activation layers and into $Z^{(1)} \in \mathbb{R}^{(T-k_1+1) \times F_1}$0.

These details indicate that the copula parameters are optimized jointly with the rest of the network rather than estimated in a separate dependence-fitting stage. A plausible implication is that dependence strength and temporal representation are intended to co-adapt during training.

6. Evaluation protocol and empirical findings

Evaluation is based on the average run length of Shewhart control charts applied to residuals $Z^{(1)} \in \mathbb{R}^{(T-k_1+1) \times F_1}$1 (Kim et al., 19 Jul 2025). The control limits are

$Z^{(1)} \in \mathbb{R}^{(T-k_1+1) \times F_1}$2

and

$Z^{(1)} \in \mathbb{R}^{(T-k_1+1) \times F_1}$3

The reported empirical results are summarized below.

Setting	Reported finding	Reported quantitative detail
Simulation study (three responses)	CNN-LSTM Clayton-ReLU gave lowest residual variance on Response$Z^{(1)} \in \mathbb{R}^{(T-k_1+1) \times F_1}$4 and Response$Z^{(1)} \in \mathbb{R}^{(T-k_1+1) \times F_1}$5	mean ARL $Z^{(1)} \in \mathbb{R}^{(T-k_1+1) \times F_1}$6-$Z^{(1)} \in \mathbb{R}^{(T-k_1+1) \times F_1}$7 vs. $Z^{(1)} \in \mathbb{R}^{(T-k_1+1) \times F_1}$8-$Z^{(1)} \in \mathbb{R}^{(T-k_1+1) \times F_1}$9 for baselines
Simulation study (three responses)	Copula activations consistently outperformed pure ReLU/sigmoid in joint-dependency tasks	no additional metric beyond the statement
METABRIC real-data	CNN-LSTM Clayton-ReLU showed the best stability	mean residual $p$0 months for time, $p$1 for event; ARL up to $p$2 for event indicator
METABRIC real-data	Sigmoid/Gumbel had large residuals and low ARL	residuals $p$3 months
Comparative architectural result	Standard LSTM fell behind CNN-LSTM in capturing local temporal patterns	especially under high censoring

The residual-control-chart framing emphasizes stability as well as pointwise predictive error. In the reported interpretation, low ARL for some baselines indicates frequent false alarms, whereas higher ARL for the copula-based CNN-LSTM is associated with improved robustness (Kim et al., 19 Jul 2025). The conclusion further states that the architecture yields lower prediction bias, reduced residual variability, and higher ARL in Shewhart control charts.

7. Interpretation, scope, and common points of confusion

A common point of confusion is to treat the copula-based components as a purely output-level device. In the reported formulation, copula-based activation functions are embedded into both the gates and the output layer of the CNN-LSTM (Kim et al., 19 Jul 2025). The method therefore modifies recurrent dynamics as well as final multivariate coupling.

A second point of confusion is to reduce the method to generic multivariate regression with a custom nonlinearity. The paper explicitly places the model within survival analysis: each margin has a density and survival function, the likelihood contains censoring indicators, and the copula density contributes when the joint density is fully observed. This means right-censoring is not an external preprocessing issue but part of the objective itself.

A third point concerns the role of the copula family. The conclusion associates Clayton with early joint failures and Gumbel with late failures, while hybrids combine both (Kim et al., 19 Jul 2025). This suggests that copula selection encodes assumptions about tail dependence rather than merely changing activation shape. It would therefore be inaccurate to interpret Clayton, Gumbel, and hybrid variants as interchangeable nonlinearities with no survival-specific meaning.

Within the scope reported, the method is supported by simulation studies and analysis of real breast cancer data, specifically METABRIC. The strongest reported configuration is CNN-LSTM Clayton-ReLU. Any broader generalization beyond these settings would be an inference rather than a direct empirical claim.