SurvNet: FDR-Controlled DNN Variable Selection

• SurvNet is an automatic backward-elimination framework for DNN variable selection that provides explicit and provable control of the false discovery rate (FDR).
• It employs a gradient-based importance measure with surrogate-variable augmentation to iteratively prune features while ensuring statistical guarantees.
• Empirical evaluations on synthetic, imaging, and genomics datasets demonstrate its effectiveness in reducing prediction error and enhancing model interpretability.

SurvNet is an automatic backward-elimination framework for variable selection in deep neural networks (DNNs) that achieves explicit and provable control over the false discovery rate (FDR) among selected variables. It is the first method in the DNN literature to provide a direct estimator for FDR and a procedure ensuring that FDR does not exceed a user-specified threshold. SurvNet is based on a global, architecture-agnostic measure of variable importance derived from loss sensitivity, surrogate null-variable augmentation, and a mathematically justified elimination schedule. Its development addresses the interpretability and feature selection challenges inherent in modern deep learning, delivering a statistically principled, practical solution applicable across supervised learning settings including high-dimensional genomics, imaging, and general multi-modal learning (Song et al., 2019).

1. Variable Importance Measure in SurvNet

SurvNet quantifies input-variable importance using the gradient of the network loss with respect to each input dimension. For a network with scalar loss $L(\mathbf{y}, f(\mathbf{x}))$, input $\mathbf{x} \in \mathbb{R}^p$, and output $f(\mathbf{x})$, the importance score for variable $j$ is

$$S_j^{(2)} = \frac{1}{n} \sum_{i=1}^n \left( \frac{\partial L}{\partial x_j}\bigl(\mathbf{y}^{(i)}, f(\mathbf{x}^{(i)}) \bigr) \right)^2$$

A large $S_j$ indicates that small perturbations in $x_j$ produce substantial changes in loss, signifying high variable importance. This gradient-based score is model-agnostic: it remains valid regardless of architectural depth, width, or activation function, and is efficiently computed using automatic differentiation in frameworks such as TensorFlow or PyTorch (Song et al., 2019). Modern DNNs naturally facilitate this computation as part of backpropagation.

2. SurvNet Backward Elimination and FDR Estimation

SurvNet performs variable selection by iterative backward elimination coupled with surrogate-variable augmentation. The process is as follows:

• Surrogate Augmentation: Begin with $p$ real input variables and $q$ surrogate (null) variables constructed by permuting feature values. This enables estimation of the null distribution of importance scores required for FDR control.
• Training: Train (or retrain) the DNN using all $r \leq p+q$ inputs. Warm starts are used to initialize weights from previous iterations, improving convergence efficiency.
• Scoring: Compute $\mathbf{x} \in \mathbb{R}^p$0 for each variable.
• FDR Estimation: Let $\mathbf{x} \in \mathbb{R}^p$1 be the number of remaining surrogate variables. The estimated FDR among selected real features is

$\mathbf{x} \in \mathbb{R}^p$2

• Stopping and Pruning: If $\mathbf{x} \in \mathbb{R}^p$3 (the user-specified FDR threshold), selection terminates and surrogates are removed. Otherwise, eliminate the $\mathbf{x} \in \mathbb{R}^p$4 variables with the smallest $\mathbf{x} \in \mathbb{R}^p$5.

The number of variables $\mathbf{x} \in \mathbb{R}^p$6 to eliminate is computed to maximize selection efficiency while ensuring that FDR control is not overshot. Specifically, if all $\mathbf{x} \in \mathbb{R}^p$7 eliminated variables are surrogates, the minimal possible FDR in the next step is

$\mathbf{x} \in \mathbb{R}^p$8

Solving for $\mathbf{x} \in \mathbb{R}^p$9 to reach $f(\mathbf{x})$0, the update is

$f(\mathbf{x})$1

where the elimination rate $f(\mathbf{x})$2 often defaults to $f(\mathbf{x})$3 for aggressive pruning (Song et al., 2019).

3. Guarantees and Theoretical Foundations

The FDR estimator $f(\mathbf{x})$4 is strictly conservative if surrogate variables are generated correctly: their marginal distributions approximate those of original features, and they contain no predictive signal. The procedure's theoretical guarantee is that as soon as $f(\mathbf{x})$5, the true FDR $f(\mathbf{x})$6 among the selected variables is also controlled below $f(\mathbf{x})$7,

$f(\mathbf{x})$8

where $f(\mathbf{x})$9 is the unknown count of truly null features (Song et al., 2019). This guarantee holds without exchangeability or mutual independence among original features, making SurvNet robust to correlation structures typical in real data such as gene expression or imaging applications.

4. Empirical Evaluation and Applications

SurvNet has been validated on synthetic datasets and real-world modalities:

• Synthetic Classification: Across four simulation paradigms (mean-shifted, correlated, variance-inflation, and non-linear regression), SurvNet maintained the true FDR below the target (e.g., $j$0 yielded actual FDR $j$1), retained nearly all true signals, and reduced test error after variable selection.
• MNIST Imaging: Applied to pixel-wise selection ($j$2), SurvNet with $j$3 consistently highlighted anatomically relevant pixels, producing interpretable saliency maps.
• Single-cell Gene Expression: For a 1,046-gene RNA-Seq task, SurvNet at $j$4 selected 145 genes, lowering test classification error and revealing both mean-difference and variance-driven predictors, as validated by external gene ontology analyses.

Iteration requires only a handful of steps even for thousands of predictors, and the use of warm starts significantly accelerates training at each stage (Song et al., 2019).

5. Limitations and Practical Considerations

SurvNet’s randomness stems from both stochastic DNN training (weight initialization, stochastic gradient descent) and the generation of surrogate variables, though empirical tests show the effect of the former dominates when $j$5. The method is not suitable when all features must be kept (as in certain convolutional or sequential architectures) or when predictive performance is driven by diffuse, small effects not detectable by marginal loss gradient (Song et al., 2019). Ensuring surrogates match the feature distribution is critical for valid FDR estimation. Ensemble runs or increasing the number of surrogates can help stabilize selection.

Standard optimization practices such as input normalization, batch normalization, and warm restarts improve both DNN and selection performance. SurvNet introduces minimal user-tunable parameters (principally $j$6 and $j$7), making it accessible for high-dimensional DNN variable selection requiring statistical guarantees.

6. Extensions and Influence: SurvNet in Multimodal Learning

SurvNet forms the foundation for more complex multimodal prognostic modeling approaches. In particular, MM-SurvNet extends the SurvNet paradigm by integrating histopathological images, gene expression, and clinical variables for survival risk stratification in breast cancer. This tri-modal network incorporates self-attention–based patch aggregation (originally introduced in SurvNet), explicit handling of genetic modalities, and dual cross-attention fusion, resulting in significant performance improvements; for example, MM-SurvNet improves the mean concordance index from approximately $j$8 (SurvNet with MaxViT+clinical) to $j$9 through tri-modal integration and cross-attention fusion (Mondol et al., 2024).

7. Impact and Significance

SurvNet addresses the black-box nature of DNNs by enabling statistically principled feature selection with FDR control, a capability previously absent in deep learning. Its application to image, genetic, and simulated data demonstrates robust and interpretable selection, facilitating biological discovery, model compression, and scientific trust. The approach has also underpinned advances in multimodal learning, where its architectural concepts are foundational to high-performing pipelines that jointly process images, sequences, and structured clinical or omics data (Song et al., 2019, Mondol et al., 2024). A plausible implication is that SurvNet and its derivatives could become standard tools in high-dimensional scientific deep learning applications demanding interpretability and type I error control.