Deep Radiomics Features (DRF)

• Deep Radiomics Features (DRF) are quantitative imaging descriptors derived from deep neural representations rather than traditional hand-crafted formulas.
• They are computed through various formulations—including CNN activations, gradient-curated features, and fusion with classical radiomics—across CT, MRI, PET, and other modalities.
• DRF enhance diagnostic accuracy, prognostic modeling, and image retrieval by integrating deep learning with conventional radiomic approaches in hybrid systems.

Deep Radiomics Features (DRF) are quantitative image descriptors derived from deep neural representations rather than solely from pre-defined radiomic formulas. Across the literature, the term covers several related constructions: penultimate representations of CNN-based radiomic sequencers trained on pathology- or biopsy-proven labels, classical radiomic descriptors computed on CNN activation maps, and hand-crafted radiomics whose contribution is evaluated or curated through deep models (Shafiee et al., 2017, Shafiee et al., 2017, Chaddad et al., 2019, Shakir et al., 3 Jun 2026). DRF emerged within the broader shift from hand-crafted radiomics to discovery radiomics and hybrid radiomics–deep learning systems, with applications spanning CT, MRI, PET, SPECT, and mammography for diagnosis, subtype recognition, treatment-response prediction, survival modeling, and image retrieval (Afshar et al., 2018, Salmanpour et al., 21 Jul 2025, Salmanpour et al., 20 Nov 2025).

1. Conceptual foundations and historical emergence

Conventional radiomics is organized around image acquisition and reconstruction, segmentation, hand-crafted feature extraction, and feature reduction or modeling. In that framework, the feature space is built from explicitly designed first-order, texture, higher-order, and shape descriptors such as GLCM, GLRLM, NGTDM, GLZLM, wavelet, LoG, and Minkowski-function-based quantities (Afshar et al., 2018). The central criticism developed in later work is that such descriptors are constrained by human prior design and may not fully characterize disease-specific phenotype.

Discovery radiomics reframed this pipeline by learning a “radiomic sequencer” directly from labeled imaging data. In lung CT, the sequencer is defined as a learned mapping from a suspicious region to a radiomic sequence tailored for characterizing tissue phenotype that differentiates cancerous from healthy tissue; in skin imaging, the same logic is extended to a deep multi-column radiomic sequencer trained on biopsy-proven images (Shafiee et al., 2017, Shafiee et al., 2017). In both cases, DRF are the elements of the learned radiomic sequence rather than manually specified histogram, texture, or shape formulas.

The broader review literature places DRF under the umbrella of deep learning-based radiomics or discovery radiomics. There, DRF may be drawn from CNN activations, autoencoder latent codes, DBN hidden layers, capsule vectors, or recurrent hidden states, and they may or may not require an explicitly segmented ROI (Afshar et al., 2018). Taken together, these formulations suggest that DRF is best understood as an umbrella term for deep, task-adapted quantitative image representations rather than a single fixed feature definition.

2. Main computational formulations of DRF

One major formulation treats DRF as the direct output of a deep radiomic sequencer. In the lung cancer detection framework, the network implements a mapping

$\mathbf{z} = f_{\theta}(x) \in \mathbb{R}^d,$

where $\mathbf{z}$ is the radiomic sequence and the classifier computes logits

$$\mathbf{y} = g_{\phi}(\mathbf{z}) \in \mathbb{R}^2.$$

The original deep radiomic sequence is the 1024-dimensional vector produced by the last convolutional layer before the fully connected classifier; in the skin cancer sequencer, two parallel columns each output 8192 features, yielding a 16384-dimensional radiomic sequence after concatenation (Shafiee et al., 2017, Shafiee et al., 2017). In these systems, DRF are latent, supervised, and explicitly optimized for benign–malignant discrimination.

A second formulation computes radiomic descriptors on CNN activation maps rather than on raw image intensities. In recurrent glioblastoma, 41 radiomic quantifiers are computed on 20 activation maps from layers 1 and 2 of a pretrained 3D CNN and then averaged to form a 41-dimensional DRF vector per patient (Chaddad et al., 2019). In glioma survival modeling with immune cell markers, 41 histogram and texture descriptors are computed on activation maps, averaged across 20 maps, and combined with four shape features to produce 45 DRFs per modality and 180 DRFs across T1-WI, T1-CE, T2-WI, and FLAIR (Chaddad et al., 2022). A related variant models the distribution of activation values inside the tumor ROI with a Gaussian mixture model, using the component means, variances, and weights as DRFs; with $k=2$ components and 21 feature maps, that yields 126 features (Chaddad et al., 2022).

A third formulation keeps the starting feature space hand-crafted but makes it “deep” through network-dependent sensitivity analysis. In Gradient-Loss Recursive Feature Elimination, 106 PyRadiomics features are supplied to an MLP and ranked by the mean absolute gradient of the loss with respect to each input,

$$I(x_i) = \frac{1}{N}\sum_{j=1}^{N}\left|\frac{\partial L_j}{\partial x_{ij}}\right|,$$

after which the least influential features are recursively removed until 15 remain (Shakir et al., 3 Jun 2026). This does not learn features from pixels, but it does define radiomic relevance through deep nonlinear interactions.

A fourth formulation is hybrid fusion. In pancreatic ductal adenocarcinoma, 1428 PyRadiomics features are combined with 2048 ResNet-50 features and 64 lung-CT-pretrained ResNet features; in lung adenocarcinoma subtype recognition, 1106 radiomics features are screened to $7k$ selected features and fused with 2048-dimensional ResNet-50 slice features through multi-head attention; in breast cancer risk prediction, a 512-dimensional deep feature vector is fused with 122 radiomics features per mammographic view (Zhang et al., 2019, Zhou et al., 2023, Yeoh et al., 2023). These studies treat DRF not as a replacement for hand-crafted radiomics but as one component of a multi-branch phenotype representation.

3. Architectures, optimization strategies, and fusion mechanisms

The architectural design of DRF systems varies widely, but several recurrent patterns are evident. One is direct CNN-based radiomic sequencing. The lung “evolved deep radiomic sequencer” begins from a LeNet-5-inspired ancestor with three convolutional layers and two fully connected layers, then applies evolutionary deep intelligence to synthesize offspring networks with fewer synapses and filters while retaining discriminative capacity (Shafiee et al., 2017). The heredity model is written as

$$P(\mathcal{H}_g \mid \mathcal{H}_{g-1}) = P(\mathbb{S}_g \mid W_{g-1}),$$

and the cluster-driven synaptic probability factorizes over filters,

$$P(\mathbb{S}_g \mid W_{g-1}) = \prod_{c \in C}\Big[P(S_g^c \mid W_{g-1})\cdot \prod_{i \in c} P(s_g^i \mid w_{g-1}^i)\Big].$$

An environmental factor constrains each generation to 80% of the ancestor’s synapses, enforcing 20% synaptic pruning per generation on average (Shafiee et al., 2017).

Another pattern is multi-column or multi-view design. The skin cancer sequencer uses two parallel convolutional columns, each with five convolutional layers and an 8192-neuron fully connected layer; their outputs are concatenated into the final radiomic sequence (Shafiee et al., 2017). In lung adenocarcinoma subtype recognition, deep features from $n$ slices and radiomics selected by Sure Independence Screening are projected into a common 128-dimensional latent space, then fused by learnable attention weights so that a single head computes

$$\bar{x} = a^r \hat{x}^{r} + \sum_{i=1}^{n} a_i^d \hat{x}_i^d,$$

with multiple heads averaged at logit level before softmax classification (Zhou et al., 2023). In breast cancer risk modeling, 3D ResNet-18 over sequential mammograms is augmented by the SHIFT attention block, a view-gating mechanism, and bilateral asymmetry-based finetuning (Yeoh et al., 2023).

Transfer learning is also central. Several MRI studies use pretrained 3D CNNs originally optimized for Alzheimer’s disease classification, then repurpose the early activation maps as substrates for radiomic summarization (Chaddad et al., 2019, Chaddad et al., 2022, Chaddad et al., 2022). In rectal cancer, VGG19 pretrained on ImageNet provides a 1472-dimensional deep feature vector from ADC maps, later reduced to the 105 highest-variance features for LASSO-logistic regression (Fu et al., 2019). These examples underscore that DRF can arise from fixed encoders, not only from end-to-end retrained networks.

4. Diagnostic, prognostic, and retrieval applications

In diagnostic settings, DRF have been reported to improve discrimination across several modalities and diseases. For pathologically proven lung cancer detection on LIDC-IDRI, the best evolved deep radiomic sequencer achieved sensitivity 93.42%, specificity 82.39%, and diagnostic accuracy 88.78%, exceeding DARS, CNN-MIL, SNRS, DRS, and a same-architecture “Last-Generation” control trained from scratch (Shafiee et al., 2017). For skin cancer detection on 9,152 biopsy-proven images, the deep multi-column sequencer achieved sensitivity 91% and specificity 75% (Shafiee et al., 2017). In locally advanced rectal cancer, a LASSO-logistic model built from VGG19-derived DRF on pretreatment ADC maps reached mean AUC 0.73, versus 0.64 for handcrafted radiomics (Fu et al., 2019). For lung adenocarcinoma, the MHA-FF fusion model reached accuracy 0.8683 and AUC 0.9056 for Pre-IA versus IA, and accuracy 0.7397 with kappa 0.6023 for IA subtype classification, outperforming radiomics-only, deep-only, and simple concatenation baselines (Zhou et al., 2023).

In prognostic and survival modeling, DRF have been used both alone and in fused systems. In recurrent glioblastoma, DRFs extracted from MRI feature maps yielded AUC 89.15% for short- versus long-term survival, compared with 78.07% for standard radiomic features (Chaddad et al., 2019). In glioma, DRFs alone achieved AUC 70.77% for short- versus long-survival prediction, and the combination of DRFs, clinical features, and immune cell markers reached AUC 72.01% with log-rank $\mathbf{z}$0 (Chaddad et al., 2022). GMM-based DRFs from pretrained 3D CNN activations predicted Macrophage M1, Neutrophils, and T Cells Follicular Helper immune marker status with AUCs of 78.67%, 83.93%, and 75.67%, respectively, and achieved the most significant survival separation when combined with immune markers and clinical variables (Chaddad et al., 2022). In resectable pancreatic ductal adenocarcinoma, a risk-score-based late fusion of handcrafted radiomics, ImageNet-pretrained ResNet-50 features, and lung-CT-pretrained ResNet features reached AUC 0.86, compared with 0.74 for the best deep-only model and 0.57 for handcrafted radiomics alone (Zhang et al., 2019). In glioblastoma post-resection survival prediction, the best 2D and 3D fused models combining radiomics, deep features, and patient-specific clinical features both reached accuracy 0.745, with AUC 0.69 and 0.71, respectively (Hu et al., 2022). An earlier chest CT study similarly compared end-to-end learned features with hand-crafted radiomics for 5-year mortality and reported mean accuracy 68.5% for the deep model versus 56% to 66% for radiomics depending on the pipeline (Carneiro et al., 2016).

DRF are also being extended beyond classification and prognosis into retrieval and large-scale evidence synthesis. A PET/SPECT review of 226 studies reported that DRF models achieved the highest mean accuracy, 0.862, जबकि fusion models yielded the highest mean AUC, 0.861; ANOVA showed significant performance differences for both accuracy and AUC (Salmanpour et al., 21 Jul 2025). In RadiomicsRetrieval, tumor-specific SAM-Med3D image embeddings are aligned with 72 PyRadiomics features and anatomical positional embeddings through contrastive learning, enabling image-based, radiomics-based, APE-only, and partial-feature queries in a shared latent space (Na et al., 11 Jul 2025). This suggests that DRF are increasingly being treated as searchable and controllable representations rather than only as classifier inputs.

5. Interpretation, efficiency, and reproducibility

Interpretability remains heterogeneous across DRF formulations. Some systems explicitly acknowledge limited interpretability: the skin cancer multi-column sequencer does not provide feature maps, t-SNE plots, saliency maps, or per-feature explanations, and the recurrent glioblastoma study likewise interprets DRF statistically rather than visually (Shafiee et al., 2017, Chaddad et al., 2019). Yet other formulations recover partial interpretability by construction. Radiomic descriptors computed on activation maps preserve familiar categories such as histogram, GLCM, NGTDM, and GLSZM, allowing associations with immune markers and survival to be expressed in terms such as Gray-Level Non-Uniformity, Large Zone Size Emphasis, or Information Correlation 2 (Chaddad et al., 2022). The GMM-based glioma work additionally visualizes activation histograms and fitted Gaussian components, and the RADIFUSION study shows SHIFT attention focusing on dense breast regions and bilateral asymmetry scores differing between cases and controls (Chaddad et al., 2022, Yeoh et al., 2023). These examples suggest that interpretability is stronger when deep representations are summarized by structured radiomic operators or linked to explicit anatomical priors.

Operational efficiency is a separate but recurrent theme. The evolved lung sequencer reduces the average number of filters from 194.0 in generation 1 to 104.5 in generation 11 and the radiomic sequence length from 3104.0 to 1672.0, while runtime decreases monotonically across generations and the authors explicitly argue that diagnosis can be run locally at the radiologist’s computer (Shafiee et al., 2017). RadSynth approaches efficiency from a different angle: instead of learning a classifier, it learns to synthesize GLCM entropy maps from post-contrast DCE-MRI, reducing generation time from approximately forty minutes for traditional entropy imaging on a $\mathbf{z}$1 image with a $\mathbf{z}$2 kernel to 11.8 seconds per patient, with average percentage difference $\mathbf{z}$3 and correlation 0.97 relative to Haralick entropy maps (Parekh et al., 2018). These studies indicate that DRF research has encompassed both representational power and deployability.

Reproducibility and standardization remain uneven. PySERA was developed as an open-source Python-native framework for automated handcrafted and deep radiomics, computing 557 features including 487 IBSI-compliant features and deep embeddings from ResNet50, DenseNet121, and VGG16; in IBSI benchmarks it achieved more than 94 percent reproducibility and reported deterministic deep embeddings under fixed preprocessing and model settings (Salmanpour et al., 20 Nov 2025). By contrast, the PET/SPECT review found that only 48% of studies adhered to IBSI standards, with common shortcomings in handling class imbalance, missing data, and population diversity, and it emphasized the lack of a DRF-specific standard analogous to IBSI (Salmanpour et al., 21 Jul 2025). A plausible implication is that standardization of preprocessing, encoder choice, layer selection, and validation remains as important for DRF as feature-definition standardization has been for hand-crafted radiomics.

6. Limitations, controversies, and future directions

A persistent limitation is that DRF do not denote a single methodological object. In some papers they are the last-layer activations of a radiomic sequencer; in others they are radiomic statistics computed on activation maps; in others they are gradient-curated hand-crafted descriptors; and in still others they are fusion-space embeddings aligned with anatomy and radiomics attributes (Shafiee et al., 2017, Chaddad et al., 2019, Shakir et al., 3 Jun 2026, Na et al., 11 Jul 2025). This heterogeneity complicates direct comparison, benchmarking, and standard reporting.

Data scale and external validation are equally recurrent concerns. The lung evolutionary work is limited to LIDC-IDRI, the skin cancer study trains on 946 balanced training images and tests on a strongly imbalanced split, the recurrent glioblastoma and glioma immune-survival studies have 100 and 151 patients, and the GBM post-resection model is derived from BraTS 2020 without external validation (Shafiee et al., 2017, Shafiee et al., 2017, Chaddad et al., 2019, Chaddad et al., 2022, Hu et al., 2022). The PET/SPECT review similarly identifies inadequate handling of class imbalance, missing data, and low population diversity as common weaknesses across recent literature (Salmanpour et al., 21 Jul 2025). These results do not negate the reported performance gains, but they do constrain claims about transportability across institutions, scanners, and acquisition protocols.

Interpretability is the other major controversy. The 2018 radiomics review explicitly contrasts the direct physical meaning of hand-crafted radiomics with the latent, often black-box nature of deep representations and identifies explainability as a central open problem (Afshar et al., 2018). Later work responds in several ways: by keeping classical radiomic operators on activation maps, by ranking hand-crafted features through deep gradients, by adding attention visualizations, or by treating anatomical position as an explicit embedding channel (Shakir et al., 3 Jun 2026, Yeoh et al., 2023, Na et al., 11 Jul 2025). This suggests that future DRF systems are likely to remain hybrid rather than purely end-to-end if interpretability is treated as a design constraint.

The main research directions already named in the literature are convergent. They include larger and multi-center validation cohorts, integration of CNN-derived image features with handcrafted radiomics and clinical or genomic data, hybrid feature selection with SHAP or LIME, extension to additional MRI modalities such as ADC and DCE, automatic segmentation and end-to-end joint pipelines, 3D and multimodal attention, foundation-model-based deep radiomics backbones, and GPU or distributed acceleration for scalable extraction (Shakir et al., 3 Jun 2026, Chaddad et al., 2022, Zhou et al., 2023, Salmanpour et al., 20 Nov 2025). Taken together, these directions suggest that the field is moving from isolated demonstrations of learned imaging biomarkers toward standardized, multimodal, and queryable radiomic representation learning.