ReFRM3D: Radiomics-Enhanced 3D Glioma Segmentation

• The paper introduces ReFRM3D, which combines deep volumetric MRI features and engineered radiomics to enhance precise glioma segmentation and classification.
• It employs a fused residual 3D U-Net backbone with hybrid upsampling and extended skip connections, achieving Dice Similarity Coefficients above 93% on BraTS datasets.
• Ablation studies validate that the multi-scale feature fusion and radiomics integration are key to outperforming existing models like WLFS and CH-UNet.

The Radiomics-Enhanced Fused Residual Multiparametric 3D Network (ReFRM3D) is a neural architecture designed to advance the segmentation and classification of gliomas in multiparametric MRI. Introduced for the characterization of brain tumors, ReFRM3D integrates a fused residual 3D U-Net backbone with multi-scale feature fusion, hybrid upsampling, extended residual skip connections, and post-segmentation radiomics-driven tumor marker classification. This model synthesizes deep volumetric MRI features with engineered radiomic descriptors to optimize both delineation and subregion classification of gliomas, achieving state-of-the-art results across the BraTS2019, BraTS2020, and BraTS2021 benchmarks (Rahman et al., 27 Dec 2025).

1. Architectural Overview

ReFRM3D’s architecture is rooted in the canonical 3D U-Net encoder–decoder paradigm, with a substantive series of augmentations designed to capture multiresolutional context while preserving structural and textural detail. Given a three-modal MRI volume $X \in \mathbb{R}^{D \times H \times W}$ (FLAIR, T1ce, T2), the encoder applies sequential 3D convolutions, ReLU activations, and max-pooling:

%%%%1%%%%

with $*$ denoting 3D convolution, $\phi(\cdot)$ as ReLU, and $\mathcal{P}$ representing max-pooling. To capture broader nonlocal context without excessive downsampling, a “context pathway” employing 3D dilated convolutions is appended at the bottleneck:

$$F_{context} = \phi(W_{dil} \diamond F_L + b_{dil}),$$

where $\diamond$ denotes convolution with dilation rate $r>1$.

In the decoder, the Hybrid Upsampling and Residual Integration (HURI) module performs spatial upsampling of decoder activations via transposed convolution, combines these with encoder features using skip connections—optionally processed through 1x1x1 convolution—and applies refining convolutions. An extended residual skip (rSkip) design further reinforces gradient flow by adding linearly projected encoder features directly into the decoder stage:

$F_{rskip} = F_d + \mathcal{R}(F_e).$

2. Multi-Scale Feature Fusion

ReFRM3D introduces multi-scale feature fusion (FMFF) to synthesize hierarchical representations across different spatial resolutions. Feature maps at each scale, $F_s$, are “aligned” by 3×3×3 convolution:

$\tilde{F}_s = W_s * F_s + b_s$

and then summed:

$F_{fused} = \sum_{s=1}^{S} \tilde{F}_s$

This aggregation preserves detailed local cues while integrating global context, facilitating more precise delineation of complex tumor subregions. The decoder’s final outputs from all levels undergo learned weighted late aggregation:

$$F_{aggr} = \sum_{l=1}^{L_{dec}} \gamma_l \cdot F_{dec}^l$$

where $\gamma_l$ are scalar, end-to-end–trained weights. Output projection to segmentation masks is achieved via a final 1x1x1 convolution.

3. Radiomics Feature Extraction

Upon completion of volumetric segmentation, ReFRM3D computes a set of classical 3D radiomics descriptors for each tumor subregion (Whole Tumor—WT, Enhancing Tumor—ET, Tumor Core—TC). These include:

• Mesh Volume ($V_{mesh}$): triangulated surface integral,

$$V_{mesh} = \sum_{f_j \in F} \frac{1}{6} |v_{i1} \cdot (v_{i2} \times v_{i3})|$$

• Voxel Volume ($V_{voxel}$): voxel count times voxel spacing,

$$V_{voxel} = n_{vox} \times \Delta x\,\Delta y\,\Delta z$$

• Surface Area ($A_{surface}$): summed triangle areas,

$$A_{surface} = \sum_{f_j \in F} \frac{1}{2} \| (v_{i2} - v_{i1}) \times (v_{i3} - v_{i1}) \|$$

• Sphericity ($\psi$): compactness measure,

$$\psi = \frac{\pi^{1/3} (6V_{mesh})^{2/3}}{A_{surface}}$$

In addition, mean ($\mu$) and standard deviation ($\sigma$) of intensity are computed by:

$$\mu = \frac{1}{N} \sum_{i=1}^{N} v_i,\quad \sigma = \sqrt{\frac{1}{N} \sum_{i=1}^{N}(v_i - \mu)^2}$$

Volumes are Z-normalized as $Z_{ijk} = (v_{ijk} - \mu)/\sigma$. Grey-level co-occurrence matrix (GLCM) and higher-order features are supported by the IBSI standard, but the core ReFRM3D evaluation focuses on geometric and first-order statistics.

4. Multi-Feature Tumor Marker Classification

For subregion-level classification, ReFRM3D combines deep segmentation features with radiomics vectors. Let $F_{seg}^* \in \mathbb{R}^d$ denote a pooled deep feature vector and $F_{rad}^* \in \mathbb{R}^4$ represent the normalized radiomics features. The composite vector is formed as:

$$F_{comb} = \alpha \cdot F_{seg}^* + \beta \cdot F_{rad}^*$$

where $\alpha, \beta$ are learned scalars. This fused representation is input to fully-connected layers with ReLU activation, followed by softmax for classification. Training utilizes cross-entropy loss with $\ell_2$ regularization:

$$\mathcal{L}_{cls} = -\sum_{c} y_c\log p_c + \lambda \|W\|_2^2$$

5. Training Protocol and Optimization

Datasets used for training and validation are BraTS2019, BraTS2020, and BraTS2021. Processing includes .nii to .npy conversion, brain isolation via 99th-percentile thresholding and connected components, PCA-guided cropping, Z-score normalization, tumor slice selection, and volume resizing to $128^3$ voxels. The Adam optimizer ($\beta_1=0.9, \beta_2=0.999$) is used with a fixed learning rate of $10^{-4}$ and batch size 2. The total loss aggregates Dice similarity and categorical focal loss:

$$\mathcal{L}_{Dice} = 1 - \frac{2\sum_{i} w_c\,y_i\,\hat{y}_i}{\sum_{i} w_c\,y_i + \sum_{i} w_c\,\hat{y}_i}$$

$$\mathcal{L}_{Focal} = -\alpha_c(1-p_t)^{\gamma}\log(p_t)$$

$$\mathcal{L}_{total} = \mathcal{L}_{Dice} + \mathcal{L}_{Focal}$$

with class weights $w_c$ and focusing $\gamma = 2$. Data augmentation is limited to random flips and intensity shifts; no learning rate scheduling is used.

6. Experimental Evaluation

Segmentation performance on BraTS datasets is quantified using Dice Similarity Coefficient (DSC), Jaccard index (JSC), sensitivity, and specificity. ReFRM3D achieves:

Dataset	DSC (WT)	DSC (ET)	DSC (TC)
BraTS2019	94.04%	92.68%	93.64%
BraTS2020	94.09%	92.91%	93.84%
BraTS2021	93.70%	90.36%	92.13%

Sensitivity and specificity for all subregions exceed 90%. Compared to WLFS and CH-UNet (with reported DSCS of 0.887–0.901 and 0.780–0.900 respectively on BraTS2019), ReFRM3D demonstrates superior consistency across tumor compartments.

Ablation studies substantiate the contribution of each architectural component:

• Base 3D U-Net: 81.63–84.97% mean DSC
• +FMFF: 85.90–89.73%
• +HURI: 90.01–91.77%
• +Residual Skips (full ReFRM3D): 93.45–93.61%

The combined deep+radiomics classifier yields subregion classification accuracies above 98% across datasets, with both sensitivity and specificity >97%.

7. Significance and Future Perspectives

ReFRM3D establishes a comprehensive strategy for brain tumor segmentation. It demonstrates that tight integration of multi-scale deep feature extraction, sophisticated decoder design, and classical radiomics provides robust performance advantages for complex medical imaging tasks (Rahman et al., 27 Dec 2025). This suggests that future architectures for volumetric segmentation and tumor phenotype prediction can achieve further gains by expanding on hybrid feature paradigms. A plausible implication is that analogous designs may generalize to other domains where geometric and deep features are both informative, given appropriate dataset curation and preprocessing workflows.