BTDNet Radiogenomic Classification

• BTDNet is a multi-modal deep neural network designed for radiogenomic classification in brain tumor analysis, integrating CNN-RNN modules, advanced data augmentation, and routing to handle variable-length 3D MRI data.
• It employs four computational modules—data augmentation, slice-wise feature extraction, routing/masking, and multi-modal fusion—to achieve state-of-the-art performance with a 66.2% macro F1 score, outperforming prior methods by 3.3%.
• The architecture leverages test-time augmentation, specialized focal loss, and rigorous ablation studies to address volumetric imaging challenges and optimize MGMT promoter methylation prediction.

BTDNet is a multi-modal deep neural architecture for radiogenomic classification in brain tumor analysis, designed to address variable-length 3D MRI volume data and volume-level annotation. It leverages four complementary computational modules—advanced data augmentation, slice-wise 3D feature extraction via CNN-RNN, a routing/masking mechanism to handle input heterogeneity, and a late fusion scheme for multi-parametric MRI integration—providing state-of-the-art performance for prediction of MGMT promoter methylation status. BTDNet is distinct in its approach to both data-level and architectural challenges posed by multi-modal, volumetric medical imaging (Kollias et al., 2023).

1. Architecture and Computational Components

BTDNet comprises four principal components:

1. Data Augmentation:

Training and inference are both augmented at multiple levels. Random geometric transforms (slice-wise flips and rotations) as well as an “MixAugment” strategy (an extension of mixup) are used to generate convex combinations of volume-label pairs, facilitating virtual data synthesis. During inference, test-time augmentation (TTA) is employed by aggregating predictions across multiple transformed instances of each test case.

2. 3D Analysis via CNN + RNN:

Each 3D MRI volume is processed as a sequence of 2D slices. A slice-wise 2D convolutional neural network (ResNet18, sans classifier) extracts per-slice features. These features feed into a uni-directional LSTM which encodes inter-slice dependencies, producing a sequence of hidden states per volume.

3. Routing and Variable-Length Handling:

To accommodate variable slice counts, the RNN outputs are padded to a fixed maximum length and concatenated. A binary mask derived from the true (unpadded) length zeros out padded positions. This masked vector passes through a fully-connected block with batch normalization and GELU.

4. Multi-Modal Fusion:

BTDNet employs four parallel CNN-RNN-routing pipelines for each MRI modality (FLAIR, T1w, T1wCE, T2), all sharing weights. The modality-specific 64-dimensional representations are concatenated (forming a 256-dimensional feature), embedded via a GELU-activated FC transform, and classified via softmax.

2. Mathematical Foundations

BTDNet's operations are precisely formulated as follows:

• Slice-wise Geometric Transforms:

$$T(X) = \left(T_1(x_1), T_2(x_2), \dots, T_t(x_t)\right)$$, with each $T_i$ randomly flipping or rotating $x_i$.

• MixAugment:

Given two augmented volumes $T(X_i)$, $T(X_j)$, with labels $y_i$, $y_j$, and $\lambda \sim \mathrm{Beta}(\alpha, \alpha)$, construct $\tilde X = \lambda T(X_i) + (1-\lambda)T(X_j)$, $\tilde y = \lambda y_i + (1-\lambda)y_j$.

• Test-Time Augmentation:

Aggregate logits $T_i$0 from original, flipped, rotated, and combined augmented inputs.

• 2D CNN per slice:

For each layer $T_i$1: $T_i$2.

• RNN with LSTM:

$T_i$3 across the $T_i$4 slices.

• Masking:

Concatenate $T_i$5; mask with a binary vector $T_i$6 indicating valid slices, then $T_i$7 (block-diagonal masking), and pass to $T_i$8.

• Fusion:

Concatenate four modalities $T_i$9, project to joint features $x_i$0, then classify $x_i$1.

3. Training Protocol, Loss, and Optimization

• Loss:

A multi-class focal loss is used, $x_i$2, where $x_i$3 denotes prediction probability. This is extended across both real and synthetic (MixAugment-generated) volumes, $x_i$4.

• Optimizer:

Stochastic gradient descent with momentum 0.9, enhanced by Sharpness-Aware Minimization (SAM), is applied. Learning rates are $x_i$5 for training each modality stream from scratch and $x_i$6 for fine-tuning multi-modal integration. No explicit weight decay is used, apart from BatchNorm (Kollias et al., 2023).

• Pre-processing:

Volumes are skull-stripped, cropped to the brain ROI, and void slices are discarded. Slices are resized to $x_i$7 and intensity-normalized to $x_i$8. Padding to fixed slice count is modality-specific: FLAIR and T2 to 250 slices, T1w and T1wCE to 200 slices. Batch size is four due to resource constraints.

4. Performance and Benchmark Analysis

BTDNet was evaluated on the RSNA-ASNR-MICCAI BraTS 2021 radiogenomics challenge dataset. The primary evaluation metric is macro F1 score.

Method	Macro F1 (%)	F1 Spread
BTDNet	$x_i$9	$T(X_i)$0
3D-ResNet10-Trick (SOTA)	$T(X_i)$1	$T(X_i)$2
EfficientNet-LSTM-mpMRI	$T(X_i)$3	$T(X_i)$4
Other baselines	$T(X_i)$5–$T(X_i)$6	—

BTDNet outperforms the best previously published method by 3.3 percentage points in mean macro F1 and demonstrates lower cross-fold variance (Kollias et al., 2023).

5. Ablation Study and Component Contributions

Extensive ablation studies were performed to quantify the effect of architectural, modal, and augmentation choices:

• CNN backbone: ResNet18 (66.2% F1) is preferred over deeper or alternative architectures, which yield lower F1.
• RNN variant/size: LSTM(128) is optimal; both smaller and larger sizes or GRU alternatives show slightly lower scores.
• Routing/Mask layer: Removing routing drops F1 to 60.9%, demonstrating the importance of explicit variable-length handling.
• Modalities: Single-modality models (FLAIR, T2, T1CE, T1) perform 1–3% worse than full four-modality fusion, confirming the utility of joint multimodal representation.
• Data augmentation: Removal of TTA (−1.0 F1), geometric transforms (−0.9), or MixAugment (−1.5) diminishes performance.
• Loss: Focal loss is superior (66.2%) compared to categorical (64.9%) or binary (64.5%) cross-entropy.

A plausible implication is that each architectural and data-centric intervention contributes additively to overall model robustness and generalization, with explicit handling of variable-length data and multi-modality being especially critical.

6. Implementation and Training Details

• CNN configuration: ResNet18, outputs 512-dim features per slice.
• RNN: Uni-directional LSTM, 128 units.
• Routing dense: 64 units, BatchNorm and GELU.
• Fusion dense: 128 units, GELU.
• Batch protocol: Padding per volume, batch size 4.
• Training protocol: Two-phase; independent pretraining of modality-specific streams, followed by multimodal fine-tuning. No regularizer other than BatchNorm is applied.

Preprocessing and network configuration are matched to the requirements of volumetric clinical MRI, maintaining strict separation between modalities until late fusion.

7. Significance and Distinctions

BTDNet demonstrates that aggressive, multi-granular data augmentation, explicit sequence modeling of slices, variable-length routing, and rigorous multi-modal integration significantly improve MGMT methylation prediction from MRI. Distinct from conventional architectures, BTDNet provides a robust template for handling variable-length, multi-volume, and weakly annotated medical imaging tasks (Kollias et al., 2023). Its modular design, coupled with rigorous benchmarking and ablation, underlines its practical and methodological significance for radiogenomic classification.