F3-Net: Unified MRI Abnormality Segmentation

• F3-Net is a unified model for full abnormality segmentation in brain MRI, designed to integrate multiple diseases into a single binary Pathoseg mask.
• It extends nnU-Net with a multi-encoder, shared-decoder architecture and uses zero-image substitution for handling missing MRI modalities.
• The approach achieves competitive Dice scores across glioma, metastasis, ischemic stroke, and white matter lesion contexts, enhancing cross-dataset generalizability.

Searching arXiv for the cited F3-Net paper and related naming usage. arxiv_search.query({"^^^^^^^^2search_query2 OR ti:\2^^^^^^^^"F3-Net\"","max_results":5,"sort_by":"relevance","sort_order":"descending"}) F3-Net is a unified, modality-flexible foundation model for full abnormality segmentation of medical images, developed to address four recurrent constraints in clinical brain MRI segmentation: reliance on complete multimodal inputs, limited generalizability across diseases and institutions, narrow task specificity, and domain shifts across scanners, protocols, and acquisition sites. It builds on nnU-Net and recasts heterogeneous lesion segmentation tasks into a single pathology-agnostic formulation, producing a binary “Pathoseg” mask that covers the main pathology and coexisting white matter hyperintensity-like lesions. In the reported evaluation, F3-Net is applied to glioma, metastasis, ischemic stroke, and white matter lesion contexts, with a single architecture that operates without disease-specific retraining (&&&^^^^2search_query2^^^^&&&).

^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^. Problem formulation and scope

F3-Net is defined around full abnormality segmentation, described as a single unified segmentation task that captures all pathological voxels present in a study, regardless of their underlying etiology or label schema. Rather than learning separate models for glioma, metastasis, infarction, or white matter hyperintensities, the method harmonizes these targets into a binary lesion objective. The resulting output, termed Pathoseg, is intended to provide an “all-in-one” detection of abnormal tissue (&&&^^^^2search_query2^^^^&&&).

The pathologies covered in the reported work span pre- and post-treatment glioma, brain metastases, ischemic stroke infarcts, and white matter hyperintensities. Two output modes are used. For whole pathology segmentation, the target is the binary union of the dataset’s main pathology mask and a co-registered WMH mask. For main-pathology comparisons, the same architecture is evaluated against tumor-only or infarct-only labels, depending on the dataset. This design separates the model’s unified training objective from benchmark-specific comparisons.

The paper frames this formulation as a response to the practical variability of clinical MRI acquisition. A central motivation is that disease-specific models typically require retraining for each pathology, whereas F3-Net is trained to learn a pathology-agnostic representation that transfers across datasets and institutions. This suggests a deployment model in which one segmentation system can be applied across multiple abnormality categories, although the output remains binary rather than subregional.

2. Architecture and modality-flexible design

F3-Net extends nnU-Net into a modality-aware, multi-pathology framework. The core architecture is a multi-encoder, shared decoder 3D U-Net variant implemented as a pure CNN, with no transformer blocks and no reported attention mechanisms. Each input modality has its own encoder, reflecting the distinct statistical characteristics and contrast mechanisms of that modality, while a single shared decoder reconstructs the segmentation at full resolution from a merged bottleneck representation (&&&^^^^2search_query2^^^^&&&).

The model processes volumetric patches of size PRESERVED_PLACEHOLDER_^^^^2search_query2^^^^ and expects up to six MRI modalities: T^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^, T^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^-Gd, T2, FLAIR, DWI, and ADC. The output is a voxel-wise binary lesion mask, either as Pathoseg for whole abnormality segmentation or as a main pathology-only mask for dataset-specific evaluation. Encoded modality features are merged at the deepest stage of the network, and U-Net-style skip connections are retained in a manner consistent with nnU-Net design.

A defining feature is the zero-image strategy for missing modalities. Missing inputs are replaced by zero-filled volumes so that the model always receives a fixed six-channel interface:

PRESERVED_PLACEHOLDER_^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^

Each modality-specific encoder $E_m$ produces hierarchical features. At the deepest encoder level $L$, F3-Net applies modality-aware feature neutralization:

$$f_m^L = E_m^L(I_m'), \qquad f_m^{L'} = s_m \cdot f_m^L.$$

The merged bottleneck representation is then formed as

$F^L = \mathrm{Merge}(\{f_m^{L'} \mid m \in M\}).$

Because $f_m^{L'}$ is multiplied by $s_m$, a missing modality contributes neither features nor gradients at the bottleneck. The paper emphasizes that this avoids explicit synthesis networks such as GANs or diffusion models and therefore avoids hallucinated modality content. No parameter count is reported, and architectural, preprocessing, and augmentation defaults otherwise follow nnU-Net conventions.

3. Datasets, modalities, and label harmonization

F3-Net is evaluated on BraTS 2^^^^2search_query22id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^, BraTS 2^^^^2search_query22^^^^4, and ISLES 2^^^^2search_query222^^^^, with additional WMH processing used to construct the whole-pathology target. The included datasets differ substantially in label schema and modality availability, which makes cross-dataset harmonization a central component of the method (&&&^^^^2search_query2^^^^&&&).

Dataset	Modalities	Role in F3-Net
BraTS 2^^^^2search_query22id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^	T^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^, T^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^-Gd, T2, FLAIR	Tumor-only and Pathoseg
BraTS-GLI 2^^^^2search_query22^^^^4	T^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^, T^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^-Gd, T2, FLAIR	Glioma-only and Pathoseg
BraTS-MET 2^^^^2search_query22^^^^4	T^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^, T^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^-Gd, T2, FLAIR	Metastasis-only and Pathoseg
ISLES 2^^^^2search_query222^^^^	FLAIR, DWI, ADC	Infarct-only and Pathoseg
WMH dataset	3D T^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^-weighted, 2D FLAIR	WMH mask generation
Multiple Sclerosis Lesion dataset (Shifts 2.^^^^2search_query2^^^^)	T^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^, FLAIR	Benchmark context only

In BraTS 2^^^^2search_query22id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^, the source annotations comprise enhancing tumor, non-enhancing tumor core, and peritumoral edema/non-enhancing FLAIR hyperintensity. In BraTS 2^^^^2search_query22^^^^4, glioma labels include enhancing tumor, non-enhancing tumor core, subtracted non-enhancing FLAIR hyperintensity, and resection cavity; metastasis labels include enhancing tumor, necrotic core, and peritumoral edema. ISLES 2^^^^2search_query222^^^^ provides infarct annotations from neuroradiologists in native space with FLAIR, DWI, and derived ADC.

The label harmonization pipeline is explicit. A vanilla nnU-Net is first trained on a WMH dataset of ^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2descending2search_query2^^^^ cases across 5 scanners to segment WMH on FLAIR. These WMH masks are then corrected by neuroradiologists, co-registered to the other available modalities, merged with the dataset-native main pathology masks, and binarized to form Pathoseg. The Multiple Sclerosis Lesion dataset is not used to train F3-Net; it is included as benchmark context for the paper’s broader generalization motivation.

4. Optimization, training protocol, and inference

F3-Net uses the standard combination of Dice and cross-entropy losses. The paper reports the Dice Similarity Coefficient as

$\mathrm{DSC}(X,Y) = \frac{2|X \cap Y|}{|X| + |Y|},$

the soft Dice form

$$\mathrm{Dice}_{soft}(p,g) = \frac{2\sum_i p_i g_i + \epsilon}{\sum_i p_i + \sum_i g_i + \epsilon},$$

the cross-entropy loss

PRESERVED_PLACEHOLDER_^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2search_query2^^^^

and the final objective

PRESERVED_PLACEHOLDER_^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^

Reported evaluation metrics are Accuracy, Sensitivity, Specificity, Precision, and Dice; HD95 is not reported (&&&^^^^2search_query2^^^^&&&).

Training follows nnU-Net preprocessing conventions and dataset-specific benchmark pipelines, including DICOM-to-NIfTI conversion where applicable, skull stripping, registration, resampling to ^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^ mm isotropic resolution, and intensity normalization. The optimization settings are SGD with momentum ^^^^2search_query2^^^^.95, weight decay PRESERVED_PLACEHOLDER_^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\22^^^^, and initial learning rate ^^^^2search_query2^^^^.^^^^2search_query2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^ with polynomial decay of power ^^^^2search_query2^^^^.9. The global batch size is 2, and training runs for up to ^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2search_query2search_query2search_query2^^^^ epochs. No self-supervised pretraining is reported, and no explicit multi-task curriculum is described.

At inference time, absent modalities are again replaced by zero-images, and the same deepest-layer feature neutralization is applied with PRESERVED_PLACEHOLDER_^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^3 for missing channels. No post-processing is reported, and the shared decoder directly produces the segmentation. Calibration and uncertainty estimation are also not reported. Beyond deterministic zero-image substitution induced by natural missingness, no explicit stochastic masking policy is described.

5. Empirical performance and comparative results

For whole pathology segmentation using the Pathoseg target, F3-Net reports average Dice scores of 94.3^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^% on BraTS-GLI 2^^^^2search_query22^^^^4, 82.^^^^2search_query2^^^^7% on BraTS-MET 2^^^^2search_query22^^^^4, 94.^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\22^^^^% on BraTS 2^^^^2search_query22id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^, and 79.92% on ISLES 2^^^^2search_query222^^^^. Representative dataset-level averages include: BraTS 2^^^^2search_query22^^^^4 aggregate Dice 93.95%, Accuracy 99.88%, Sensitivity 93.5^^^^2search_query2^^^^%, Specificity 99.94%, and Precision 94.87%; BraTS-GLI 2^^^^2search_query22^^^^4 Dice 94.3^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^%, Accuracy 99.89%, Sensitivity 93.97%, Specificity 99.95%, and Precision 95.22%; BraTS-MET 2^^^^2search_query22^^^^4 Dice 82.^^^^2search_query2^^^^7%, Accuracy 99.84%, Sensitivity 8^^^^2search_query2^^^^.74%, Specificity 99.93%, and Precision 86.27%; BraTS 2^^^^2search_query22id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^ Dice 94.^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\22^^^^%, Accuracy 99.88%, Sensitivity 93.5^^^^2search_query2^^^^%, Specificity 99.95%, and Precision 94.54%; and ISLES 2^^^^2search_query222^^^^ Dice 79.92%, Accuracy 99.66%, Sensitivity 78.22%, Specificity 99.84%, and Precision 83.8^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^% (&&&^^^^2search_query2^^^^&&&).

For main-pathology comparisons, the reported DSC values are as follows. On BraTS-GLI 2^^^^2search_query22^^^^4, F3-Net achieves 93.^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^3 versus nnU-Net 88.9^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^, Swin UNETR 82.^^^^2search_query2^^^^4, and MedNeXt 86.^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^4. On BraTS-MET 2^^^^2search_query22^^^^4, F3-Net achieves 8^^^^2search_query2^^^^.95 versus nnU-Net 72.82, Swin UNETR 6^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^.^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^7, and MedNeXt 7^^^^2search_query2^^^^.66. On BraTS 2^^^^2search_query22id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^, F3-Net achieves 93.42 versus nnU-Net 9^^^^2search_query2^^^^.89, Swin UNETR 9^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^.8^^^^2search_query2^^^^, and MedNeXt 9^^^^2search_query2^^^^.55. On ISLES 2^^^^2search_query222^^^^ infarct-only segmentation, F3-Net achieves 77.28, while nnU-Net and SegResNet achieve 8^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^.87 and 82.23, respectively.

The paper interprets these findings as showing that unified training achieves state-of-the-art or near-state-of-the-art accuracy across glioma and metastasis without disease-specific retraining, while remaining competitive for ischemic stroke segmentation. It also states that the datasets are multi-institutional, multi-scanner, and multi-protocol, and that F3-Net sustains high average DSC under these conditions. Under missing modalities, the zero-image strategy with feature neutralization is presented as yielding stable performance, although quantitative ablations on modality removal are not reported.

6. Limitations, positioning, and naming clarification

Several limitations are stated explicitly. Zero substitution cannot recover modality-specific biological signals, so performance may degrade when critical modalities are absent, such as T^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^-Gd for enhancing components or DWI/ADC for acute infarcts. The full abnormality target is a binary union mask rather than a subregion label set, so subregion segmentation of enhancing tumor, necrotic core, edema, or resection cavity is outside the target formulation. Systematic cross-institution generalization experiments such as leave-one-site-out and statistical significance analyses are not reported. Resource profiling is also limited: the batch size of 2 is reported, but scalability, speed, inference memory, and compute footprint are not documented (&&&^^^^2search_query2^^^^&&&).

The paper positions F3-Net as a foundation-style segmentation model on three grounds: training across multiple diseases while integrating WMH into a single full-abnormality objective, native handling of incomplete modality inputs through zero-image substitution and modality-aware feature neutralization, and strong performance under clinical heterogeneity and domain shifts without retraining per disease. It further contrasts F3-Net with prior multi-disease segmentation approaches by emphasizing explicit label harmonization into Pathoseg and the avoidance of modality synthesis networks.

A recurrent naming issue warrants clarification. F3-Net in this context refers to the medical image segmentation model introduced in “F3-Net: Foundation Model for Full Abnormality Segmentation of Medical Images with Flexible Input Modality Requirement.” It should not be conflated with “Fast Feature Field (PRESERVED_PLACEHOLDER_^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^4): A Predictive Representation of Events,” whose authors explicitly state that they use the name Fast Feature Field (PRESERVED_PLACEHOLDER_^^^^2id:(Otaghsara et al., 11 Jul 2025) OR ti:\2^^^^5) rather than “F3-Net” (Das et al., 29 Sep 2025). This distinction is important because the two works address different problem domains, architectures, and data modalities.