MedicalPatchNet: Patch-Based Chest X-ray Classification

• MedicalPatchNet is a patch-based imaging framework that divides 512×512 chest X-rays into 64 non-overlapping 64×64 patches using an adapted EfficientNet-B0.
• It aggregates individual patch logits via a fixed arithmetic mean to produce 14-class probabilities with competitive AUROC and accuracy compared to traditional models.
• The design enables intrinsic interpretability by mapping patch-level logits to coarse saliency maps, balancing context granularity with computational efficiency.

MedicalPatchNet is a patch-based medical imaging architecture in which an image or volume is decomposed into patches, each patch is processed by shared or structured modules, and the resulting patch-level evidence is aggregated for prediction and visualization. The designation is most directly associated with a chest X-ray classification model that splits 512×512 radiographs into 64 non-overlapping 64×64 patches, extracts patch features with an EfficientNet-B0 adapted to one input channel, averages patch logits, and applies a sigmoid to obtain 14-class probabilities (Wienholt et al., 9 Sep 2025). Its design belongs to a broader patch-based lineage that includes PatchNet for interpretable image classification (Radhakrishnan et al., 2017), Patch Network for medical image segmentation (Song et al., 2023), and sMRI-PatchNet for Alzheimer disease diagnosis and discriminative atrophy localisation with structural MRI (Zhang et al., 2023).

1. Terminological scope and lineage

The literature uses closely related names for several architectures built around patch decomposition, but their tasks and internal mechanisms differ substantially. The common element is that local image regions are treated as first-class computational units rather than merely intermediate receptive fields.

Variant	Imaging task	Defining mechanism
PatchNet	Image classification	Shared patch subnet and simple average over patch scores (Radhakrishnan et al., 2017)
MedicalPatchNet	Chest X-ray classification	EfficientNet-B0 on non-overlapping patches with mean patch-logit aggregation (Wienholt et al., 9 Sep 2025)
PNet	Medical image segmentation	Encoder-decoder CNN with atrous-convolution “Patch block” (Song et al., 2023)
sMRI-PatchNet	AD diagnosis from structural MRI	SHAP-based top-$K$ patch selection, position embeddings, and global-local convolutional blocks (Zhang et al., 2023)

This terminology should not be collapsed into a single canonical blueprint. PatchNet and the chest X-ray MedicalPatchNet are primarily classification models with explicit interpretability mechanisms; PNet is a segmentation network whose “patching” is implemented through cascaded atrous convolutions rather than patch-level voting; sMRI-PatchNet is a 3D diagnosis model in which explainable patch selection precedes patch-based feature extraction.

2. Context-restricted patch classification

The foundational formulation comes from PatchNet, which was introduced as a context-restricted architecture for interpretable image classification. Given an input image $I \in \mathbb{R}^{m \times n \times c}$, PatchNet extracts $l$ overlapping or non-overlapping patches $P^{(j)}$ of size $m' \times n' \times c$. In the melanoma study, a typical choice was $m' = 21$, $n' = 21$ on resized 192×256×3 images. Each patch is processed by a shared local feature extractor $S$, which returns a scalar score

$$Q_{1|P^{(j)}} = \sigma(\hat Z_{P^{(j)}}) \in (0,1).$$

The subnet $S$ comprises seven convolutional layers, each with 64 filters of size 3×3, stride $I \in \mathbb{R}^{m \times n \times c}$0, padding $I \in \mathbb{R}^{m \times n \times c}$1, followed by ReLU, with no pooling. The last convolutional layer produces 64 feature maps of shape $I \in \mathbb{R}^{m \times n \times c}$2; each map is collapsed to one number by a learned linear dot-product plus a $I \in \mathbb{R}^{m \times n \times c}$3 nonlinearity, producing 64 scalars, and a final fully connected layer combines those scalars into $I \in \mathbb{R}^{m \times n \times c}$4.

Global classification is deliberately simple. Given the patch outputs, the image-level probability is

$I \in \mathbb{R}^{m \times n \times c}$5

with no further trainable weights. Training uses the global binary cross-entropy

$I \in \mathbb{R}^{m \times n \times c}$6

Although a more general $I \in \mathbb{R}^{m \times n \times c}$7 can be written, the original formulation sets $I \in \mathbb{R}^{m \times n \times c}$8; context restriction is controlled implicitly by patch size.

The central theoretical result is a trade-off between context and error. As $I \in \mathbb{R}^{m \times n \times c}$9 decrease, global context per patch shrinks, so the network must learn more local features to minimize classification loss. In the paper’s toy two-image analysis, with $l$0 and $l$1,

$l$2

and the optimum occurs at $l$3 and $l$4. The resulting global scores become $l$5 versus $l$6, so the model misclassifies half the time, yet every patch containing a 1 is driven to $l$7, guaranteeing discovery of all local features. In the more general case where class-1 images contain $l$8 patches of shared features $l$9 and $P^{(j)}$0 patches of class-1-specific features $P^{(j)}$1, convergence yields $P^{(j)}$2 and $P^{(j)}$3 for $P^{(j)}$4, so patches purely from $P^{(j)}$5 are flagged strongly and patches purely from $P^{(j)}$6 moderately. The paper’s practical consequence is that smaller patches produce higher training and validation loss but sharper, more localized feature maps, whereas larger patches lower loss but blur the heatmaps (Radhakrishnan et al., 2017).

3. Chest X-ray MedicalPatchNet

The 2025 chest X-ray model formalizes MedicalPatchNet as a patch-based self-explainable multi-label classifier. A single-channel chest X-ray $P^{(j)}$7 is split into $P^{(j)}$8 non-overlapping rectangular patches $P^{(j)}$9, where $m' \times n' \times c$0, $m' \times n' \times c$1, $m' \times n' \times c$2, $m' \times n' \times c$3, and $m' \times n' \times c$4. Thus each patch is 64×64 px. A shared backbone $m' \times n' \times c$5, specifically an EfficientNet-B0 with the first convolutional layer adapted to one input channel, extracts patch features, and a linear head $m' \times n' \times c$6 outputs $m' \times n' \times c$7 pathology logits: $m' \times n' \times c$8 Aggregation is fixed and permutation-invariant: $m' \times n' \times c$9

Training computes a multi-label binary cross-entropy only on the global predictions $m' = 21$0 and image-level targets $m' = 21$1. Optimization uses AdamW, batch size 16 images, effective patch-batch $m' = 21$2, and a OneCycle schedule for 20 epochs with peak learning rate $m' = 21$3 reached at 5% of training, followed by linear annealing. Weight decay is default AdamW $m' = 21$4. Training was performed on Nvidia H100, with training time $m' = 21$5 h/model. The training set is CheXpert train with 223,414 chest X-rays and 64,540 patients, with frontal and lateral treated separately; validation and test use the official CheXpert splits, and localization evaluation uses CheXlocalize val with 234 images and test with 668 images. On-the-fly augmentation consists of random resized crop with scale 0.5–1.0 of full image, random rotation $m' = 21$6, and brightness jitter with factor in $m' = 21$7.

The reported classification performance is close to EfficientNet-B0. For 14 classes, MedicalPatchNet achieves AUROC 0.902 and accuracy 0.848, while EfficientNet-B0 achieves AUROC 0.911 and accuracy 0.843; for the 10 classes in CheXlocalize, MedicalPatchNet achieves AUROC 0.907 and accuracy 0.836, while EfficientNet-B0 achieves AUROC 0.908 and accuracy 0.823, with $m' = 21$8. Code for training, inference, and visualization is publicly available, as are pretrained weights; the implementation uses PyTorch 2.5.1, timm, albumentations or torchvision, and a standard scientific Python stack (Wienholt et al., 9 Sep 2025).

4. Self-explanation and localization

The interpretability mechanism in MedicalPatchNet is intrinsic rather than post-hoc. Because the final decision is an arithmetic mean of the patch logits, each $m' = 21$9 directly votes for positive entries or against negative entries for each class. For class $n' = 21$0, overlaying the scalar $n' = 21$1 at the spatial location of patch $n' = 21$2 produces a coarse saliency map. An optional global-aware variant scales each patch encoding by the global prediction,

$n' = 21$3

and these scaled logits can also be visualized.

To reduce block artifacts, the method can generate multiple saliency maps by shifting the input by offsets $n' = 21$4 in both axes with zero-padding, recomputing $n' = 21$5 for each shift, and averaging the resulting maps. A full pixel-wise shift with $n' = 21$6 yields $n' = 21$7 forward passes and a near-continuous map. Localization is evaluated with hit rate,

$n' = 21$8

where $n' = 21$9, and with mIoU computed after thresholding $S$0 at an optimized cutoff $S$1.

On CheXlocalize, MedicalPatchNet with scaled encodings achieves mean hit rate 0.485, compared with 0.376 for Grad-CAM on EfficientNet-B0. The corresponding mIoU across TP+FP+FN is 0.069 for MedicalPatchNet and 0.052 for Grad-CAM. On TP only, however, MedicalPatchNet achieves 0.168 and Grad-CAM 0.227, with the supplement noting that Grad-CAM++ is marginally best on TP only. This metric split is important: the architecture improves mean hit rate and mIoU across all predictions, but not every localization metric favors the same method. The design choice not to replace average aggregation with max-pool or a learned attention aggregator was explicit; those alternatives were not pursued in order to preserve transparency by design (Wienholt et al., 9 Sep 2025).

5. Other medical instantiations

A segmentation-oriented variant appears as PNet, a purely convolutional encoder-decoder network for medical image segmentation that borrows the Swin-Transformer intuition of processing image patches but implements patching via cascaded atrous convolutions rather than self-attention. The input $S$2 passes through four stages of downsampling, each with a learned 5×5 stride-2 convolution and one Patch block. The Patch block applies two serial atrous 3×3 convolutions with dilation rates $S$3 and $S$4,

$S$5

with BatchNorm and ReLU after each convolution. The decoder performs a single 8× upsampling of the deepest feature map, concatenates it with the earliest high-resolution encoder feature, and applies convolutions, dropout, and two 1×1 projections to produce final per-pixel logits. The original model has no linear patch embedding, no QKV projection, no multi-head dot-product attention, and no positional biases. Using Adam, learning rate $S$6, 200 epochs, and batch sizes 2 for CVC and ETIS and 4 for ISIC-2018 Skin, PNet reports IoU/Dice of 0.9332/0.9599 on CVC-ClinicDB, 0.9405/0.9646 on ETIS, and 0.8946/0.9340 on ISIC-2018 Skin, with 3.38 M parameters and inference speeds up to 126.2 FPS on 224×224 skin lesion inputs (Song et al., 2023).

A second variant is sMRI-PatchNet for Alzheimer disease diagnosis with structural MRI. After AC–PC realignment, B1 inhomogeneity correction, skull-stripping, linear registration to the Colin27 template, and resampling to 181×217×181 voxels, each T$S$7-weighted MRI is partitioned into non-overlapping 25×25×25 cubes, yielding 598 positions. Explainable Patch Localisation and Selection uses a SHAP-based perturbation method relative to a transfer-learning MedicalNet AD classifier, hierarchically partitioned into three levels, and retains the top $S$8 patches. Each selected patch $S$9 is linearly projected to dimension $$Q_{1|P^{(j)}} = \sigma(\hat Z_{P^{(j)}}) \in (0,1).$$0 with a learned position embedding,

$$Q_{1|P^{(j)}} = \sigma(\hat Z_{P^{(j)}}) \in (0,1).$$1

and the resulting $$Q_{1|P^{(j)}} = \sigma(\hat Z_{P^{(j)}}) \in (0,1).$$2 representation is processed by alternating Global Spatial Information and Local Patch Information blocks. The GSI block uses a 6×6 convolution across the patch axis with residual connection; the LPI block uses a 1×1 convolution. The final network depth is $$Q_{1|P^{(j)}} = \sigma(\hat Z_{P^{(j)}}) \in (0,1).$$3, followed by average pooling over patch positions and a single fully connected layer to two outputs with softmax. With Adam, initial learning rate $$Q_{1|P^{(j)}} = \sigma(\hat Z_{P^{(j)}}) \in (0,1).$$4 decayed to $$Q_{1|P^{(j)}} = \sigma(\hat Z_{P^{(j)}}) \in (0,1).$$5, batch size 8, and cross-entropy supervision, sMRI-PatchNet reports for AD vs NC on ADNI-1 an accuracy of $$Q_{1|P^{(j)}} = \sigma(\hat Z_{P^{(j)}}) \in (0,1).$$6 and AUC $$Q_{1|P^{(j)}} = \sigma(\hat Z_{P^{(j)}}) \in (0,1).$$7; for pMCI vs sMCI it reports accuracy $$Q_{1|P^{(j)}} = \sigma(\hat Z_{P^{(j)}}) \in (0,1).$$8 and AUC $$Q_{1|P^{(j)}} = \sigma(\hat Z_{P^{(j)}}) \in (0,1).$$9; and for training on ADNI-1 and testing on ADNI-2/3 it reports accuracy $S$0 and AUC 0.925 (Zhang et al., 2023).

6. Interpretability claims, trade-offs, and limitations

The central interpretability claim across PatchNet-derived models is not merely that they can be visualized, but that patch scores are structurally tied to the prediction rule. In PatchNet, the mean-field-like ensemble averaging is stated to force the subnet to highlight every local feature that helps classification rather than concentrating on a single most discriminative region. In the melanoma experiment, PatchNet-21 achieved validation loss 0.499, validation accuracy 77.3%, test loss 0.519, and test accuracy 75.3%, compared with CAM at 0.538/76.7%/0.604/60.4% and VGG-11 at 0.565/70.0%/0.538/74.2%. Against expert dermoscopic masks on the 90 validation images containing at least one feature, PatchNet-21 achieved average exact match 76.9%, average recall 68.8%, and average AUROC 0.788; CAM achieved 62.6%, 62.2%, and 0.667; Grad-CAM achieved 87.3%, 28.5%, and 0.725. The qualitative description is that PatchNet heatmaps align tightly with pathologist masks, whereas CAM and Grad-CAM blur or sometimes highlight bandages (Radhakrishnan et al., 2017).

These advantages are coupled to explicit constraints. In PatchNet, the accuracy-versus-interpretability trade-off is governed by patch size: smaller patches provide less context, increase training and validation loss, and produce sharper feature maps, while larger patches reduce loss and discover fewer distinct features. In the medical reinterpretation, the model can underperform if lesion features occupy only a tiny fraction of the image because the global vote may drown out rare positive patches, and there is no explicit mechanism to enforce spatial coherence across patches other than averaging. In the chest X-ray implementation, smoothing by shifted patch voting improves visual continuity but increases inference cost to $S$1 forward passes for a full pixel-wise shift. The authors also report that the patch size $S$2 was selected as a trade-off between localization granularity and computational cost, and that max-pool or learned attention aggregation was not adopted in order to preserve transparency by design (Wienholt et al., 9 Sep 2025).

This suggests that MedicalPatchNet is best understood less as a single fixed network than as a patch-centric design principle. In some settings, as in chest radiography, the principle is literal patch-wise classification with arithmetic averaging; in others, as in sMRI-PatchNet, patch selection and positional structure are added to retain global and local information; and in segmentation PNet, the patch idea is translated into dilated-convolution context modeling rather than explicit patch voting. Across these variants, the governing question is the same: how much global context can be sacrificed, or reintroduced in controlled form, in order to obtain sharper, more faithful localization without unacceptable degradation in predictive performance.