Category Attention Block (CAB) for DR Classification

• Category Attention Block (CAB) is an attention module that partitions feature channels per class to enhance diabetic retinopathy grading and address imbalanced data.
• CAB operates by projecting features into fixed per-class channels, computing class-wise scores and semantic maps, and applying multiplicative gating before pooling.
• The design distinguishes itself from similar acronyms by emphasizing class-specific feature reweighting, unlike MALUNet’s CAB used for multi-stage encoder-decoder fusion in segmentation.

Searching arXiv for the cited CAB-related papers to ground the article in the primary sources. Category Attention Block (CAB) is an attention module introduced for diabetic retinopathy grading to learn discriminative, category-specific attention for $L$-class classification under imbalanced data. In the cited formulation, CAB explicitly allocates a fixed number of feature channels per class, computes class-wise summary scores and semantic maps, and applies a multiplicative attention map to the incoming feature tensor before global average pooling and final classification. In recent arXiv literature, however, the acronym “CAB” is not unique: in the diabetic retinopathy model of (Hannan et al., 25 Jul 2025), it denotes Category Attention Block, whereas MALUNet consistently uses Channel Attention Bridge Block for a different multi-stage channel-attention mechanism in skin lesion segmentation (Ruan et al., 2022).

1. Terminological scope and disambiguation

The most immediate point of clarification is nomenclature. In “Enhancing Diabetic Retinopathy Classification Accuracy through Dual Attention Mechanism in Deep Learning” (Hannan et al., 25 Jul 2025), CAB is the Category Attention Block, a class-aware attention module intended for diabetic retinopathy grading with $L=5$ disease grades. Its central design choice is to partition channels by class and to aggregate them in a category-aware fashion so that each diabetic retinopathy grade receives equal representational capacity.

By contrast, MALUNet uses the same acronym for a different construct. In “MALUNet: A Multi-Attention and Light-weight UNet for Skin Lesion Segmentation” (Ruan et al., 2022), CAB is the Channel Attention Bridge Block, not “Category Attention Block.” That module fuses multi-stage encoder features to generate channel-axis attention vectors for segmentation. The distinction is substantive rather than nominal: the diabetic retinopathy CAB is class-partitioned and classifier-adjacent, whereas MALUNet’s CAB is a bridge attention module for encoder–decoder feature fusion.

This terminological overlap can create confusion in secondary summaries. A plausible implication is that any discussion of “CAB” in recent medical vision literature should be anchored to the full expansion and architectural context rather than to the acronym alone.

2. Architectural role in diabetic retinopathy classification

Within the diabetic retinopathy framework, CAB is inserted after a backbone feature extractor, a $1 \times 1$ convolution for channel adjustment, and a Global Attention Block (GAB), and before the GAP/FC classifier. The sequence is explicitly given as:

backbone feature extractor $\rightarrow$ $1 \times 1$ Conv (channel adjust/reduction) $\rightarrow$ GAB $\rightarrow$ CAB $\rightarrow$ GAP/FC (Hannan et al., 25 Jul 2025).

The paper uses this arrangement in three pre-trained backbones: MobileNetV3-small, EfficientNet-b0, and DenseNet-169. Only one GAB and one CAB are used per network, and CAB is not inserted at multiple intermediate backbone stages. The design is therefore single-branch and sequential rather than multi-stage or multi-branch.

CAB receives an intermediate tensor

$$F_{\text{CAB-in}} \in \mathbb{R}^{h \times w \times c}$$

and outputs

$$F_{\text{CAB-out}} \in \mathbb{R}^{h \times w \times c}$$

through an attention map

$L=5$0

The input and output retain the same spatial and channel dimensions, with reweighting occurring through multiplicative gating. The stated purpose is to learn category-specific attention for diabetic retinopathy grading and to mitigate bias caused by imbalanced class distributions, particularly where EyePACS is reported as 73.3% No DR, 6.9% Mild, 15.2% Moderate, 2.6% Severe, and 2.0% PDR.

The paper’s rationale is that conventional CNN pipelines fuse all channels irrespective of class, whereas CAB partitions channels per class and aggregates them in a class-aware way, thereby preventing channel “prejudice” against rare classes. This is the defining conceptual claim of the block in its original presentation.

3. Computational formulation

CAB first projects the incoming tensor to $L=5$1 channels using a $L=5$2 convolution:

$L=5$3

where $L=5$4 is the number of diabetic retinopathy grades and $L=5$5 is the number of channels per class. The paper sets $L=5$6 in Algorithm 1 (Hannan et al., 25 Jul 2025).

During training, dropout removes half the projected features to encourage the $L=5$7 maps per class to capture complementary discriminative regions, producing

$L=5$8

Dropout is disabled at inference. The tensor is then interpreted class-wise, with $L=5$9 denoting the $1 \times 1$0-th feature map assigned to class $1 \times 1$1.

Class scores are computed by global max pooling:

$1 \times 1$2

In parallel, class-wise semantic maps are formed by category-wise cross-channel averaging:

$1 \times 1$3

The paper defines the category attention map as

$1 \times 1$4

and applies it multiplicatively to the input features:

$1 \times 1$5

As written in the source, $1 \times 1$6 is derived from $1 \times 1$7, while the text also identifies $1 \times 1$8 as class-specific semantic maps. This suggests that the module combines an explicit class-partitioned representation with a scalarized category-attention construction that is then broadcast back to the spatial feature tensor.

The forward pass reported for the full dual-attention model is:

1. $1 \times 1$9,
2. $\rightarrow$0,
3. $\rightarrow$1,
4. $\rightarrow$2,
5. $\rightarrow$3 with softmax.

4. Dual-attention coupling with GAB

CAB is not presented as a standalone attention block in the diabetic retinopathy system; it is paired with a preceding Global Attention Block. GAB itself applies channel attention followed by spatial attention:

$\rightarrow$4

$\rightarrow$5

Here, GAP denotes global average pooling over spatial dimensions, Conv2 denotes two consecutive $\rightarrow$6 convolutions, $\rightarrow$7 is the sigmoid, and $\rightarrow$8 denotes cross-channel average pooling (Hannan et al., 25 Jul 2025).

The paper’s stated ordering is GAB first and CAB second. GAB is said to extract and preserve global lesion context so that CAB can refine it with category-specific cues. The authors further note that reversing the order harms performance because CAB loses fine-grained details. In this formulation, GAB is category-agnostic and global, while CAB is category-specific and discriminative.

The fusion strategy is sequential and multiplicative throughout. Within GAB, both channel and spatial gates are multiplicative. CAB also reweights features multiplicatively through $\rightarrow$9. No additive or concatenation fusion is reported, and no cross-stage fusion is described for the diabetic retinopathy model.

This design yields a clear division of labor. GAB performs global context refinement; CAB imposes class-structured selectivity. A plausible implication is that the effectiveness of CAB in this setting depends partly on the prior preservation of lesion context by GAB rather than on category-wise attention alone.

5. Class-imbalance handling, optimization, and reported results

The diabetic retinopathy paper frames CAB primarily as an architectural response to imbalanced data. The balancing mechanism is not introduced through a weighted loss, focal loss, or explicit resampling. Instead, CAB enforces that each class receives exactly $1 \times 1$0 channels in the attention projection

$1 \times 1$1

irrespective of class frequency. The paper states that this equal channel allocation counteracts the tendency of the network to allocate more capacity to majority classes and thereby prevents channel prejudice against rare classes (Hannan et al., 25 Jul 2025).

Training uses standard cross-entropy loss with Adam:

$1 \times 1$2

No $1 \times 1$3 or $1 \times 1$4 terms are reported. The training setup reported in the System Specifications section is: input resize to $1 \times 1$5, augmentations consisting of rotations of $1 \times 1$6, $1 \times 1$7, and $1 \times 1$8 plus horizontal flips, train/validation/test split of 50%/30%/20%, batch size 32, initial learning rate $1 \times 1$9, learning-rate reduction by 0.8 after 3 consecutive validation plateaus, and 40 epochs. Transfer learning is performed by loading ImageNet-pretrained backbones, freezing pre-trained layers to train upper layers, then unfreezing and fine-tuning.

The reported ablation on APTOS with DenseNet-169 attributes measurable gains to CAB. The baseline achieves Acc 79.0%, F1 80.0%, and 16.8M parameters. Adding GAB only yields Acc 82.2% and F1 81.5%; adding CAB only yields Acc 82.1% and F1 81.2%; adding GAB+CAB yields Acc 83.6%, F1 82.0%, and 17.0M parameters. The paper therefore reports that both attention blocks individually help and that their combination produces the strongest result with an approximately +0.2M parameter increase over baseline.

At the dataset level, the reported top-line accuracies are 83.6% for DenseNet-169, 82% for MobileNetV3-small, and 80% for EfficientNet-b0 on APTOS, and 80% for EfficientNet-b0, 76.68% for MobileNetV3-small, and 75.43% for DenseNet-169 on EyePACS. The overall reported metrics include Accuracy 83.6%, F1 82.0%, Precision 82.1%, Sensitivity 83.0%, Specificity 95.5%, and Kappa 88.2%.

6. Relation to other attention mechanisms and to MALUNet’s CAB

The diabetic retinopathy paper contrasts its CAB with prior attention mechanisms such as SE, ECA, and CBAM. The stated distinction is conceptual: SE is channel-only and class-agnostic, ECA is lightweight channel-only, and CBAM is sequential channel-plus-spatial attention but still class-agnostic. CAB differs by partitioning channels per class, enforcing equal capacity across classes, and building attention through per-class aggregation (Hannan et al., 25 Jul 2025). In that sense, category awareness rather than merely channel or spatial gating is the module’s defining characteristic.

This differs sharply from MALUNet’s CAB, which should not be conflated with Category Attention Block. In MALUNet, the Channel Attention Bridge Block operates on encoder features from Stages 1–5, computes stage-wise channel descriptors by global average pooling, concatenates them into a descriptor of size $\rightarrow$0 for widths $\rightarrow$1, applies a Conv1D for local fusion, and uses per-stage fully connected mappings to generate channel attention vectors:

$\rightarrow$2

followed by residual channel-wise gating

$\rightarrow$3

for each stage (Ruan et al., 2022).

Its purpose is therefore multi-stage feature fusion for skin lesion segmentation, with explicit local and global fusion across encoder stages to reduce the semantic gap between encoder and decoder features. Empirically, adding MALUNet’s CAB to BasicUNet on ISIC2018 increases parameters from 0.11M to 0.12M, leaves GFLOPs at 0.07, and improves mIoU from 78.22 to 78.98 and DSC from 87.77 to 88.26. These are segmentation-specific properties and should not be transferred to the diabetic retinopathy Category Attention Block.

The comparison illustrates that identical acronyms can encode unrelated attention designs. One CAB is class-partitioned and classifier-oriented; the other is cross-stage, channel-axis, and encoder–decoder bridging. The shared acronym reflects no common canonical definition.

7. Limitations and interpretive considerations

The diabetic retinopathy study explicitly notes several limitations of the dual-attention design. The model is trained with image-level supervision only and does not localize or detect specific lesion types such as microaneurysms, hemorrhages, or soft/hard exudates (Hannan et al., 25 Jul 2025). Small lesions remain challenging to pinpoint precisely, and Grad-CAM is described as showing improved focus rather than explicit lesion detection. The imbalance mitigation is architectural rather than loss-based, since the system uses unweighted cross-entropy and does not report class weighting or focal loss.

The paper also suggests future improvement through synthetic data, including diffusion models, for pretraining to improve robustness and generalization. This suggests that CAB’s category-partitioned inductive bias was viewed as beneficial but not sufficient to solve all difficulties associated with long-tailed retinal grading.

More broadly, the published formulation implies a specific operating regime. CAB is designed for a fixed number of classes, uses a fixed per-class channel budget $\rightarrow$4, and is deployed once near the classifier head rather than throughout the network. A plausible implication is that its benefits are tied to scenarios where class imbalance and class-discriminative regional emphasis are more central than dense prediction or multi-scale feature bridging. That is precisely why the diabetic retinopathy CAB and MALUNet’s Channel Attention Bridge Block, despite sharing the acronym CAB, occupy different places in the design space of attention modules.