SLENet: EfficientNet with Spatial Non-local Attention

• The paper introduces SLENet, a CNN classifier that replaces traditional SE modules with a SECA mechanism and adds Non-local attention to capture long-range dependencies, achieving 96.31% accuracy.
• The model adapts EfficientNet’s MBConv backbone by enhancing local spatial detail and reducing observer variability in four-way classification of rat estrous stages.
• Empirical evaluations demonstrate that the combined attention strategy boosts performance over baseline EfficientNet, with robust precision, recall, and F1 scores despite increased computational overhead.

Spatial Long-distance EfficientNet (SLENet) is a CNN-based classifier for automatic staging of the rat estrous cycle from stained vaginal smear microscopy images. It is built on EfficientNet’s Mobile Inverted Bottleneck Convolution (MBConv) backbone, but modifies the attention pathway in two places: it replaces the Squeeze-and-Excitation (SE) sub-block inside each MBConv with a Spatial Efficient Channel Attention (SECA) mechanism, and it inserts a Non-local attention block after the final convolutional layer to model long-range dependencies. The network is designed for four-way classification into Proestrus (P), Estrus (E), Metestrus (M), and Diestrus (D), and on a test set of 531 images it achieved 96.31% accuracy, compared with 94.20% for the baseline EfficientNet model (Wang et al., 25 Jul 2025).

1. Biological task and problem formulation

Accurate determination of the estrous cycle is consequential in rodent experiments because hormone-driven physiology affects gene expression, proteins, behavior, and pharmacokinetics or therapeutics. In this setting, variation in estrous stage can alter experimental outcomes, so staging is used to reduce interference in reproductive, pharmacological, and related rat-based studies. Manual staging from vaginal smear cytology is described as time-consuming, subjective, and error-prone, particularly in transitional phases in which mixed cell populations complicate visual assessment (Wang et al., 25 Jul 2025).

SLENet addresses this problem as a four-class image-classification task. The class definitions are cytology-based. Proestrus is characterized by nucleated epithelial cells, few keratinized cells, and no leukocytes. Estrus contains anucleated keratinized epithelial cells only. Metestrus contains a mixture of keratinized cells, nucleated epithelial cells, and leukocytes. Diestrus contains many leukocytes and few nucleated epithelial cells. These distinctions are morphologically interpretable, but the paper emphasizes that borderline stages remain difficult because stage boundaries are gradual rather than discrete.

The stated motivation is not merely automation of annotation workload. The model is intended to standardize staging and reduce observer-dependent variability. This suggests a role for SLENet as a reproducibility-oriented component in experimental workflows where estrous stage is a confounder.

2. Dataset, image formation, and annotation protocol

The dataset comprises 2,655 microscopy images of stained rat vaginal epithelial cells in JPG format, divided by a stratified random 6:2:2 split into 1,593 training images, 531 validation images, and 531 test images. The class distribution is near-balanced across the four phases: P = 646, E = 672, M = 670, and D = 667 (Wang et al., 25 Jul 2025).

Daily vaginal smears were collected over four weeks and stained with 0.2% methylene blue. To diversify cytomorphology, rats were administered cyclophosphamide to induce immunosuppression and leucovorin to mitigate toxicity. The paper states that this produced broader morphological variability representative of different cycle stages, with the intended effect of improving generality across real-world cytological variation.

All images were rescaled to $224 \times 224$ using bicubic interpolation. The kernel function and composite pixel interpolation are given as

$$W(x) = \begin{cases} (a+2)|x|^3 - (a+3)|x|^2 + 1, & |x| \le 1 \ a|x|^3 - 5a|x|^2 + 8a|x| - 4a, & 1 < |x| < 2 \ 0, & \text{otherwise} \end{cases}$$

and

$$I(x,y) = \sum_{i=-1}^{2} \sum_{j=-1}^{2} W(x-x_i)W(y-y_j)I(x_i,y_i).$$

The parameter is reported as $a=-0.5$ or $1$. The resizing strategy is described as a compromise between detail preservation and computational efficiency.

Annotation was performed by one expert with more than six years of experience in veterinary science and reproductive physiology. Labels were assigned using established cytology criteria based on cell types, morphology, and proportions, specifically cornified epithelial cells, leukocytes, and nucleated epithelial cells. Team spot checks reportedly found no significant discrepancy. Preprocessing and augmentation comprised brightness adjustments of $\pm 15\%$, contrast adjustments of $\pm 10\%$, random flips, and random rotations, explicitly to account for smear thickness and staining variability.

3. EfficientNet backbone and SLENet architecture

The baseline comparator is described simply as “EfficientNet,” without specification of the exact B0–Bx variant or the choice of scaling coefficient $\Phi$. The paper reproduces EfficientNet’s compound scaling principles:

$$D = \alpha^\Phi,\quad W = \beta^\Phi,\quad R = \gamma^\Phi,$$

subject to

$\alpha \cdot \beta^2 \cdot \gamma^2 \approx 2.$

The baseline architecture is summarized as 16 stacked MBConv modules together with 2 convolutional layers, 1 global average pooling layer, and 1 fully connected classifier (Wang et al., 25 Jul 2025).

SLENet retains EfficientNet’s macro-structure: initial convolution, MBConv stages, final convolution, global pooling, and fully connected classification. Its architectural novelty lies in two substitutions. First, every MBConv block replaces the original SE module with the proposed SECA block. Second, after the last convolutional layer of the backbone, a Non-local attention module is inserted and its output is fed directly into the pooling layer.

Within the MBConv paradigm, the paper follows the standard sequence of expansion, depthwise convolution, attention, projection, and residual connection. However, several implementation details are explicitly not provided: the exact expansion ratios per stage, depthwise kernel sizes per stage, activation functions, normalization type, stochastic depth, initialization, and pretraining. A dropout layer with rate 0.2 is placed before the fully connected layer as regularization.

In relation to standard EfficientNet, SLENet can therefore be understood as a task-specific attention reconfiguration rather than a new macro-architectural family. It preserves the compound-scaled MBConv backbone but alters how local salience and global dependency structure are encoded.

4. SECA and Non-local attention mechanisms

The paper motivates SECA from a limitation of standard SE. For an input tensor $$W(x) = \begin{cases} (a+2)|x|^3 - (a+3)|x|^2 + 1, & |x| \le 1 \ a|x|^3 - 5a|x|^2 + 8a|x| - 4a, & 1 < |x| < 2 \ 0, & \text{otherwise} \end{cases}$$0, SE computes a channel descriptor by global average pooling,

$$W(x) = \begin{cases} (a+2)|x|^3 - (a+3)|x|^2 + 1, & |x| \le 1 \ a|x|^3 - 5a|x|^2 + 8a|x| - 4a, & 1 < |x| < 2 \ 0, & \text{otherwise} \end{cases}$$1

then applies a two-layer MLP,

$$W(x) = \begin{cases} (a+2)|x|^3 - (a+3)|x|^2 + 1, & |x| \le 1 \ a|x|^3 - 5a|x|^2 + 8a|x| - 4a, & 1 < |x| < 2 \ 0, & \text{otherwise} \end{cases}$$2

with $$W(x) = \begin{cases} (a+2)|x|^3 - (a+3)|x|^2 + 1, & |x| \le 1 \ a|x|^3 - 5a|x|^2 + 8a|x| - 4a, & 1 < |x| < 2 \ 0, & \text{otherwise} \end{cases}$$3 and $$W(x) = \begin{cases} (a+2)|x|^3 - (a+3)|x|^2 + 1, & |x| \le 1 \ a|x|^3 - 5a|x|^2 + 8a|x| - 4a, & 1 < |x| < 2 \ 0, & \text{otherwise} \end{cases}$$4, followed by channel-wise rescaling,

$$W(x) = \begin{cases} (a+2)|x|^3 - (a+3)|x|^2 + 1, & |x| \le 1 \ a|x|^3 - 5a|x|^2 + 8a|x| - 4a, & 1 < |x| < 2 \ 0, & \text{otherwise} \end{cases}$$5

The stated concern is that SE compresses spatial information through global average pooling before channel reweighting, which may obscure diagnostically important local patterns in cytology images such as leukocyte clusters or arrangements of keratinized cells (Wang et al., 25 Jul 2025).

SECA is introduced to jointly capture channel and spatial features while remaining computationally efficient. According to the paper’s description, SECA replaces SE’s fully connected layers by local 1D convolutions, introduces two convolutional layers with a scaling ratio of 4 and a kernel size of 7, and uses a Sigmoid activation to learn spatial weights that are applied to the feature maps. The intended effect is to preserve spatial information and increase sensitivity to locally salient structures.

The paper does not provide a fully explicit mathematical specification of SECA. It does, however, state that a consistent instantiation is to use a channel-mixing branch with local 1D convolutions and a spatial branch that produces a Sigmoid-gated attention map, with fusion of channel and spatial weights. This suggests a hybrid attention operator in which efficient local channel interaction and explicit spatial weighting are coupled inside the MBConv block.

The second attention component is a Non-local module placed after the final convolutional layer. Its stated purpose is to expand the receptive field and model long-range dependencies across the entire image. The paper gives the scaled dot-product self-attention form

$$W(x) = \begin{cases} (a+2)|x|^3 - (a+3)|x|^2 + 1, & |x| \le 1 \ a|x|^3 - 5a|x|^2 + 8a|x| - 4a, & 1 < |x| < 2 \ 0, & \text{otherwise} \end{cases}$$6

and

$$W(x) = \begin{cases} (a+2)|x|^3 - (a+3)|x|^2 + 1, & |x| \le 1 \ a|x|^3 - 5a|x|^2 + 8a|x| - 4a, & 1 < |x| < 2 \ 0, & \text{otherwise} \end{cases}$$7

where $$W(x) = \begin{cases} (a+2)|x|^3 - (a+3)|x|^2 + 1, & |x| \le 1 \ a|x|^3 - 5a|x|^2 + 8a|x| - 4a, & 1 < |x| < 2 \ 0, & \text{otherwise} \end{cases}$$8 is Softmax, $$W(x) = \begin{cases} (a+2)|x|^3 - (a+3)|x|^2 + 1, & |x| \le 1 \ a|x|^3 - 5a|x|^2 + 8a|x| - 4a, & 1 < |x| < 2 \ 0, & \text{otherwise} \end{cases}$$9, $$I(x,y) = \sum_{i=-1}^{2} \sum_{j=-1}^{2} W(x-x_i)W(y-y_j)I(x_i,y_i).$$0, and $$I(x,y) = \sum_{i=-1}^{2} \sum_{j=-1}^{2} W(x-x_i)W(y-y_j)I(x_i,y_i).$$1 are learnable projections, and $$I(x,y) = \sum_{i=-1}^{2} \sum_{j=-1}^{2} W(x-x_i)W(y-y_j)I(x_i,y_i).$$2 is the key or channel dimension. The paper also provides the generic non-local operator

$$I(x,y) = \sum_{i=-1}^{2} \sum_{j=-1}^{2} W(x-x_i)W(y-y_j)I(x_i,y_i).$$3

Conceptually, this tail module allows any spatial position to attend to all others, enabling the network to encode global distribution cues such as relative proportions and spatial arrangement of different cell types. The complementarity posited by the architecture is therefore explicit: SECA emphasizes preservation of local spatial detail within MBConv, whereas the Non-local block captures whole-image dependency structure after feature extraction.

5. Training protocol and evaluation methodology

Training uses CrossEntropyLoss and the AdamW optimizer with learning rate 0.01. AdamW is reported to have been chosen over SGD and Adam on the basis of validation accuracy. The batch size is 16 and training proceeds for 130 epochs, with early stopping of patience 20 monitored on validation accuracy. Input resolution is $$I(x,y) = \sum_{i=-1}^{2} \sum_{j=-1}^{2} W(x-x_i)W(y-y_j)I(x_i,y_i).$$4, and the augmentation pipeline comprises the brightness and contrast adjustments, flips, and rotations described for preprocessing (Wang et al., 25 Jul 2025).

The learning-rate schedule, AdamW betas, and weight decay are not specified. The paper also does not specify initialization or whether any form of pretraining was used. These omissions are relevant for reproducibility, particularly because EfficientNet-family models are often sensitive to training policy and initialization details.

Evaluation uses accuracy, precision, recall, and F1, together with one-vs-rest ROC and PR curves. Experiments were repeated five times under different random seeds, and results are reported as mean $$I(x,y) = \sum_{i=-1}^{2} \sum_{j=-1}^{2} W(x-x_i)W(y-y_j)I(x_i,y_i).$$5 95% confidence intervals. Statistical significance was assessed by paired $$I(x,y) = \sum_{i=-1}^{2} \sum_{j=-1}^{2} W(x-x_i)W(y-y_j)I(x_i,y_i).$$6-tests on per-class F1 scores, averaged to produce an overall $$I(x,y) = \sum_{i=-1}^{2} \sum_{j=-1}^{2} W(x-x_i)W(y-y_j)I(x_i,y_i).$$7-value.

This evaluation design is notable for combining thresholded classification metrics with ranking-based diagnostics. The inclusion of repeated runs and confidence intervals places emphasis on stability, not only point performance.

6. Empirical results, ablations, computational trade-offs, and limitations

On the test set of 531 images, SLENet achieved 96.31% accuracy, compared with 94.20% for the EfficientNet baseline. Across five runs, the mean performance of SLENet was reported as precision 96.27% $$I(x,y) = \sum_{i=-1}^{2} \sum_{j=-1}^{2} W(x-x_i)W(y-y_j)I(x_i,y_i).$$8 3.65%, recall 96.30% $$I(x,y) = \sum_{i=-1}^{2} \sum_{j=-1}^{2} W(x-x_i)W(y-y_j)I(x_i,y_i).$$9 5.76%, and F1 96.26% $a=-0.5$0 4.18% (Wang et al., 25 Jul 2025).

Per-class F1 means were Estrus 99.63 $a=-0.5$1 0.35, Metestrus 93.53 $a=-0.5$2 0.88, Diestrus 95.03 $a=-0.5$3 0.78, and Proestrus 96.86 $a=-0.5$4 1.15. The paper states that SLENet attained the highest F1 in all four classes relative to the other CNN baselines considered. Statistical significance at $a=-0.5$5 was achieved against most baselines; for EfficientNet specifically, the comparison yielded $a=-0.5$6, so the improvement over that baseline was not statistically significant under the reported test, despite higher metric values.

The ROC and PR analyses report AUC and AP above 0.99 for all classes, indicating strong ranking performance and reliable probability calibration despite some residual errors. The confusion matrix analysis identifies the greatest confusion between Metestrus and Diestrus, attributed to similar leukocyte abundance with only subtle proportion shifts. Proestrus is sometimes misclassified as Estrus because late Proestrus shows increasing keratinization resembling Estrus. Estrus and Diestrus are generally well separated due to their distinct morphology.

The ablation study attributes the best performance to the combination of SECA and Non-local attention rather than to either module in isolation. The reported results are: EfficientNet baseline, F1 94.35 $a=-0.5$7 6.69 and accuracy 94.20 $a=-0.5$8 0.99; baseline plus SECA alone, F1 94.78 $a=-0.5$9 5.52 and accuracy 94.55 $1$0 0.86; baseline plus Non-local alone, F1 93.82 $1$1 4.95 and accuracy 94.12 $1$2 0.56; baseline plus Non-local plus CBAM, F1 91.29 $1$3 9.36 and accuracy 91.29 $1$4 2.15; baseline plus Non-local plus Coordinate Attention, F1 94.50 $1$5 5.51 and accuracy 94.39 $1$6 0.65; and baseline plus SECA plus Non-local, namely SLENet, F1 96.26 $1$7 4.18 and accuracy 96.31 $1$8 0.43. The paper’s interpretation is that local spatial sensitivity and global dependency modeling are complementary, while introducing only one component is insufficient or may even be detrimental for this task.

The computational trade-off is quantified as follows. Parameter count increases from 4.01M in the baseline to 14.19M in SLENet, approximately a 3.5-fold increase. FLOPs increase from 6.58G to 9.35G, approximately a 42% increase. Inference latency rises from 32.32 ms to 34.58 ms, approximately a 7% increase. Memory footprint is not reported. The paper characterizes the latency increase as modest relative to the added parameter and FLOP overhead, implying that the inserted attention computations are efficiently implemented.

The limitations are explicit. The model is trained only on microscopy images and does not incorporate temporal tracking of daily sequences or biochemical context such as hormone assays. Distinguishing Metestrus from Diestrus remains difficult because the task depends on gradual changes in leukocyte proportion from static cytology snapshots. The proposed extensions are temporal data, multi-modal inputs, and broader datasets or species. A plausible implication is that the current formulation is optimized for static cytology-based staging, whereas future gains may depend more on richer input structure than on additional architectural complexity alone.