Sliding Mask Confidence Entropy (SMCE)

• Sliding Mask Confidence Entropy (SMCE) is a model-driven uncertainty quantification technique that applies localized masking to measure Shannon entropy over classifier outputs.
• It uses a sliding mask procedure to perturb input regions, creating a spatial entropy map that highlights instability in predictions for tasks like adversarial detection and OCR error localization.
• Empirical results show high detection accuracy (up to 96.5%) without retraining models, demonstrating SMCE’s practical impact on enhancing neural network reliability.

Sliding Mask Confidence Entropy (SMCE) is a model-driven uncertainty quantification technique suited for identifying local instability or error-prone regions in neural network predictions, particularly under controlled perturbations such as masking or occlusion. Introduced independently in the analysis of both adversarial robustness in deep learning and error localization in transformer-based OCR, SMCE operationalizes Shannon entropy over classifier confidence under a combinatorial family of localized input corruptions, yielding a scalar or spatial field indicative of prediction volatility.

1. Formal Definition and Mathematical Formulation

Let $I \in \mathbb{R}^{H\times W\times C}$ denote an image and $f(I) = (p_1, ..., p_m)$ the model’s softmax-normalized output across $m$ classes. For a square mask $M_i$ of side length $s$ applied at position $i$, the masked input $I \odot M_i$ sets all pixels in the covered region to zero (or to the dataset mean). The classifier output for the masked image is $p^{(i)} = f(I \odot M_i) = (p_{i1}, ..., p_{im})$. For each mask, the Shannon entropy is

$H_i = -\sum_{j=1}^m p_{ij} \log_2 p_{ij}$

SMCE is defined as the average entropy over all $n$ mask placements:

$$H_{\mathrm{SMCE}}(I) = \frac{1}{n} \sum_{i=1}^n H_i = -\frac{1}{n} \sum_{i=1}^n \sum_{j=1}^m p_{ij} \log_2 p_{ij}$$

By construction, $0 \leq H_{\mathrm{SMCE}}(I) \leq \log_2 m$, with lower values indicating confidence stability under occlusion and higher values signifying increased classification uncertainty or volatility (Li et al., 7 Nov 2025).

2. Sliding Mask Procedure and Algorithmic Details

SMCE measurement requires systematic traversal of the image with a fixed-size, stride-$t$ mask:

• Mask size $s \times s$ governs the granularity of local perturbation (e.g., $s\in\{3,7,9\}$ for CIFAR-10).
• Stride $t=1$ ensures dense coverage.
• For each valid placement $M_i$:
  1. Construct $I \odot M_i$ by masking the specified region.
  2. Obtain softmax $p^{(i)}$ via model $f$.
  3. Compute entropy $H_i$ as above.

The process yields a spatial “mask entropy field map” (MEFM), where $H_i$ values can be visualized at the centers of the mask placements, and the overall SMCE is the mean across positions.

This algorithmic approach is lightweight, requiring only forward passes at test time, with computational cost linear in the number of valid mask placements.

3. Theoretical Underpinnings and Discriminative Power

SMCE exploits the distinct response of clean versus adversarial or erroneous instances to local occlusion. In clean images, class-relevant features often persist despite partial masking, so outputs remain sharply peaked. Adversarial examples, residing close to decision boundaries or relying on fragile patterns, exhibit sharp confidence drops or increased entropy when critical perturbing pixels are masked. Thus, adversarial inputs or OCR errors manifest as high SMCE, due to pronounced prediction volatility under sliding occlusion (Li et al., 7 Nov 2025, Kaltchenko, 30 Apr 2025).

This suggests that SMCE captures aspects of local spatial robustness not measured by traditional scalar confidence metrics.

4. Practical Implementation in Adversarial Detection and OCR

Adversarial Example Detection

On CIFAR-10, SMCE was used to distinguish adversarial from clean samples across nine canonical attack types (FGSM, PGD, DeepFool, JSMA, BIM, FFGSM, APGD, One-Pixel, PIFGSMPP). The procedure is as follows:

1. For each test image, compute $H_{\mathrm{SMCE}}(I)$ using a dense sliding mask.

2. Compare $H_{\mathrm{SMCE}}(I)$ to a threshold $\theta$ (e.g., $0.10$).
3. Classify as “adversarial” if $H_{\mathrm{SMCE}}(I) > \theta$, else “clean.”

Performance varied by attack strength but reached accuracy $\geq 96.5\%$ for JSMA and $> 75\%$ for most attacks with false-positive rates $\leq 10\%$. The method, termed SWM-AED, does not require adversarial retraining and integrates into any softmax-based classifier (Li et al., 7 Nov 2025).

Transformer OCR Error Localization

For vision-LLMs (e.g., GPT-4o) performing OCR, SMCE can be analogously applied at the token level:

• Each token $y_i$ has a (truncated) Shannon entropy $H_i$, computed from the top-$k$ token log-probabilities and a tail bucket,

$$H_i = -\sum_{j=1}^k p_{i,j} \log_2 p_{i,j} - p_{\text{tail}(i)} \log_2 p_{\text{tail}(i)}$$

where

$p_{\text{tail}(i)} = 1 - \sum_{j=1}^k p_{i,j}$

• A fixed-length window (size $W$) slides over $\{H_1, ..., H_n\}$,

$A_i = \frac{1}{W}\sum_{r=i}^{i+W-1} H_r$

• The highest $M$ windows in $A_i$ are flagged as likely OCR error “hotspots”, with strong correspondence to human-identified errors found empirically (Kaltchenko, 30 Apr 2025).

5. Statistical Analysis and Empirical Results

Adversarial Example Context

The empirical distribution of $H_{\mathrm{SMCE}}(I)$ over clean and adversarial samples reveals:

• Clean images: $H_{\mathrm{SMCE}} \in [0.02, 0.08]$
• Adversarial images: $H_{\mathrm{SMCE}} \in [0.10, 0.30]$ (dependent on attack)
• Histogram right-shift and broadening for adversarial samples, with minimal overlap in typical settings
• Discriminative thresholds (e.g., $\theta=0.10$) result in high detection rates and low false-positive rates across models (ResNet-18, ResNet-50, VGG-11)

OCR Error Localization

• Sliding-window mean entropy $A_i$ successfully localizes $>90\%$ of actual GPT-4o OCR transcription errors within the top 3 flagged hotspots.
• Window size $W=10$ offered the best trade-off between localization precision and review coverage.
• Review load constrained to $\sim 10$–$15\%$ of the transcript content (Kaltchenko, 30 Apr 2025).

6. Extensions, Visualizations, and Implications

SMCE supports several window-level summaries beyond mean, including maximal and sum statistics, broadening its applicability for both dense error heatmaps and global scalar volatility measures. Mask Entropy Field Maps visualize spatial uncertainty, revealing global instability patterns characteristic of adversarial perturbations.

These findings establish SMCE as a robust, computationally efficient primitive for both detection (of adversarial or unstable instances) and localization (of likely error regions in sequence prediction). It avoids the need for model retraining, is compatible with standard architectures, and its discriminative power improves with model accuracy.

A plausible implication is that “occlusion stability”—invariance of prediction to local masking perturbations—may serve as a promising future axis for defense and calibration strategies in deep learning models.

7. Summary Table: Parameterization and Reported Ranges

Context	Mask/Window Size	Threshold(s) Used	Typical SMCE Values
CIFAR-10	$s = 3, 7, 9$ (pixels)	$\theta = 0.10$	Clean: $0.02$–$0.08$; Adv: $0.10$–$0.30$
OCR	$W = 5, 10, 20$ (tokens)	$M=3$ top hotspots	$A_i$: application-dependent

Both use stride 1 sliding. For OCR tasks, top-$k$ probabilities with truncation via a “tail” event are recommended to bound the entropy appropriately (Kaltchenko, 30 Apr 2025, Li et al., 7 Nov 2025).