Threats to Arabic Handwriting Recognition: Investigating Black-Box Adversarial Attacks on embedded ConvNet models

Abstract: Arabic handwriting recognition (AHR) has made significant progress with deep learning models. AHR research has largely focused on performance, with security receiving little attention. This study provides what appears to be a new line of inquiry by demonstrating the vulnerability of high-performing models to adversarial black-box attacks. The focus on black-box attacks reflects real-world scenarios where the attacker has no prior knowledge of the model architecture. Extensive experiments were conducted on two benchmark AHR datasets containing Arabic handwritten Characters. Results demonstrated the effectiveness of the attacks, with the Pixle attack achieving an attack success rate of 99-100\% on most models. Other, less aggressive attacks achieved success rates of 50-96\% across most experiments. Despite the higher attack success rate, the attacks maintain the structural integrity of the characters, rendering them almost imperceptible to the human eye. The findings indicate the higher vulnerability of the studied models to adversarial manipulation. This underscores the need to strengthen efforts to secure these models and ensure their reliability in AHR real-world applications.

• The paper provides a systematic security audit of four lightweight ConvNets across AHCD and IFHCDB, showing that Pixle reduced most models to near-zero accuracy with attack success rates up to 100%.
• The findings show that compact architectures do not guarantee robustness: ShuffleNet and MnasNet were generally most vulnerable, while SqueezeNet showed the strongest resistance in several tests, including 84.13% accuracy against Pixle on AHCD.
• The results demonstrate that visually subtle pixel rearrangements, randomized square perturbations, and transferable FGSM or PGD attacks can compromise real-world applications such as document digitization, banking, and automated grading.

Motivation and problem statement

Arabic handwriting recognition (AHR) has advanced considerably through ConvNet-based architectures, but the literature has concentrated almost exclusively on recognition accuracy. The paper under review addresses a gap that its authors state has not been previously examined: no prior study has systematically evaluated the security of AHR models against adversarial threats (2605.18058). This omission matters because AHR systems are deployed in settings where misclassification carries real consequences—automated grading of handwritten exams, recognition of religious texts, banking data entry, and document digitization in energy and mining industries. An adversary who can subtly corrupt a scanned character image can therefore corrupt downstream records while leaving the image visually intact to human inspectors.

The study focuses on black-box attacks, in which the attacker has no access to model architecture or parameters and must rely on query outputs or transferable perturbations. This threat model reflects realistic deployment conditions for embedded and mobile systems, where models are exposed as opaque services.

Experimental design

The authors evaluate four lightweight ConvNet architectures selected explicitly for embedded deployment: MobileNetV3-small (2.5M parameters), MnasNet-0.5 (2.2M), ShuffleNet V2 ×0.5 (1.4M), and SqueezeNet 1.1 (1.2M). For comparison, they note that top ImageNet models such as CoAtNet-7 and ViT-G/14 require 1.4–2.4 billion parameters—three orders of magnitude larger—justifying the choice of compact models for resource-constrained environments.

Experiments use two benchmarks: AHCD (16,800 isolated Arabic characters at $32\times32$) and IFHCDB (70,120 images; only Arabic characters retained), with clean test accuracies ranging from 86.13% (MnasNet on AHCD) to 98.08% (MobileNet on IFHCDB). Models were trained under three transfer learning regimes (from scratch, full fine-tuning, half fine-tuning), and the best variant per architecture was selected for adversarial evaluation.

Five attacks were applied. Three are direct black-box attacks: Gaussian Noise (GN), Square Attack (query-efficient randomized square-shaped perturbations), and Pixle (pixel-rearrangement attack preserving natural image statistics). Two white-box attacks, FGSM and PGD, were converted to black-box scenarios by generating adversarial examples on an EfficientNet surrogate and transferring them to the targets. Evaluation metrics are clean accuracy, attack accuracy, and attack success rate (ASR).

Attack effectiveness

The central empirical finding is that Pixle is overwhelmingly effective. On IFHCDB it reduces SqueezeNet and ShuffleNet to 0.00% accuracy (ASR of 100%) and MobileNet to 0.02% (ASR 99.98%); on AHCD it drives MobileNet, ShuffleNet, and MnasNet below 0.6% accuracy (ASRs of 99.47–99.93%). The single exception is instructive: SqueezeNet retains 84.13% accuracy under Pixle on AHCD (ASR 11.44%), making it the only model–dataset combination to withstand the attack. The authors attribute this tentatively to better generalization on shallower or non-noisy data, but they do not provide a mechanistic explanation—a limitation acknowledged implicitly in their call for future work on why certain models resist certain attacks.

GN and Square attacks also perform strongly but less uniformly. GN achieves ASRs above 89% on most AHCD models (96.27% for MobileNet) yet only 52.24% on SqueezeNet for IFHCDB; Square reaches 96.75% ASR on MnasNet (AHCD) and 89.56% on IFHCDB, generally slightly exceeding GN.

Transferred gradient-based attacks (PGD, FGSM) show markedly dataset-dependent behavior. On IFHCDB they have limited effect—MobileNet retains roughly 85% accuracy—whereas on AHCD they halve MobileNet's accuracy to about 50%. ShuffleNet and MnasNet are consistently more susceptible to transferred perturbations than SqueezeNet and MobileNet. Because transferred attacks carry zero execution cost against the target, the authors identify them as a severe practical threat despite lower ASRs than Pixle.

Model-level vulnerability patterns

Per-model analysis reveals consistent rankings. ShuffleNet is uniformly fragile, dropping to near-zero accuracy under both Pixle and Square across datasets. MnasNet is highly vulnerable in nearly all configurations (5.42% accuracy under Pixle, 9.60% under Square on IFHCDB). MobileNet combines extreme Pixle susceptibility with moderate resilience to transferred attacks. SqueezeNet exhibits the strongest resistance profile overall, besting all models against GN and Square on IFHCDB and uniquely resisting Pixle on AHCD.

Two broader conclusions follow directly from these results. First, compact architectures offer no inherent robustness advantage; their efficiency-oriented designs appear to correlate with heightened adversarial fragility, particularly to sparse pixel-level manipulations. Second, robustness is jointly determined by architecture and dataset characteristics—the same model can be nearly invulnerable on one benchmark and catastrophically vulnerable on another—which implies that security assessments of AHR systems cannot rely on single-dataset evaluations.

A notable property emphasized throughout is perceptual stealth: because Pixle rearranges rather than adds pixel values, adversarial examples preserve structural integrity and remain essentially indistinguishable to human observers, meaning standard human-in-the-loop verification would not detect the manipulation.

Limitations and open questions

Several caveats qualify the findings. The study covers only isolated character recognition; word- and sentence-level AHR pipelines, which dominate practical deployments, are not evaluated. The surrogate-based adaptation of FGSM and PGD introduces dependence on transferability between EfficientNet and the target architectures, so the reported transferred-attack results conflate attack strength with surrogate–target similarity. The paper reports no ASR values for PGD and FGSM, complicating direct comparison with the direct attacks. Finally, the anomalous SqueezeNet result on AHCD—and more generally the architectural and dataset factors governing differential robustness—are left unexplained; the authors explicitly flag identifying these causes as future work, along with evaluating adaptive defenses such as adversarial training, input randomization, and denoising under a robustness–efficiency trade-off.

Conclusion

This work provides the first systematic security audit of ConvNet-based Arabic handwritten character recognition under black-box threat models. Its headline results—that Pixle attains 99–100% ASR on most model–dataset pairs while remaining imperceptible, and that even untargeted noise and zero-cost transferred attacks degrade accuracy substantially—demonstrate that high clean accuracy offers no protection against adversarial manipulation in this domain. The observed interaction between architecture, dataset, and attack type establishes a baseline against which future defense mechanisms tailored to AHR can be measured.