Where to Defend? Layer-Wise Adversarial Training for Robust Transformer-Based Semantic Communications

Abstract: Deep learning-based semantic communication (DeepSC), a Transformer-based encoder-decoder, achieves semantic fidelity over noisy channels but remains vulnerable to adversarial perturbations injected at multiple stages of the pipeline. We present a layer-wise robustness framework that compares fast gradient sign method (FGSM), projected gradient descent (PGD), and l2-normalized fast gradient method (FGM) defenses at the embedding output, and then uses PGD to analyze three attack and defense points: the embedding output, encoder output, and channel-encoder bottleneck. We evaluate reconstruction on Europarl and UK Hansard and sentiment classification on SST2 and YELP under additive white Gaussian noise (AWGN) and Rayleigh fading. A first-order damage budget, {epsilon} times the l1 norm of the clean-input loss gradient at each injection point, predicts the attack-severity ordering, and the transfer matrix reveals asymmetric defense transfer: the embedding defense transfers strongly to encoder attacks, whereas encoder defenses degrade robustness against upstream attacks. For reconstruction, encoder-point training yields the largest matched gain but fails severely under embedding attacks; joint embedding-plus-encoder training retains comparable gains under encoder attacks while mitigating this mismatch failure, and Rayleigh fading attenuates robustness gains and degradation. For classification, three of four defenses collapse to constant predictors; only the channel-encoder defense remains non-degenerate, suggesting a protective role for the 128D-to-16D bottleneck. At a signal-to-noise ratio (SNR) of 9 dB and perturbation budget {epsilon} = 0.3, the matched encoder defense recovers bilingual evaluation understudy (BLEU) from ~ 0.10 to ~ 0.64 on Europarl/AWGN, whereas the mismatched encoder defense yields a -0.444 BLEU change relative to the undefended baseline under an embedding attack.