Activation-Scaling Guard (ASGuard)

• Activation-Scaling Guard (ASGuard) is a family of mechanisms that use multiplicative scaling to recalibrate neural network activations and mitigate vulnerabilities.
• Different ASGuard instantiations apply scaling at various granularities, enabling targeted interventions for adversarial robustness, optimization stability, and quantization error reduction.
• By precisely modulating activation distributions with minimal overhead, ASGuard addresses challenges in CNN safety, transformer training, and LLM targeted jailbreak mitigation.

Searching arXiv for the cited ASGuard-related papers and closely related activation-scaling work. Activation-Scaling Guard (ASGuard) denotes a family of mechanisms that guard neural computation by multiplicatively recalibrating intermediate activations. In the cited literature, the term appears both as a broad conceptual umbrella for activation-recalibration methods and as the specific name of a 2025 framework for mitigating targeted tense jailbreaking in LLMs. Across convolutional networks and transformers, ASGuard-like methods use activation scaling to suppress adversarially harmful responses, control residual-path variance, tighten activation ranges for quantization, steer internal computations with sparse interventions, or reroute safety-relevant circuitry during alignment. The shared primitive is multiplicative scaling; the major technical differences concern scaling granularity, parameterization, placement, gradient handling, and whether the scaling remains at inference time or is used only during training or preventative fine-tuning (Zhang et al., 2022, Chen et al., 27 Jun 2025, Wickramasinghe et al., 3 Jun 2026, Gonçalves et al., 26 Jun 2026, Stoehr et al., 2024, Park et al., 30 Sep 2025).

1. Conceptual scope and nomenclature

Across recent work, ASGuard-like methods instantiate a common design pattern: identify an activation subspace or activation statistic that correlates with an undesirable or inefficient computation, then rescale it before the downstream layer consumes it. The undesirable computation differs by domain. In CNN robustness, the target is adversarial over-activation or under-activation. In Pre-LayerNorm transformers, it is layerwise activation-variance growth and residual-path dominance. In post-training quantization, it is activation outliers that waste quantizer range. In mechanistic steering, it is a sparse set of activation vectors whose signed magnitudes can be strengthened or weakened without rotating directions. In targeted LLM safety repair, it is a small set of attention heads causally linked to a tense-based jailbreak (Zhang et al., 2022, Chen et al., 27 Jun 2025, Wickramasinghe et al., 3 Jun 2026, Gonçalves et al., 26 Jun 2026, Stoehr et al., 2024, Park et al., 30 Sep 2025).

Instantiation	Scaling object	Principal use
EWAS	Class-aware element-wise mask on CNN feature maps	Adversarial robustness
GPAS / ASGuard	Layerwise scalar with stop-gradient	Variance control in Pre-LN training
QuBLAST activation scaling	Per-block, per-channel static scaling map	PTQ outlier mitigation
A3	Per-sample, per-channel bounded scaling	Adversarial robustness
ActivScalar	Sparse scalars on selected activation vectors	Steering and interpretation
ASGuard	Channel-wise scaling on vulnerable attention heads	Targeted tense-jailbreak mitigation

This taxonomy suggests that ASGuard is better understood as a mechanism class rather than a single architecture. A plausible implication is that the essential object is not the network family but the computational bottleneck being guarded: adversarial sensitivity, quantization fragility, optimization pathology, or safety-circuit vulnerability.

2. Core operator and parameterization patterns

The most general ASGuard operation is multiplicative rescaling of an intermediate tensor. In the CNN setting, Element-Wise Activation Scaling (EWAS) attaches a plug-in module to layer $l$, constructs a class-aware mask $\mathbf{m}\in\mathbb{R}^{C\times H\times W}$, and applies

$\tilde{z}^l = z^l \otimes \mathbf{m}.$

The mask is not computed from pooled statistics or an auxiliary attention block; instead, it is the raw reshaped class-specific weight vector of an Auxiliary Linear Classifier (ALC), selected by ground-truth class during training or by ALC argmax during inference (Zhang et al., 2022).

Gradient-Preserving Activation Scaling (GPAS) uses a different operator. Its forward behavior scales activations, but its backward behavior preserves gradients with respect to the scaled tensor:

$S_s(x)=x-(1-s)\cdot \mathrm{sg}(x),$

where $\mathrm{sg}(\cdot)$ is stop-gradient. Forward-wise, $S_s(x)=sx$; backward-wise, $\partial S_s/\partial x = I$. This removes the chain attenuation that would arise from repeated naive multiplications by $s_l<1$, while still reducing forward variance by a factor $s^2$ (Chen et al., 27 Jun 2025).

A3 introduces a bounded, differentiable, per-sample per-channel scale derived from activation magnitudes and a Gumbel-Softmax mask. If $z\in\mathbb{R}^{C\times H\times W}$, the channel-wise attenuation and amplification paths are

$\mathbf{m}\in\mathbb{R}^{C\times H\times W}$0

The same learned parameters support both modes; the sign flip distinguishes attenuation from amplification (Gonçalves et al., 26 Jun 2026).

In language-model steering, activation scaling is even more minimal. At an intervention point $\mathbf{m}\in\mathbb{R}^{C\times H\times W}$1, the activation vector $\mathbf{m}\in\mathbb{R}^{C\times H\times W}$2 is modified as

$\mathbf{m}\in\mathbb{R}^{C\times H\times W}$3

so the intervention changes only signed magnitude, not direction except when the factor becomes negative. Dynamic activation scalars further replace $\mathbf{m}\in\mathbb{R}^{C\times H\times W}$4 by a learned function of the normalized activation itself, enabling generalization across prompts of varying lengths (Stoehr et al., 2024).

The explicitly named ASGuard for tense-jailbreak mitigation uses head-local channel-wise scaling:

$\mathbf{m}\in\mathbb{R}^{C\times H\times W}$5

with $\mathbf{m}\in\mathbb{R}^{C\times H\times W}$6 learned only for a sparse set of vulnerable heads identified by circuit analysis. Because the scaling is channel-wise at the head output, it can be fused into the head’s output projection $\mathbf{m}\in\mathbb{R}^{C\times H\times W}$7, yielding zero inference-time overhead when deployed as fused scaling, and zero additional deployment components after preventative fine-tuning when the learned scaling is detached (Park et al., 30 Sep 2025).

3. CNN robustness instantiations

EWAS is the canonical fine-grained ASGuard for CNNs. It was introduced as a response to channel-level defenses such as Channel-wise Activation Suppressing and CIFS, which uniformly scale every element in a channel. EWAS argues that such uniform suppression is too coarse: localized robust evidence can coexist with adversarially harmful elements within the same channel. By selecting a class-aware element-wise mask in late layers, EWAS suppresses adversarially “negative” elements while preserving or promoting “positive” ones. The preferred insertion points are late: after the 15th layer in ResNet-18, after the 19th layer in WideResNet-28-10, and after the 25th layer in WideResNet-32-10. Training is joint adversarial training under $\mathbf{m}\in\mathbb{R}^{C\times H\times W}$8 perturbations with $\mathbf{m}\in\mathbb{R}^{C\times H\times W}$9, PGD-10 inner maximization, and an auxiliary ALC loss weighted by $\tilde{z}^l = z^l \otimes \mathbf{m}.$0 (Zhang et al., 2022).

Its reported gains are substantial. On CIFAR-10 with ResNet-18 under AT+EWAS, clean accuracy is 84.73%, FGSM 65.78%, PGD-20 64.84%, and C&W 82.35%. Against C&W, adversarial accuracy increases by 37.65 percentage points over the AT baseline, from 44.70% to 82.35%, leaving only a 2.38% gap between clean and C&W accuracy. On WideResNet-32-10, AT+EWAS reaches PGD-20 59.90% and C&W 73.01%, while MART+EWAS reaches PGD-20 65.40% and C&W 76.72%. On SVHN with ResNet-18, AT+EWAS yields clean 92.18%, FGSM 71.57%, PGD-20 59.01%, and C&W 69.67%, improving C&W by up to 19.54 percentage points over the AT baseline. The method is, however, placement-sensitive and $\tilde{z}^l = z^l \otimes \mathbf{m}.$1-sensitive; on CIFAR-10, $\tilde{z}^l = z^l \otimes \mathbf{m}.$2 is best, larger values degrade clean and robust accuracy, and $\tilde{z}^l = z^l \otimes \mathbf{m}.$3 fails to converge. It also trails CAS slightly under AutoAttack on CIFAR-10 ResNet-18, where EWAS reaches 63.22% robust accuracy versus 65.31% for CAS (Zhang et al., 2022).

A3 generalizes the CNN robustness use of activation scaling in another direction. Instead of only attenuating suspicious channels, it jointly learns attenuation and amplification. A3 computes a differentiable binary-like mask through Gumbel-Softmax, derives bounded scalars from activation magnitudes using a logarithmic formulation, and trains the network so that amplification deliberately degrades predictions while attenuation improves them. The training objective combines a main adversarial training loss with a hinge-style cross-entropy ranking loss and a contrastive logit loss; default hyperparameters are $\tilde{z}^l = z^l \otimes \mathbf{m}.$4, $\tilde{z}^l = z^l \otimes \mathbf{m}.$5, $\tilde{z}^l = z^l \otimes \mathbf{m}.$6, and $\tilde{z}^l = z^l \otimes \mathbf{m}.$7. At inference, only the attenuation path is used (Gonçalves et al., 26 Jun 2026).

The empirical pattern is consistent across backbones and training methods. On CIFAR-10 with ResNet-18, A3 improves AT from PGD-100 47.51 to 57.01 and AutoAttack from 44.28 to 47.28; under TRADES, PGD-100 improves from 50.48 to 54.26 and AutoAttack from 46.98 to 49.27; under MART, PGD-100 improves from 51.08 to 57.05. On WideResNet-34-10, A3 raises PGD-100 and AutoAttack under AT, TRADES, and MART, and on Tiny ImageNet it improves both AutoAttack and ensemble robustness. Its overhead is small: for ResNet-18, parameters increase by approximately +0.27M and FLOPs by +0.0021G; for WRN-34-10, parameters increase by approximately +0.41M and FLOPs by +0.0091G. A common misconception is that activation scaling in adversarial defense is equivalent to feature deletion. A3 explicitly contradicts that view: amplification is part of the training signal, and robustness is improved by learning from deliberately worsened activations as negative references (Gonçalves et al., 26 Jun 2026).

4. Transformer training dynamics and quantization guards

In Pre-LN transformers, GPAS treats activation scaling as an optimization and stability mechanism rather than a defense. The problem is exponential-like growth of activation variance across layers: LayerNorm normalizes the input to attention and MLP sublayers, but the residual path remains unnormalized, so the outputs of gain-amplifying sublayers accumulate onto increasingly large hidden states. The paper reports that this induces residual-path dominance, underutilization of deeper layers, slower convergence, and reduced parameter efficiency. GPAS inserts gradient-preserving scaling after attention and MLP outputs, before residual addition, with learnable gates parameterized as $\tilde{z}^l = z^l \otimes \mathbf{m}.$8. The stop-gradient construction preserves $\tilde{z}^l = z^l \otimes \mathbf{m}.$9, so gradients are not attenuated even when forward activations are scaled down (Chen et al., 27 Jun 2025).

The reported effects are both theoretical and empirical. Forward variance is reduced because $S_s(x)=x-(1-s)\cdot \mathrm{sg}(x),$0, while the backward chain no longer accumulates multiplicative $S_s(x)=x-(1-s)\cdot \mathrm{sg}(x),$1 factors. Empirically, GPAS compresses activation variance across depth by about 50%, halves peak variance, regularizes gradient norms, and improves pretraining perplexity across model sizes from 71M to 1B. At 1B, Pre-LN improves from 16.92 to 16.25, LayerNorm Scaling from 15.65 to 14.88, and Sandwich-LN at 350M from 21.54 to 19.79. DeepNorm at 1B diverges at approximately 1404 perplexity, whereas DeepNorm+ASGuard is stable at approximately 18.1. In a partial 7B run to 40K steps on 16B tokens, evaluation perplexity improves from 15.27 to 13.82. The paper also reports higher downstream 0-shot accuracy after SFT with frozen gates: +2.49% for Pre-LN, +1.18% for LNS, and +1.04% for Sandwich-LN (Chen et al., 27 Jun 2025).

QuBLAST applies activation scaling in an orthogonal setting: post-training quantization of LLMs. Its activation scaling strategy is a per-block, per-channel static map $S_s(x)=x-(1-s)\cdot \mathrm{sg}(x),$2 applied as

$S_s(x)=x-(1-s)\cdot \mathrm{sg}(x),$3

with the explicit goal of shrinking the magnitude of outlier channels so that low-bit activation quantization is feasible without complex transformations. This scaling is embedded in a two-stage pipeline. First, QuBLAST performs sensitivity analysis using cross-entropy loss changes when quantizing one block at a time, locking out block-bitwidth pairs whose loss increase exceeds a user-defined threshold $S_s(x)=x-(1-s)\cdot \mathrm{sg}(x),$4, tested at 2% and 5%. Second, it searches for a mixed-precision configuration with minimum memory subject to a global perplexity increase constraint of less than 5% (Wickramasinghe et al., 3 Jun 2026).

The quantitative role of activation scaling is unusually explicit. Across Qwen3-8B, Llama3-8B, Mistral v0.1-7B, and Falcon H1R-7B, QuBLAST reduces model size by 40.0% to 45.2% while maintaining perplexity increases within 5% on WikiText-2 and WikiText-103. With activation quantization enabled, memory savings reach 42.4% to 48.20% with perplexity increases within 2% on WikiText-2. The ablation shows that activation scaling is essential for W8A8: on Llama3-8B, perplexity is 5.60 with activation scaling and 608,566 without it; on Mistral v0.1-7B, 4.73 versus 24,423; on Qwen3-8B, 8.51 versus 1,372,634; on Falcon H1R-7B, 9.60 versus 723. The overhead is negligible: one elementwise multiply per block before quantization, plus storage of $S_s(x)=x-(1-s)\cdot \mathrm{sg}(x),$5, which is $S_s(x)=x-(1-s)\cdot \mathrm{sg}(x),$6 for per-channel scaling (Wickramasinghe et al., 3 Jun 2026).

5. Steering, circuit-level safety repair, and the explicit ASGuard method

Activation scaling also functions as an intervention language for interpretability. In activation steering, the objective is not robustness in the adversarial-training sense but a three-way trade-off among effectiveness, faithfulness, and minimality. The intervention is sparse and multiplicative: a few selected activation vectors are scaled by learned scalars so that a wrong token can be flipped with a correct token, the rest of the output distribution is minimally disturbed, and the intervention remains sparse under an $S_s(x)=x-(1-s)\cdot \mathrm{sg}(x),$7 penalty. The full objective combines an effectiveness term based on reversible hinge margins for $S_s(x)=x-(1-s)\cdot \mathrm{sg}(x),$8 and $S_s(x)=x-(1-s)\cdot \mathrm{sg}(x),$9, a faithfulness term based on KL divergence between intervened and base output distributions, and a minimality term $\mathrm{sg}(\cdot)$0. ActivScalar is reported to achieve comparable effectiveness-faithfulness trade-offs to steering vectors while being much more minimal: for Pythia-1.5B with $\mathrm{sg}(\cdot)$1, $\mathrm{sg}(\cdot)$2, and one site per layer, the parameter count is 912 scalars for ActivScalar versus 1,459,200 parameters for SteerVec (Stoehr et al., 2024).

Dynamic activation scalars extend the same idea to variable-length prompts by learning

$\mathrm{sg}(\cdot)$3

This yields on-the-fly scalars from activation directions rather than per-instance optimized values. On the CCC benchmark, the paper reports effectiveness values with margin $\mathrm{sg}(\cdot)$4: ActivScalar gives $\mathrm{sg}(\cdot)$5 on train, $\mathrm{sg}(\cdot)$6 on test, and $\mathrm{sg}(\cdot)$7 on different template; SteerVec gives $\mathrm{sg}(\cdot)$8, $\mathrm{sg}(\cdot)$9, and $S_s(x)=sx$0; dynamic scalars give $S_s(x)=sx$1, $S_s(x)=sx$2, and $S_s(x)=sx$3. The reported interpretation is that dynamic scalars generalize better to template shift than fixed-position scalars, while maintaining the direction-preserving nature of the intervention (Stoehr et al., 2024).

The explicitly named ASGuard in LLM safety applies activation scaling to a narrower target: a targeted jailbreak in which harmful present-tense requests are refused but past-tense reformulations are answered compliantly. Its pipeline has three stages. First, edge attribution patching with integrated gradients constructs faithful circuits for successful and unsuccessful tense jailbreaks, and the heads appearing only in successful False-to-True circuits are labeled tense-vulnerable. Second, only those heads receive learned channel-wise scaling vectors $S_s(x)=sx$4, trained to maximize refusal on harmful past-tense prompts while the base model weights remain frozen. Third, the learned scaling is used in preventative fine-tuning so that the base model learns a more robust refusal pathway; after convergence, the scaling is detached, leaving no additional inference-time component (Park et al., 30 Sep 2025).

The empirical results are reported on Llama-3.1-8B-Instruct, Qwen-2.5-7B-Instruct, and Gemma-2-9B-it. Baseline attack success rates are 42%, 51%, and 38%, respectively. Head ablation reduces ASR by 4% to 13%, whereas random-head ablation reduces it by only approximately 1% to 2%, supporting causal relevance of the identified heads. “Only Scaling” reduces ASR to 13% on Llama, 37% on Qwen, and 26% on Gemma, but Gemma shows utility risk. Full ASGuard, meaning Scaling + PFT, reduces ASR to 8% on Llama, 8% on Qwen, and 19% on Gemma while maintaining OR-Bench-Toxic, OR-Bench-Hard-1K, and MMLU scores consistent with a Pareto-optimal safety-utility balance. For Llama-3.1-8B, the final metrics are ASR 8%, Toxic 96.4, Hard 66.8, MMLU 68.2, R-Score 71.8, and Overall 52.9; for Qwen-2.5-7B, ASR 8%, Toxic 98.0, Hard 70.5, MMLU 74.0, R-Score 74.6, and Overall 58.8; for Gemma-2-9B, ASR 19%, Toxic 99.0, Hard 88.0, MMLU 72.2, R-Score 70.1, and Overall 44.6 (Park et al., 30 Sep 2025).

6. Trade-offs, misconceptions, and open problems

Several cross-paper regularities emerge. First, activation scaling is not equivalent to uniform suppression. EWAS uses raw class-conditioned element-wise masks and explicitly does not try to make adversarial activations resemble natural ones; instead, it learns masks that produce distinct distributions yet improve robust accuracy. A3 uses both attenuation and amplification, so the “guard” includes controlled worsening during training. GPAS scales activations down while preserving gradients, making it fundamentally different from naive residual scaling. QuBLAST uses static, calibration-derived per-channel maps rather than learned online gates. The steering work uses sparse scalar interventions on selected residual computations. The targeted jailbreak ASGuard applies scaling only to a small set of causally identified attention heads, then removes the scaling after preventative fine-tuning (Zhang et al., 2022, Chen et al., 27 Jun 2025, Wickramasinghe et al., 3 Jun 2026, Gonçalves et al., 26 Jun 2026, Stoehr et al., 2024, Park et al., 30 Sep 2025).

Second, most ASGuard variants are highly placement-sensitive or hyperparameter-sensitive. EWAS performs best in late CNN layers and does not benefit from multiple insertions beyond the best single position. A3 is stronger when inserted in deeper residual blocks and can over-attenuate when stacked. GPAS recommends insertion after each sublayer output and before residual addition; other placements are less effective. QuBLAST depends on accurate sensitivity profiling and robust calibration statistics. The LLM jailbreak ASGuard depends on whether the vulnerability is localizable to a sparse circuit; the paper notes that more entangled architectures could complicate transfer. In several cases, incorrect scaling intensity degrades either robustness or utility: large $S_s(x)=sx$5 harms EWAS convergence, removing stop-gradient negates GPAS gains, and naive scaling alone can reduce capability in Gemma before PFT (Zhang et al., 2022, Chen et al., 27 Jun 2025, Wickramasinghe et al., 3 Jun 2026, Park et al., 30 Sep 2025).

Third, overhead is usually modest but not uniform. EWAS adds $S_s(x)=sx$6 ALC parameters per insertion, on the order of $S_s(x)=sx$7 for typical CIFAR-10 late-layer shapes. A3 adds a single $S_s(x)=sx$8 projection and very small FLOP increases. QuBLAST’s scaling adds only one per-block multiply and minimal storage. GPAS uses learnable gates and a stop-gradient operator rather than an auxiliary network. The explicit jailbreak ASGuard ultimately incurs zero inference-time overhead after the scaling is detached post-PFT, and even before detachment it can fuse scaling into head output projections (Zhang et al., 2022, Chen et al., 27 Jun 2025, Wickramasinghe et al., 3 Jun 2026, Gonçalves et al., 26 Jun 2026, Park et al., 30 Sep 2025).

Several limitations remain open. EWAS does not report robustness to common corruptions or transfer attacks beyond AutoAttack. GPAS results are limited by compute at very large scales, with only partial 7B experiments reported. QuBLAST relies on calibration data and static scaling maps, so distribution shift remains a natural concern. The activation-steering results are centered on synthetic tasks and medium-scale Pythia models. The targeted ASGuard method addresses a specific semantic perturbation—tense jailbreaking—and requires fresh mechanistic analysis for other jailbreak families. This suggests that ASGuard is best regarded as a general intervention paradigm whose success depends on whether the failure mode can be localized to a stable activation structure and whether multiplicative recalibration is sufficient to alter the downstream computation without unacceptable collateral effects (Stoehr et al., 2024, Park et al., 30 Sep 2025).