Quantization as a Malicious Task: Removing Quantization-Conditioned Backdoors via Task Arithmetic

Abstract: Model quantization is widely adopted to reduce memory usage and inference cost when deploying deep neural networks on resource-constrained devices. However, recent studies have revealed a new security threat known as Quantization-Conditioned Backdoors (QCBs), where a model behaves normally in full precision but activates malicious behavior only after quantization. Existing defenses typically modify quantization procedures or correct activation statistics, often introducing additional computational overhead or relying on specific quantization settings. Here, we present QVec, a parameter-space perspective for defending against QCBs. We observe that the weight difference between a full-precision model and its quantized counterpart encodes a structured behavioral shift, which can be interpreted as a malicious task vector rather than random quantization noise. Based on this insight, QVec counteracts this malicious direction through controlled parameter correction prior to deployment. QVec requires no retraining, no trigger samples, and only a single quantization pass to estimate the parameter shift, together with a lightweight hyperparameter search. Extensive experiments across image classification benchmarks and multiple LLM attack scenarios demonstrate that QVec consistently suppresses backdoor activation while preserving clean performance.

• The paper introduces QVEC, a novel defense that neutralizes quantization-conditioned backdoors by correcting structured parameter shifts.
• The methodology leverages task arithmetic to compute a correction vector, enabling quantization-agnostic defense with minimal impact on clean performance.
• Empirical results demonstrate near-zero attack success rates and maintained clean accuracy within 1-2% across various architectures.

Quantization-Conditioned Backdoors: Parameter-Space Correction via Task Arithmetic

Background and Motivation

Model quantization is a widely adopted technique for compressing deep neural networks, enabling deployment on resource-constrained devices by reducing memory and inference costs. Historically, quantization's impact on performance and robustness has been largely characterized as random rounding noise, with adverse effects mainly attributed to loss of representational fidelity. However, the emergence of quantization-conditioned backdoors (QCBs) has expanded the threat landscape: models behave benignly in full precision but activate a backdoor only post-quantization. This paradigm subverts traditional backdoor defenses, which typically operate in full-precision regimes and rely on observable anomalous behaviors prior to deployment.

Prior solutions for QCBs—such as EFRAP and LAC/PDA—focus on modifying quantization operations or correcting activation drift, often at the cost of increased computational burden and with dependency on quantization configuration. The paper "Quantization as a Malicious Task: Removing Quantization-Conditioned Backdoors via Task Arithmetic" (2606.20254) introduces QVEC, a quantization-agnostic defense that leverages insights from task arithmetic and parameter-space edits to suppress QCBs efficiently.

Theoretical Perspective: Task Vectors in Quantization

The authors observe that the quantization-induced parameter shift $d = Q(W) - W$ behaves not as random noise, but as a structured task vector mapping benign behavior to quantization-triggered malicious behavior. This aligns with the principle that neural network modifications—whether via fine-tuning or merging—can be abstracted as vectors in parameter space encoding semantic task transformations. Interpolating model weights along this vector direction yields a continuous transition between tasks, a phenomenon previously exploited for behavioral editing and interference mitigation.

Empirical and gradient-based analyses show significant alignment between quantization-induced shifts and the backdoor objective, both in model behavior and sensitivity of loss gradients. Statistical evidence distinguishes the shift distribution in clean models from that in QCB-attacked counterparts, confirming its task-like nature and structured alignment.

QVEC: Methodology

QVEC operates by neutralizing the malicious task vector prior to quantization:

1. Parameter-Space Correction: For a full-precision model $W$, QVEC computes the quantization-induced shift $d = Q(W) - W$ and constructs a corrected model $W' = W - \alpha d$, where $\alpha$ is a scalar hyperparameter controlling trade-off between clean accuracy and backdoor suppression.
2. Hyperparameter Search: $\alpha$ is selected via a lightweight validation-based search, requiring only one forward quantization pass and no retraining.
3. Deployment: The final quantized model $Q(W')$ is deployed, exhibiting suppressed backdoor activation with preserved clean performance.

This approach is agnostic to quantization granularity (per-tensor, per-channel, etc.) and does not require trigger samples or access to clean reference data, making it compatible with practical supply-chain scenarios.

Experimental Validation

Extensive evaluations cover both vision architectures (ResNet-18, VGG13, MobileNetV2) and LLMs (Gemma-2B, StarCoder-1B), spanning CIFAR-10 and Tiny-ImageNet datasets as well as diverse malicious behaviors including vulnerable code generation, content injection, and over-refusal.

Key empirical findings:

• Suppression of Backdoor Activation: QVEC reduces Attack Success Rate (ASR) to near zero across all tested architectures and quantization regimes (8-bit, 4-bit), outperforming both specialized defenses and Gaussian noise injection.
• Retention of Clean Performance: Clean accuracy (CA) remains within a minimal tolerance of the original model (within 1-2% in most cases), emphasizing the efficacy of directionally targeted parameter correction over random perturbations.
• Generalization to LLMs: QVEC attenuates quantization-triggered backdoors in LLMs, restoring security metrics (keyword occurrence rates, code security rates, refusal rates) to benign baselines while maintaining task-specific benchmarks (MMLU, TruthfulQA, HumanEval).
• Robustness Against Adaptive Attacks: Even when adaptive attack strategies introduce extreme outliers to distort quantization intervals, QVEC remains effective, albeit at the cost of some degradation in clean accuracy for the attacker. The defense's efficacy persists due to its reliance on observable structured shifts.

Layer-Localized Analysis

Layer-wise ablations reveal that QCB activation is distributed across the network, with progressively stronger effects in middle-to-late blocks (especially MLP components in transformers), consistent with the functional role of these layers in decision boundary formation and propagation of trigger conditions. Corrections targeted at the quantization-induced shift in these layers yield maximal suppression, further validating the structured, task-vector interpretation.

Implications and Future Directions

Theoretically, this work reframes quantization as not merely a process of numerical compression, but as a potential catalyst for latent behavioral transformations in neural networks, contingent on alignment between discretization-induced shifts and adversarial training objectives. Practically, parameter-space corrections prior to quantization offer a lightweight, configuration-agnostic, and data-efficient defense against a class of supply-chain attacks that elude conventional detection and purification methodologies.

Potential future developments include:

• Extending the framework to stochastic quantization and mixed-precision regimes.
• Integrating parameter-space corrections into automated deployment pipelines for real-time supply-chain hardening.
• Investigating compound task-vector corrections for simultaneous mitigation of conventional and quantization-conditioned backdoors.
• Formalizing theoretical guarantees on task-vector correction efficacy under nonlinear network architectures and adversarial adaptability.

Conclusion

QVEC establishes a principled, robust, and practical defense against quantization-conditioned backdoors by recognizing and counteracting structured behavioral shifts in parameter space. The work underscores the significance of deployment-time transformations and offers a methodological blueprint for supply-chain model hardening, with broad relevance across vision and LLMs. Empirical results demonstrate both the superiority and generality of parameter-space correction over traditional noise-based or quantization-specific defenses, suggesting a shift in focus from activation-space analysis to parameter-space arithmetic for next-generation neural network security.