---
title: 'WAVS: Adversarial Vulnerability Metric'
url: https://www.emergentmind.com/topics/weighted-adversarial-vulnerability-score-wavs
type: topic
---

# WAVS: Adversarial Vulnerability Metric

The Weighted Adversarial Vulnerability Score (WAVS) is a formal metric family that quantifies the susceptibility of machine learning systems—including deep neural networks, vision-language models, and large language models—to adversarial attacks. By aggregating per-instance vulnerability assessments with explicit normalization and domain-relevant weighting, WAVS enables robust, interpretable evaluation of both individual model weaknesses and systemic threats arising from diverse adversarial strategies. The metric has emerged in multiple research areas with instantiations tailored for vision, language, and multimodal models, providing a unified scoring paradigm for both defense benchmarking and red teaming.

## 1. Core Mathematical Framework

WAVS centers on the aggregation of per-instance or per-scenario adversarial vulnerability scores into a scalar that summarizes robustness. While specific forms vary, the essential components are:

- **Instance-level vulnerability**: A function capturing how susceptible a given input (or model decision) is to adversarial perturbation, based on margin, loss, or attack-induced score change.
- **Normalization**: Transformation of component scores to common scales, typically via min–max normalization or division by a base value.
- **Weighting and aggregation**: Convex combination or arithmetic mean pooling of vulnerability components, yielding a single dataset-level or scenario-level measure.

For example, in adversarially trained vision models, per-example WAVS is defined as the rescaled perturbation radius:
$$
\text{WAVS}_i = \Delta_i = \varepsilon \cdot \exp(\alpha v_i)
$$
where $v_i$ is a margin- or deviation-based vulnerability metric, $\varepsilon$ is the fixed base radius, and $\alpha > 0$ modulates the scaling. For dataset-level evaluation:
$$
\text{WAVS}_{\text{dataset}} = \frac{1}{n} \sum_{i=1}^n \Delta_i
$$
This directly reflects the mean perturbation budget encountered in adversarial training [2403.04070].

In evaluation of LLM-based scientific assessment, WAVS takes the form:
$$
\text{WAVS}_M = \frac{1}{|P|\,|A|} \sum_{p\in P} \sum_{a\in A} V(p,a;M)
$$
with
$$
V(p,a;M) = w_S N_S(\Delta S) + w_F N_F(I_{\text{flip}}) + w_R N_R(\omega_{\text{severity}})
$$
where $N_{(\cdot)}$ denotes normalization, $w_S+w_F+w_R=1$, and the score integrates score inflation, semantic flip, and risk weights [2512.10449].

## 2. Notable Instantiations

WAVS is instantiated with domain-adapted vulnerability measures and aggregation procedures, the most prominent being:

| WAVS Variant      | Domain                 | Core Vulnerability Measure(s)               |
|-------------------|-----------------------|---------------------------------------------|
| MWPB/SDWPB [2403.04070] | Vision (DNNs)          | Margin or logit stddev (per-sample)         |
| Non-uniform Minimax [2010.12989] | Vision (DNNs)          | Exponential of negative adversarial margin  |
| VLM Safety [2502.16361]      | Vision-Language          | Accuracy drops under random/FSMG noise      |
| Peer Review LLM [2512.10449]  | LLMs (review systems)   | Score inflation, flip severity, risk label  |

In all settings, WAVS provides a mechanism for principled risk-sensitive robustness diagnostics, aligning with practical attack models and domain-specific threat priorities.

## 3. Methodologies for Computing and Aggregating WAVS

**Vision models (AT, TRADES, MART):** Per-sample vulnerability is determined via either the multi-class margin
$$
d_m(x_i, y_i; \theta) = f_\theta(x_i)_{y_i} - \max_{k\neq y_i} f_\theta(x_i)_k
$$
or the standard deviation of logits about the correct class
$$
d_{\text{std}}(x_i, y_i; \theta) = \sqrt{ \frac{1}{|C|} \sum_{k=1}^C (f_\theta(x_i)_k - f_\theta(x_i)_{y_i})^2 }
$$
with per-sample perturbation budgets:
$$
\Delta_i = \varepsilon \cdot \exp(\alpha v_i)
$$
Dataset-level WAVS is computed via the mean of $\Delta_i$ over all $i$ [2403.04070].

**Weighted Minimax (non-uniform attacks):** The adversarial risk is weighted by
$$
s_i = \exp(-\alpha \, \mathrm{margin}(f_\theta, x_i', y_i))
$$
and can be normalized over test points:
$$
\widetilde{s}_i = \frac{s_i}{\sum_{j=1}^{N_{\text{test}}} s_j}
$$
yielding a per-example selection probability interpreted as WAVS [2010.12989].

**Vision-language models:** WAVS is a convex combination of the relative drop in accuracy under composite random noise versus under FGSM:
$$
\text{WAVS} = w_1 \cdot \text{Noise Impact Score} + w_2 \cdot \text{FGSM Impact Score}
$$
with each score defined as a relative decrease from the clean baseline [2502.16361].

**LLM-based review systems:** WAVS for each (paper, attack) is the normalized weighted sum of score delta, semantic flip severity, and risk alignment, then aggregated by average over all instances [2512.10449].

## 4. Empirical Insights and Evaluations

**Margin/Stddev-Weighted Adversarial Training (MWPB/SDWPB):** Integrating WAVS-driven per-sample budgets in adversarial training led to consistent improvements in robust accuracy across CIFAR-10, SVHN, TinyImageNet, and both white-box and black-box threats (PGD-20, CW, AA, Square, SPSA). For example, on ResNet-18/CIFAR-10:

- Standard AT: PGD-20 accuracy 52.72%
- MWPB-AT: PGD-20 accuracy 56.25% (+3.53)
- SDWPB-AT: PGD-20 accuracy 56.69% (+3.97)

WAVS directly summarized the mean perturbation intensity encountered, providing both a training control signal and an evaluation statistic [2403.04070].

**Non-uniform attack resistance:** Weighted minimax (WAVS-aligned) adversarial training preserved or slightly improved uniform robust accuracy while raising adversarial accuracy under non-uniform attacks by several points. For CIFAR-10 (20-step PGD, $\epsilon=0.031$):

| Defense      | $\mathcal{A}_\text{rob}$ | $\mathcal{A}_\text{sa}$ | $\mathcal{A}_\text{tr}$ |
|--------------|--------------------------|--------------------------|--------------------------|
| PGD (base)   | 49.29%                   | 11.66%                   | 9.72%                    |
| PGD+WAVS     | 49.53%                   | 13.19%                   | 10.81%                   |

By learning to emphasize highly vulnerable examples (larger $s_i$), the defense adapts to structured, threat-adaptive adversaries [2010.12989].

**Vision-language safety:** On CLIP/Caltech-256, WAVS captured the stark contrast between natural and adversarial noise, quantifying both information-theoretic and gradient-based sensitivity in a single interpretable value. With baseline accuracy of 95%, composite random noise dropped accuracy by ≈29 points, FGSM by ≈90, yielding a balanced WAVS of ≈59.8 (for $w_1=w_2=0.5$) [2502.16361].

**LLM-based review vulnerability:** WAVS separated superficial score-nudging from deep semantic flips and risk, revealing robust differences between open-source and proprietary models, and across attack strategies. Top strategies on open models achieved WAVS ≈0.81, while GPT-5 maintained WAVS near zero, demonstrating strong defense alignment [2512.10449].

## 5. Theoretical Motivation and Interpretability

WAVS addresses several deficiencies in classical adversarial vulnerability metrics:

- **Non-uniform adversarial focus:** WAVS quantifies not just global robustness, but the distribution of vulnerability across instances/scenarios, reflecting adversary adaptivity.
- **Integration of multiple risk dimensions:** Especially in LLM settings, WAVS fuses score inflation, flip risk, and ground-truth severity, yielding actionable decompositions.
- **Hyperparameterized focus:** The weighting scheme allows tuning according to operational risk—natural perturbations versus targeted attacks [2502.16361, 2512.10449].
- **Direct training and diagnostic utility:** In AT, WAVS surfaces as both a loss-reweighting (during training) and an evaluative summary (after training) [2403.04070, 2010.12989].

## 6. Practical Considerations and Limitations

**Assumptions and normalization:** WAVS calculations assume access to per-instance logits, risk labels, or adversarial deltas as appropriate. Normalization choices (e.g., min–max) can be sensitive to outliers, and must be consistent across evaluations for fair comparison [2512.10449]. In peer review settings, mislabeling of paper risk can distort WAVS.

**Domain-specific weighting:** The parameters governing the mix of components (e.g., $w_S$, $w_F$, $w_R$ for LLMs; $w_1$, $w_2$ for VLMs) are currently heuristic but allow for alignment with domain risk profiles. Sensitivity analyses indicate that stable comparative rankings are obtained if critical components (flip severity/risk) receive at least 60% cumulative weight [2512.10449].

**Threat space coverage:** For an adversarially comprehensive WAVS, representative attack and perturbation strategies must be included. The metric does not account for "backfire" robustness (cases where attacks inadvertently increase accuracy), which is a limitation in its current forms [2512.10449].

## 7. Open Questions and Future Directions

Open questions regarding WAVS include:

- **Optimal weighting for novel domains:** The heuristic assignment of component weights invites further theoretical and empirical justification tailored to new application areas (e.g., grant review, medical triage).
- **Extension to multimodal and interactive systems:** Initial WAVS variants exist for images, text, and multimodal I/O, but rigorous frameworks for video, tabular, and interactive decision-making are a prospective research direction [2512.10449].
- **Defensive strategies for WAVS minimization:** Identifying adversarial training regimes, regularizers, or alignment objectives that directly minimize WAVS, particularly in the context of dynamic, adaptive attackers, remains an open engineering and theoretical problem.

WAVS provides a principled, adaptable, and interpretable lens on adversarial vulnerability, enabling rigorous cross-domain robustness evaluation and targeted defense development. Its practical deployment requires careful threat modeling, control of normalization and weighting, and continued calibration against evolving adversarial paradigms.

Source: https://www.emergentmind.com/topics/weighted-adversarial-vulnerability-score-wavs