--- title: Performance under No Attack (PNA) Baselines url: https://www.emergentmind.com/topics/performance-under-no-attack-pna type: topic --- # Performance under No Attack (PNA) Baselines Performance under No Attack (PNA) quantifies the behavior of a classifier, detector, or security enhancement in the absence of adversarial manipulation or malicious activity. It serves as the canonical baseline for any empirical investigation of robustness in adversarial machine learning, intrusion detection, runtime attestation, and related domains. PNA isolates the intrinsic generalization performance or operational overhead of a system, positioning it as the reference point against which the severity and trade-offs associated with adversarial threats are measured. ## 1. Formal Definition and Theoretical Foundations PNA is defined as the evaluation of a system’s performance when both training and testing data are drawn from the original, stationary data distribution $p_0(X,Y)$, without adversarial modification. Let $A$ be a Boolean variable indicating if a sample has been tampered with. For PNA, the adversary does not intervene ($p(A\!=\!T|Y)=0$ for all $Y$), implying: - Training and test label marginals revert to $p_{tr}(Y) = p_{ts}(Y) = p_0(Y)$. - No adversarial conditional distributions $p_{tr}(X|Y,A=T)$ or $p_{ts}(X|Y,A=T)$ are considered. - All samples satisfy $p_{tr}(X|Y,A=F)=p_{ts}(X|Y,A=F)=p_0(X|Y)$ [1709.00609]. Metrics employed for PNA include error rate (ERR), classification accuracy (ACC), and Receiver Operating Characteristic Area Under the Curve at low false-positive rates ($\mathrm{AUC}_{10\%}$), among others. In biometrics, the Genuine Acceptance Rate (GAR) versus False Accept Rate (FAR) at specified thresholds constitutes the PNA evaluation [1709.00609]. ## 2. Evaluation Protocols and Mathematical Formulation To assess PNA, established resampling protocols such as $k$-fold cross-validation or bootstrap sampling generate pairs $(\mathcal{D}_{TR}^i, \mathcal{D}_{TS}^i)$ from $p_0(X,Y)$. No attack data are injected. Model selection hyperparameters (e.g., SVM $C$, kernel $\gamma$, feature set size) are optimized via the chosen metric computed on $\mathcal{D}_{TR}^i$, and the system’s final PNA is the mean evaluation metric across all folds [1709.00609]. For a model with parameters $\theta$, a loss function $L(\cdot)$, and folds $i = 1 \ldots k$: \[ \mathrm{PNA}^i = L\bigl[f_\theta(X), Y\bigr] \text{ for } (X,Y) \sim p_0 \] \[ \mathrm{PNA} = \frac{1}{k} \sum_{i=1}^k \mathrm{PNA}^i \] The resulting value represents the performance in the hypothetical, attack-free operational regime. ## 3. PNA in Detector-Augmented Classification and Security Architectures For systems that cascade a detector with the main decision function, such as DNN classifiers with adversarial anomaly detectors, PNA must be evaluated conditionally on the detector’s false positive rate (FPR). Here, every input $x$ is first passed to the detector; only if $x$ is not flagged as an attack is the DNN classifier invoked. The conditional “clean-data classification accuracy” is then: \[ \mathrm{PNA}(\alpha) = \text{classification accuracy on clean samples with } D_{KL} \leq \tau \] where the detector threshold $\tau$ is set such that $\mathrm{FPR}(\tau)=\alpha$. Experimental results indicate that reducing FPR as low as $5\%$ leads to a marginal drop in PNA (≤ 0.1 percentage points), confirming that well-calibrated detection incurs negligible trade-off in attack-free accuracy [1712.06646]. ## 4. Empirical PNA: Representative Results Across Domains Benchmark results illustrate the typical range and implications of PNA as a baseline: | System/Domain | Metric | PNA Value(s) | Source | |-------------------------|-----------------------|-------------------------------------|---------------| | MNIST, LeNet–5 | Test Accuracy | 98.10% | [1712.06646] | | CIFAR-10, 16-layer DNN | Test Accuracy | 89.47% | [1712.06646] | | Spam Filtering, LR/SVM | $\mathrm{AUC}_{10\%}$ | 0.056–0.085 (LR), 0.057–0.082 (SVM) | [1709.00609] | | Fingerprint+Face Fusion | GAR @ FAR=$10^{-3}$ | 0.90 | [1709.00609] | | Microcontroller Attest. | Overhead (Surf, BEEBS)| 4.7% geometric mean | [2512.12729] | | Microcontroller Attest. | Overhead (CoreMark) | 1.1% geometric mean | [2512.12729] | Additional scenarios include clean-accuracy under noise-augmented training ($\sim$97.2–97.5% for MNIST) and parameterized feature-selection or kernel settings in traditional pattern classification [1709.00609], confirming that PNA is protocol-invariant but application-specific. ## 5. PNA and the Adversarial Robustness Trade-off PNA anchors empirical robustness studies by quantifying the initial, “ideal” promise of a system before exposure to adversarial threat models. In security evaluation frameworks, degradation from PNA to performance-under-attack quantifies the system’s specific vulnerability to simulated attacks. For example, classifiers with indistinguishable PNA values may display divergent robustness profiles upon manipulation of feature sets or training data: a system with $\mathrm{AUC}_{10\%}=0.085$ in spam filtering may degrade gracefully, while others with similar PNA collapse under word obfuscation [1709.00609]. This differential signals that naive model selection based on PNA alone may be insufficient for adversarial resilience. In adversarial training settings, traditional methods (e.g., PGD-AT, TRADES) routinely sacrifice several percent of PNA compared to standard models, exposing a tension between robustness and benign-sample generalization [2409.11754]. Null-space constrained adversarial training (NPDA, NPGD) achieves near-zero deterioration in PNA—recovering clean-error rates within $\lesssim1\%$ of the base model—substantially outperforming unconstrained adversarial methods in benign scenarios [2409.11754]. ## 6. Interpretation in Performance-Overhead and Embedded Systems In embedded systems security, PNA is realized as the run-time and energy overhead of enabling defense mechanisms relative to the uncompromised baseline. For PACBTI-based runtime attestation, PNA corresponds to the geometric-mean increase in execution time and power consumption for application workloads when the protection is active, but no attacks are present. Observed overheads remain within 1 – 4.7% for industry-standard benchmarks, with function-call-heavy code seeing up to 28% overhead per test [2512.12729]. These results confirm that robust attestation can be deployed in real-time, resource-constrained environments without violating system specification in the absence of attack. ## 7. Recommendations and Limitations PNA serves as both a necessary reference and a limiting criterion. Sound adversarial evaluation requires first establishing high PNA for the application, then tracking its degradation across a spectrum of attack capabilities. However, models that optimize solely for PNA may have significantly inferior robustness. Conversely, designs with modestly sacrificed PNA may be far more resilient under attack. Model selection, defense layering, and empirical “what-if” analysis should therefore use PNA as a baseline but prioritize robustness curves for operational deployment [1709.00609]. In deep learning, recent innovations such as null-space projected adversarial training demonstrate that zero-deterioration PNA is feasible under structural constraints, but depend on access to a high-accuracy reference model and sufficient latent dimensionality [2409.11754]. A plausible implication is that the utility of any security intervention or adversarial mitigation must ultimately be measured by its effect on PNA as well as by the delta between PNA and performance under attack. Overly conservative defenses or expensive runtime checks may degrade PNA, negating their security benefits when real-world threat likelihood is low. Hence, reporting PNA and its associated implementation costs establishes the operational relevance of security enhancements across domains. Source: https://www.emergentmind.com/topics/performance-under-no-attack-pna