Wide-Spectrum Security Evaluation

• Wide-spectrum security evaluation methodology is a comprehensive approach that spans multiple threat sources, system layers, lifecycle phases, and evidence types to ensure comparability across heterogeneous systems.
• It integrates qualitative and quantitative metrics, combining models like attack trees, probabilistic risk, and evidence aggregation to link security requirements, controls, and observed behaviors.
• The methodology supports lifecycle integration and domain-specific adjustments, enabling continuous assurance and adaptive security assessments across ICT, IoT, and cyber-physical systems.

Wide-spectrum security evaluation methodology denotes a security assessment approach that spans multiple threat sources, system layers, lifecycle phases, and evidence types, while preserving comparability across heterogeneous systems. In the literature, this notion is associated with methodologies that connect security requirements, implemented controls, observed behavior, and risk; combine qualitative and quantitative evaluation; and aggregate evidence from components to systems rather than treating threat modeling, testing, and assurance as isolated activities (Shukla et al., 2021, Buriachok et al., 2019). A recurring conclusion is that no single metric, nor any set of metrics, can measure “security as a whole,” so wide-spectrum evaluation is typically organized as a structured combination of taxonomies, attack models, metrics, assurance arguments, and process controls adapted to the domain under study (Longueira-Romero et al., 2021).

1. Conceptual foundations and scope

A central conceptual anchor is system security assurance, defined as “the confidence that a system meets its security requirements and is resilient against security vulnerabilities and failures” (Shukla et al., 2021). In this view, evaluation is not merely defect counting or compliance checking; it is the production of evidence-backed confidence that security features, practices, procedures, and architecture mediate and enforce policy under realistic attack and failure conditions. The literature therefore treats a wide-spectrum methodology as one that explicitly links security requirements, security metrics, system and environment models, and assurance methods across governance, construction, deployment, and operation (Shukla et al., 2021).

The scope of such methodologies is explicitly cross-domain. The review literature spans ICT systems and cyber-physical systems, including telecommunications, cloud and multi-cloud, IoT and Industrial IoT, self-adaptive systems, industrial control and manufacturing, railway automation, e-government, WLANs, and access-control enforcement in applications (Shukla et al., 2021). More specialized works generalize from a nominal application domain to broader heterogeneous information systems, including distributed wireless systems, computer and telecommunication systems, information networks at commercial enterprises, and critical infrastructure facilities (Buriachok et al., 2019). This breadth is one reason the term “wide-spectrum” is used: the same methodological principles are expected to accommodate different architectures, threat environments, and assurance obligations.

The security properties covered likewise extend beyond the classical confidentiality–integrity–availability triad. The assurance literature explicitly includes authenticity, accountability, privacy, non-repudiation, authorization, conformity, utility, and possession (Shukla et al., 2021). Embedded-systems work aligned to IEC 62443 adds access control, use control, restrict data flow, and timely response to events (Longueira-Romero et al., 2021). This suggests that wide-spectrum evaluation is not defined by a single canonical property set; rather, it is defined by the ability to map multiple security objectives to appropriate metrics and evidence.

A common misconception is that wide-spectrum evaluation implies a universal scalar “security level.” The metrics literature argues the opposite: security is better discussed in terms of metrics than a single scalar “level,” and no universal security metric exists (Shukla et al., 2021, Longueira-Romero et al., 2021). Where scalar summaries are used, they are typically normalized aggregates derived from richer underlying structures rather than substitutes for them.

2. Structural decomposition of systems under evaluation

Wide-spectrum methodologies usually begin by decomposing the target into evaluable units. One recurring pattern is a taxonomy of assets, security dimensions, and metric properties. For embedded systems, the asset classes are hardware, software, data, services, crypto keys, and communications; the security dimensions are confidentiality, integrity, availability, access control, use control, restrict data flow, and timely response to events; and each metric must specify automation, scale, measurement type, and computing cost (Longueira-Romero et al., 2021). This decomposition turns a vague notion of “system security” into a matrix of asset–property pairs.

A second pattern is layering. The Robot Security Framework uses four layers—Physical, Network, Firmware, and Application—and argues that internal and external communication security should be treated as equally relevant (Vilches et al., 2018). For complete quantum communication systems, a seven-layer implementation hierarchy is defined: Q1 optics, Q2 analog electronics interface, Q3 driver and calibration algorithms, Q4 operation cycle, Q5 post-processing, Q6 application interface, and Q7 installation and maintenance (Sajeed et al., 2019). These layers are ordered by information flow and control hierarchy rather than by hardware packaging alone.

A third pattern is assurance decomposition by process. Governance, construction, and deployment are separated in the assurance literature; requirements, design/modeling, verification, and monitoring/management are treated as distinct but connected assurance loci (Shukla et al., 2021). This process-oriented decomposition is orthogonal to technical layering and is often used in parallel with it.

Decomposition	Elements	Source
Asset taxonomy	hardware, software, data, services, crypto keys, communications	(Longueira-Romero et al., 2021)
Robotics layers	Physical, Network, Firmware, Application	(Vilches et al., 2018)
Quantum communication sub-layers	Q1 optics to Q7 installation and maintenance	(Sajeed et al., 2019)

These decompositions are not interchangeable, but they are compatible. A plausible implication is that wide-spectrum methodology is best understood as a family of aligned decompositions: assets and properties for metric selection, layers for attack-surface coverage, and lifecycle phases for evidence collection.

3. Threat, attack, and scenario modeling

Threat elicitation is only one part of wide-spectrum evaluation, but it remains foundational. One line of work formalizes attacks through a “zombie” model of multi-stage, malware-driven compromise with victim study, attack, and remove-traces phases, and quantifies attack efficiency as

$c = n \cdot s \cdot \Delta t \cdot C,$

where $n$ is the number of potential servers, $s$ is the number of computers that work directly with one server, $\Delta t$ is the time the system remains in the “zombie” state, and $C$ is the cost of the attack (Buriachok et al., 2019). The same paper also introduces a formal attack model with analyzed objects, attack purposes, scenarios, options, exploit sets, and attack trees, explicitly linking vulnerability databases and expert knowledge to attack-option generation (Buriachok et al., 2019).

A broader integration of threat modeling appears in STRIDE-centric evaluation. There, STRIDE is not used only for early elicitation, but as the unifying classification across threat modeling, attack scenario analysis, risk analysis, and countermeasure recommendation (Jawad et al., 24 Mar 2025). Threats are first generated from data-flow diagrams, then grouped into STRIDE-based attack-tree subgoals, then turned into risks in NASA’s Defect Detection and Prevention matrices, and finally mapped to countermeasure sets targeted at the corresponding security objective such as integrity or availability (Jawad et al., 24 Mar 2025). This makes the threat taxonomy persistent across the full evaluation chain.

Adversary-behavior taxonomies can also be used as the organizing abstraction. An enterprise scorecard based on MITRE ATT&CK defines per-technique, per-tactic, and overall organization protection scores using ATT&CK tactics and techniques, impact, exploitability, success or failure of tested techniques, and coverage of the matrix (Manocha et al., 2021). More recent agent-security benchmarks generalize this logic to dual-source threat models that distinguish user-level and environment-level attacks, then evaluate internal reasoning, behavioral trajectory, and task outcome separately (Ying et al., 11 Oct 2025, Luo et al., 31 May 2025). Although these latter frameworks target LLM or LVLM agents, they exemplify a general wide-spectrum principle: attack modeling should cover both attack origin and failure manifestation.

Across these variants, the methodological constant is the simultaneous use of several models: attack scripts and trees, exploit/vulnerability mappings, adversary taxonomies, and scenario semantics (Buriachok et al., 2019). This suggests that wide-spectrum evaluation does not privilege one attack formalism; it coordinates several, provided they can populate risk parameters, security characteristics, and evidence structures.

4. Metrics, normalization, and quantitative assessment

Quantitative evaluation in wide-spectrum methodologies is rarely a single formula. Instead, it combines risk, protection, assurance-process, and evidence-quality metrics. One influential security-rating approach reviews five assessment methods—denial-of-service probability, expected vulnerability damage from the $i$-th threat, fuzzy-set security requirements, threats-and-losses assessment, and degree of security—then proposes a normalized method for the degree of security assurance that operates with “not less than 3” characteristics and allows comparative analysis of heterogeneous information systems (Buriachok et al., 2019).

The classical weighted security degree is

$S = \sum_{i=1}^{N} W_i \cdot G_i,$

but this is criticized because systems with different characteristic sets cannot be directly compared and because the dependency between weight and characteristic value is not represented (Buriachok et al., 2019). The normalized alternative defines

$S^* = \sum_{i=1}^{N} W^*(x_i) \cdot G^*(x_i),$

with

$$W^*(x_i) = \frac{f_w(x_i)}{\max_x[f_w(x)]}, \qquad G^*(x_i) = \frac{G_i}{G_{i,\max}},$$

so that $S^* \le 1$ and heterogeneous systems become comparable on a normalized scale (Buriachok et al., 2019). This is one of the clearest formalizations of cross-system comparability in the literature.

Risk-oriented frameworks instead model event likelihood and impact explicitly. TELSAFE, a hybrid risk assessment framework for security gaps between standards and implementation, builds event trees over finite outcome spaces and defines path risk as

$n$0

where $n$1 is the probability of an event-tree path and

$n$2

aggregates confidentiality, integrity, and availability impacts (Siddiqui et al., 9 Jul 2025). The same framework uses normalized impact and normalized risk scores and explicitly argues that probabilistic modeling can eliminate the influence of expert opinion bias in the quantitative phase (Siddiqui et al., 9 Jul 2025).

The assurance literature broadens the metric space further. It emphasizes control-effectiveness metrics, exposure and vulnerability metrics, incident metrics, assurance-process metrics such as coverage, depth, rigour, and independence, and measurement-data confidence levels (Shukla et al., 2021). Embedded-systems work adds metric-quality constraints—comparability, cost effectiveness, measurability, repeatability, and reproducibility—and shows that metric selection must account for scales, units, reference values, automation, static versus dynamic measurement, and computing cost (Longueira-Romero et al., 2021). One explicit conclusion is that it is “not resource-effective to measure everything,” so selection of a practical subset of metrics is itself part of the methodology (Longueira-Romero et al., 2021).

A further quantitative issue is sensitivity. In the expected-damage method for the $n$3-th threat, abrupt saturation in the original piecewise mapping of attack frequency and damage can cause loss of sensitivity; the proposed improvement replaces it with a smooth hyperbolic tangent mapping over the full domain (Buriachok et al., 2019). This is representative of a broader methodological concern: wide-spectrum evaluation requires metrics that remain informative across wide ranges of scale, topology, and consequence.

5. Lifecycle integration, assurance evidence, and process maturity

Wide-spectrum evaluation is consistently described as lifecycle-integrated rather than point-in-time. The assurance review classifies methods by SDLC phase—governance, construction, deployment—and argues that assurance must be embedded into requirements, design/modeling, implementation, verification and validation, and deployment and operation, with explicit assurance checkpoints in each phase (Shukla et al., 2021). Requirements must be elicited, traced, analyzed, and linked to assurance techniques, metrics, and evidence sources; deployment must include continuous monitoring, metrics collection, anomaly detection, incident response, and periodic reassessment (Shukla et al., 2021).

A practical realization of this principle appears in agile e-government evaluation. There, an OWASP ASVS 3.0 workbook is used as a catalogue of security controls and verification requirements, a selection and scoping tool for Agile sprint security work, and a repeatable, evidence-friendly mechanism that aligns Agile activities with traditional security assurance expectations (Harrison et al., 2016). ASVS requirements are turned into security user stories, acceptance criteria, and Definition-of-Done items; “Create Test Scope” compiles selected controls into sprint or release scopes; and the resulting artifacts can be composed into RMADS-style documentation or assurance reports (Harrison et al., 2016). The methodological significance is that wide-spectrum evaluation becomes continuous and iterative even in fast-changing delivery settings.

Assurance cases provide the main evidence-integration artifact in the systematic-review literature. Structured arguments, often in GSN form, link claims, evidence, and context for CPS, IIoT, and service-oriented systems, and can incorporate properties assurance, component assurance, feature assurance, and security management assurance (Shukla et al., 2021). This is how heterogeneous evidence—testing, monitoring, formal verification, vulnerability prediction, and runtime metrics—can be integrated into a coherent assurance judgment without collapsing everything into a single metric.

Process maturity is itself an evaluable dimension. CRSTIP defines four maturity dimensions—legal and compliance assessment, security risk assessment, security testing, and tool support and integration—each on a four-level ordinal scale, so that a maturity profile is represented as

$n$4

with higher values indicating more systematic, quantitative, integrated, and continuous practice (Molnar et al., 2017). This does not replace system-level risk or security scoring; it evaluates the maturity of the assessment process that produces those scores.

Where expert input remains central, some methodologies include reliability checks. The distributed-wireless security-rating framework computes a coefficient of concordance,

$n$5

to assess the quality of expert ranking and weighting; $n$6 ranges from 0 to 1, with $n$7 indicating complete agreement, and a nominal $n$8 is adopted for computer technology (Buriachok et al., 2019). This is a reminder that wide-spectrum evaluation often depends not only on what is measured, but on the reliability of the evaluators themselves.

6. Domain instantiations, limitations, and research directions

The methodology appears in markedly different technical domains, but with recognizable structural similarities. In distributed wireless systems and heterogeneous information networks, it takes the form of multi-model attack analysis, normalized multi-characteristic security scoring, functional-cost analysis, and expert-concordance validation (Buriachok et al., 2019). In embedded systems, it appears as a metric taxonomy over hardware, software, data, services, crypto keys, and communications, aligned to industrial control security dimensions and constrained by repeatability and resource cost (Longueira-Romero et al., 2021). In robotics, it becomes a layered assessment over Physical, Network, Firmware, and Application, with explicit rejection of “security by obscurity” for internal communications (Vilches et al., 2018). In quantum communication, it becomes a seven-layer implementation assessment combined with hardness-of-protection levels, risk review, testing, patching, and re-assessment (Sajeed et al., 2019). A related QKD methodology extends this further with wide-band transmittance characterization over 400 to 2300 nm and attack-specific modeling of Trojan-horse, induced-photorefraction, and detector-backflash attacks (Tan et al., 21 Aug 2025).

Modern AI systems have generated new instantiations rather than overturning the pattern. SecureWebArena defines a unified evaluation suite with six environments, six attack vectors, 2,970 trajectories, and a multi-layered evaluation protocol for internal reasoning, behavioral trajectory, and task outcome (Ying et al., 11 Oct 2025). AgentAuditor evaluates LLM-agent safety and security through experiential memory, feature extraction, reasoning memory, and retrieval-augmented judgment over 2,293 annotated records, 15 risk types, and 29 scenarios (Luo et al., 31 May 2025). These systems do not replace earlier methodologies; they adapt wide-spectrum principles—multi-source threats, multi-layer analysis, structured evidence, and domain-specific taxonomies—to a new class of targets.

Several limitations recur across the literature. Traditional approaches such as Common Criteria are described as complex, time-consuming, focused on single products, poorly suited for composed systems, and methodologically weak for continuous assurance (Shukla et al., 2021). Static and “offline” assurance is repeatedly criticized because it ignores patches, reconfiguration, and environmental change (Shukla et al., 2021). Expert-driven methods can suffer from subjectivity; probabilistic frameworks such as TELSAFE explicitly aim to reduce the influence of expert opinion bias, but their applicability depends on the availability of structured empirical data (Siddiqui et al., 9 Jul 2025). Metric sets remain imbalanced: one embedded-systems survey reports that 77.5% of retained metrics are software-only and only 0.6% are hardware-only, showing a pronounced hardware-security gap (Longueira-Romero et al., 2021). Some authors also note that their own methods still require “more detailed consideration and the introduction of step-by-step instructions” (Buriachok et al., 2019).

The main research directions are correspondingly clear. The assurance review calls for continuous and adaptive assurance, compositional scoring, data-driven methods, and better integration with agile, DevOps, and model-driven engineering (Shukla et al., 2021). The distributed-wireless work suggests machine-learning-based risk prediction, dynamic re-evaluation, integration with standardized frameworks, automation of attack-graph generation, and functional-cost optimization (Buriachok et al., 2019). Embedded-systems work calls for selecting reduced metric sets from broader taxonomies, empirical validation on real systems, and improved hardware metrics (Longueira-Romero et al., 2021). TELSAFE identifies model checking as a future validation path (Siddiqui et al., 9 Jul 2025). Taken together, these directions suggest that the future of wide-spectrum security evaluation lies not in abandoning layered, metric-based, and scenario-based methodologies, but in making them more continuous, compositional, and empirically grounded.