Five-Criteria Evaluation Framework

Updated 5 January 2026

Five-criteria evaluation framework is a systematic approach that assesses technologies across five distinct, interdependent dimensions.
It employs multi-criteria decision analysis with methods like normalized weighting, Likert scales, and rank aggregation to derive composite scores.
The framework enhances transparency, supports sensitivity analysis, and drives evidence-based decision-making in diverse application domains.

The five-criteria evaluation framework is a widely-adopted paradigm in multi-dimensional assessment of technologies, platforms, algorithms, and machine learning systems. This model stipulates evaluating candidate systems across five orthogonal yet interdependent axes, with each criterion designed to isolate a distinct functional or quality aspect relevant to the target application domain. By structuring evaluation around these five dimensions, practitioners enable transparent aggregation, targeted diagnosis of strengths/weaknesses, and evidence-based decision-making.

1. Structural Overview and Aggregation Principles

A typical five-criteria evaluation framework operationalizes multi-criteria decision analysis (MCDA) via explicit scoring rubrics, normalized weighting, and formal aggregation algorithms. For example, Lamanna et al. (Lamanna, 21 Oct 2025) formalize the composite score for Low-Code Platform Selection as

$S = \sum_{i=1}^5 w_i \, s_i$

where $s_i \in \{1, \ldots, 5\}$ denotes criterion $i$ 's score, and $w_i$ is its normalized weight subject to $\sum_i w_i = 1$ . Weights are elicited via structured stakeholder engagement, priority assessment, and sensitivity analysis protocols. Scoring typically uses Likert scales, rank normalization, or continuous metrics, contextualized by application-specific sub-criteria. Sensitivity analysis of weight perturbations enforces ranking robustness.

2. Canonical Domains and Criterion Definitions

The five-criteria template reappears across diverse domains; representative instantiations include:

Enterprise Platform Selection (Lamanna, 21 Oct 2025): 1) Business Process Orchestration (BPO) 2) UI/UX Customization and Flexibility (UCF) 3) Integration and Interoperability (I&I) 4) Governance and Security (G&S) 5) AI-Enhanced Automation (AEA) Each criterion is decomposed further (e.g., BPO: BPMN compliance, workflow engine sophistication).
Watermarking in LLMs (Zhang et al., 24 Mar 2025): 1) Detectability 2) Fidelity of Text Quality 3) Embedding Cost (Usability) 4) Robustness (Scrubbing Resistance) 5) Imperceptibility (Spoof Resistance)
Cyber Range Assessment (Kampourakis et al., 11 Dec 2025): 1) Realism Fidelity 2) Security Isolation 3) Scalability 4) Flexibility/Extensibility 5) Training-Effectiveness Measurement
Chatbot Quality Evaluation (Liang et al., 2021): 1) Readability 2) Relevance 3) Consistency 4) Informativeness 5) Naturalness
Interpretability in ML/XAI (Pinto et al., 2024): 1) Plausibility 2) Intelligibility 3) Faithfulness 4) Stability 5) Usefulness

Domains prescribe precise criterion wording and scope, delineate sub-dimensions, and specify reference annotation or measurement protocols.

3. Weighting, Scoring, and Rank Aggregation Algorithms

Weight setting is a critical stage, typically realized either via stakeholder workshops (Lamanna et al.), analytic hierarchy process (AHP, as in (Kampourakis et al., 11 Dec 2025)), or defaulting to uniform weighting ( $w_i=1/5$ ) for non-expert fusion (HRA (Goula et al., 2024)). For AHP, the pairwise comparison matrix $A = [a_{ij}]$ is constructed with

$a_{ij} = \text{importance of } C_i \text{ over } C_j$

Its principal eigenvector yields $(w_1, \ldots, w_5)$ , and the consistency ratio CR ( $\leq$ 0.10) validates rationality.

Aggregation of per-criterion scores is carried out through:

Weighted sums (MCDA standard) for utility profile construction.
Hierarchical robust TOPSIS in the context of metaheuristics (Goula et al., 2024):
- Rank-based normalization across multiple performance matrices.
- Euclidean distances to positive/negative ideal solutions.
- Closeness coefficients (CC) as final ranking scores.

Applied frameworks rigorously balance additive, rank-based, or distance-based aggregation, all supporting multi-layer hierarchical evaluation.

4. Criterion-Specific Measurement Procedures

Each criterion mandates a context-specific scoring protocol:

Domain	Criterion	Measurement Principle	Typical Metric/Scale
LLM Watermarking (Zhang et al., 24 Mar 2025)	Detectability	ROC AUC, TPR/FPR	$S_D = \frac{\mathrm{AUCROC}-0.5}{0.5}$
Metaheuristics (Goula et al., 2024)	Robustness	Performance std / mean / best / worst	Rank-normalized, R-TOPSIS CC
Chatbots (Liang et al., 2021)	Readability	Human annotation, Likert scale	$\overline{s} = \frac{1}{N} \sum s_i$
XAI (Pinto et al., 2024)	Faithfulness	Surrogate accuracy, simulation	Model-output match over dataset

Human-centric evaluations (chatbots, XAI) rely on calibrated annotator panels, explicit anchor examples, and repeated reliability checks ( $\alpha$ , $\kappa$ statistics). Algorithmic or system evaluations exploit quantitative benchmarks and statistical summaries.

5. Empirical Validation, Sensitivity, and Benchmarking

Empirical studies validate frameworks by correlating decision outcomes, improving evaluation efficiency, and ensuring requirement coverage. For LCDP selection (Lamanna, 21 Oct 2025), adoption increased confidence by 30–40%, reduced decision time by 25–35%, and systematized requirement mapping. Sector-specific weight distributions (e.g., G&S = 28% in financial services) are established through real-world project data.

In algorithmic benchmarking (HRA (Goula et al., 2024)), large-scale comparisons (≥30 functions × 13 algorithms × 4 dimensions) are aggregated using hierarchical rank fusion, yielding robust portfolios unaffected by scale, outlier, or indicator bias.

Sensitivity analysis governs weight robustness, typically shifting $w_i$ by ±10–20%. Rankings should remain stable given small perturbations, attesting to methodological soundness.

6. Explainability, Standardization, and Extensibility

Standardization arises from transparent scoring matrices, reproducible weight elicitation, and publishable “evaluation profiles.” Explainability is enhanced by LLM-simulated expert panels (Kampourakis et al., 11 Dec 2025), which generate paired comparison rationales. Framework extension is feasible by adding new criteria (e.g., convergence indicators in optimization (Goula et al., 2024)), swapping aggregation algorithms (ELECTRE, VIKOR), or customizing domain weights. Evaluation profiles and matrices facilitate cross-organizational benchmarking and continuous improvement.

Frameworks admit several limitations:

Rank normalization may obscure absolute magnitude differences (as in HRA).
Uniform weighting may undervalue domain-specific priorities absent expert calibration.
Final scores often abstract away convergence properties, real-time feedback, or secondary effects.
Individual criterion definitions may require periodic revalidation as domains evolve (e.g., new regulatory requirements).

A plausible implication is the necessity for ongoing refinement—periodic score recalibration, weight re-elicitation, or criterion replacement—to maintain evaluative relevance and rigor as systems and user contexts shift.

This summary synthesizes canonical five-criteria frameworks across multiple research domains, with cross-references to arXiv sources (Lamanna, 21 Oct 2025, Goula et al., 2024, Zhang et al., 24 Mar 2025, Kampourakis et al., 11 Dec 2025, Liang et al., 2021), and (Pinto et al., 2024). Framework adaptability, quantification protocols, and auditing methodology are central to contemporary system evaluation.