Multi-Criteria Framework

• Multi-Criteria Framework is a structured model that decomposes complex decisions into explicit, weighted criteria.
• It integrates methods like AHP, fuzzy evaluation, and Bayesian models to aggregate scores and manage uncertainty.
• The framework enhances transparency and auditability, supporting diverse applications from software assessment to NLP tasks.

A multi-criteria framework is a structured scheme for evaluating, ranking, selecting, or controlling alternatives when several criteria must be considered simultaneously. In the literature, the expression covers at least three closely related uses: classical multi-criteria decision-making, in which objectives, criteria, and alternatives are organized and aggregated; decomposition frameworks that replace a single opaque judgment with several explicit criteria, as in relevance assessment; and task settings in which “criteria” denotes alternative annotation conventions, as in multi-criteria Chinese word segmentation (Wang et al., 17 Feb 2025, Farzi et al., 13 Jul 2025, Ke et al., 2020). Across these uses, the common aim is to make trade-offs explicit, preserve task structure, and support auditable inference.

1. Conceptual scope and terminological variants

In the classical decision-analytic sense, a multi-criteria framework assumes a problem structure with objectives, criteria, and alternatives with attributes; criteria may be organized hierarchically, and may include both quantitative and qualitative factors (Wang et al., 17 Feb 2025). This formulation underlies frameworks for software quality assessment, low-code platform selection, cloud infrastructure comparison, real-estate redevelopment, construction-product circularity, and fusion-facility siting (Basciani et al., 2023, Lamanna, 21 Oct 2025, Menzel et al., 2011, garrone, 15 Jan 2026, Meng et al., 10 Apr 2025, Abdussami et al., 24 Jun 2025).

A second usage treats the framework as a decomposition of a complex latent notion into explicit dimensions. In information retrieval, relevance is decomposed into exactness, coverage, topicality, and contextual fit, each judged independently before aggregation (Farzi et al., 13 Jul 2025). In retrieval-augmented decision support, criteria are extracted from documents, layered into a hierarchy, weighted, and then used to score alternatives with explicit evidence chains (Wu et al., 24 May 2025).

A third usage is task-specific rather than preference-specific. In multi-criteria Chinese word segmentation, “criteria” refers to dataset-specific segmentation conventions such as CTB, MSRA, and PKU; the framework is therefore a unification mechanism across incompatible annotation guidelines rather than a weighted trade-off among objectives (Ke et al., 2020).

A further extension is meta-methodological: a multi-criteria framework can itself be used to select an MCDA method. One such framework models the decision problem with a hierarchical set of nine descriptors, analyzes 56 MCDA methods, and resolves incomplete problem descriptions through a rule base with horizontal intersections and vertical unions (Wątróbski et al., 2018).

Interpretation	Core objects	Representative source
Decision-analysis framework	Objectives, criteria, alternatives, weights	(Wang et al., 17 Feb 2025)
Relevance decomposition	Exactness, coverage, topicality, contextual fit	(Farzi et al., 13 Jul 2025)
Annotation unification	Dataset-specific segmentation conventions	(Ke et al., 2020)
Method-selection framework	Problem descriptors mapped to MCDA methods	(Wątróbski et al., 2018)

This suggests that “multi-criteria framework” is best understood as a family resemblance term: the invariant is not a single algorithm, but explicit structure over multiple evaluative axes.

2. Structural design patterns and mathematical forms

Most frameworks in this literature adopt a hierarchical representation. Goal $\rightarrow$ Criteria $\rightarrow$ Subcriteria $\rightarrow$ Indicators is explicit in RAD, in the sustainable corn planning model, and in MMC circularity assessment; software quality assessment similarly organizes quality attributes in a tree and computes aggregates bottom-up (Wu et al., 24 May 2025, Haris et al., 2024, Meng et al., 10 Apr 2025, Basciani et al., 2023). Low-code platform selection formalizes this with a decision matrix $D=[s_{ij}]$, and sub-matrices $D^{(j)}=[s_{ijm}]$ when criteria have sub-criteria (Lamanna, 21 Oct 2025).

The dominant aggregation template is the weighted sum. A standard form is

$S_i=\sum_{j=1}^{K} w_j \,\tilde{s}_{ij},$

or, with sub-criteria,

$$S_i=\sum_{j=1}^{K} w_j \left(\sum_{m=1}^{M_j}\alpha_{jm}\,\tilde{s}_{ijm}\right).$$

This appears directly in enterprise platform selection, RAD, cloud comparison, and several other frameworks (Lamanna, 21 Oct 2025, Wu et al., 24 May 2025, Menzel et al., 2011). Weighted sums are favored for transparency, direct traceability from rubrics to scores, and straightforward sensitivity analysis (Lamanna, 21 Oct 2025).

Weight derivation is often handled through AHP. The canonical formulation is

$A w=\lambda_{\max} w,$

with consistency diagnostics

$$\mathrm{CI}=\frac{\lambda_{\max}-n}{n-1}, \qquad \mathrm{CR}=\frac{\mathrm{CI}}{\mathrm{RI}}.$$

This pattern appears in general LLM-based MCDM, low-code platform selection, sustainable corn planning, RAD, and probabilistic circularity assessment (Wang et al., 17 Feb 2025, Lamanna, 21 Oct 2025, Haris et al., 2024, Wu et al., 24 May 2025, Meng et al., 10 Apr 2025).

The weighted sum is not the only aggregation family. Fuzzy Comprehensive Evaluation computes a fuzzy evaluation vector through

$B=W\cdot R,$

and then maps it to grades or scalar scores (Wang et al., 17 Feb 2025). Mechatronic design uses the Choquet integral to encode importance and interactions among criteria, thereby modeling synergy, redundancy, and veto/pass effects beyond additive independence (Mohebbi et al., 2020). IR relevance decomposition includes both prompt-based aggregation and a summation-based scheme,

$\rightarrow$0

followed by thresholding into labels $\rightarrow$1 (Farzi et al., 13 Jul 2025).

At the more formal end, scenario theory generalizes single-criterion robustness to a collective treatment of multiple criteria and multiple datasets, certifying regions $\rightarrow$2 in which the vector of individual risks lies with high confidence (Garatti et al., 1 Apr 2026). Bayesian frameworks similarly elevate the structure from point weights to posterior distributions over weights, utilities, subgroup memberships, and rankings (Mohammadi, 2022).

3. Weighting, uncertainty, and consistency management

A major axis of variation concerns how a framework acquires and propagates preference information. Direct rating, stakeholder consultation, swing weighting, budget allocation, AHP, and pairwise-comparison methods all appear in the surveyed work (Lamanna, 21 Oct 2025, Menzel et al., 2011, Wang et al., 17 Feb 2025). Software quality modeling embeds SMARTER and SMARTS directly in the metamodel, allowing rank-order weights via ROC, RS, or RR, or swing weights normalized from stakeholder ratings (Basciani et al., 2023). Fusion-facility siting uses FUCOM and F-FUCOM so that comparative significance ratios are elicited with minimal comparisons and checked through a full-consistency optimization model (Abdussami et al., 24 Jun 2025).

Uncertainty is represented in several distinct ways. Fuzzy numbers and fuzzy measures appear in mechatronic design and in F-FUCOM-based siting (Mohebbi et al., 2020, Abdussami et al., 24 Jun 2025). Probabilistic MMC assessment samples indicator weights by Monte Carlo with Latin Hypercube Sampling, normalizes them within each criterion, and propagates them through a MIVES-based hierarchy to obtain distributions of sustainability and circularity scores rather than single values (Meng et al., 10 Apr 2025). Evidential Reasoning treats each criterion as a belief distribution over ordered grades and aggregates them with Dempster–Shafer combination, leaving unassigned mass to represent ignorance (Barahona et al., 2016).

Bayesian formulations push this further by treating preferences as random variables linked to latent weights through likelihoods such as multinomial, Dirichlet, Bradley–Terry–Luce, or Thurstone models, while logistic-normal or CLR-based priors accommodate criteria correlation (Mohammadi, 2022). Large-scale group heterogeneity is handled through finite mixtures or, as a plausible extension noted in the source, nonparametric mixtures (Mohammadi, 2022).

Consistency control is therefore not peripheral. AHP uses $\rightarrow$3 and $\rightarrow$4; FUCOM minimizes a deviation from perfect consistency; evidential reasoning monitors conflict; and scenario theory derives collective certificates that are sharper than naive union-bound constructions because the risks associated with individual criteria are treated jointly (Wang et al., 17 Feb 2025, Abdussami et al., 24 Jun 2025, Barahona et al., 2016, Garatti et al., 1 Apr 2026).

4. Representative instantiations across domains

In information retrieval, the Multi-Criteria framework for LLM-based relevance judgments decomposes passage relevance into exactness, topicality, coverage, and contextual fit, each graded on a common $\rightarrow$5–$\rightarrow$6 scale in Phase One and then aggregated in Phase Two, either by an LLM aggregator or by equal-weight summation with tuned thresholds (Farzi et al., 13 Jul 2025). This design is explicitly motivated by the observation that direct single-score prompting is often difficult to interpret and sensitive to prompt wording, model version, and temperature (Farzi et al., 13 Jul 2025).

RAD applies a closely related but document-centric logic. It retrieves document segments, extracts criteria, derives inter-criterion relations, layers them via ISM, assigns weights through a multi-agent AHP process, and evaluates alternatives through a weighted sum over normalized scores. The final report records criteria, weights, scores, reasoning, and citations, thereby making the decision pipeline auditable (Wu et al., 24 May 2025).

In Chinese NLP, the unified BERT-based MCCWS model uses a fully shared architecture conditioned by a prepended criterion token such as <pku> or <msra>, fuses BERT and bigram features through a gated mechanism, contextualizes the result with multi-head attention, and adds an auxiliary criterion classification head to preserve criterion-discriminative information (Ke et al., 2020). Here the framework resolves conflicts among multiple segmentation standards rather than among weighted preferences. A plausible implication is that multi-criteria frameworks are not confined to utility theory; they also function as unification devices across incompatible labeling regimes.

Autonomous-driving trajectory evaluation adopts yet another pattern. Safety is quantified from adaptive ellipsoidal safety zones through a time-indexed interaction metric based on overlap area or minimum gap; comfort is modeled with longitudinal and lateral jerk; efficiency is measured by total travel time; and the resulting criteria are aggregated into a single objective optimized with PSO (Naidja et al., 1 Sep 2025). The framework therefore couples geometric risk modeling, kinematic comfort constraints, and travel-time efficiency in one decision functional.

Domain-specific decision support offers many further instances. Enterprise low-code platform selection is organized around five key criteria—Business Process Orchestration, UI/UX Customization and Flexibility, Integration and Interoperability, Governance and Security, and AI-Enhanced Automation—and uses a weighted scoring model with optional AHP and risk-adjusted utility (Lamanna, 21 Oct 2025). Real-estate redevelopment frames intended-use selection through expected economic value, market and operational risk, technical and managerial complexity, and time-to-income, with an explicitly real-options-aware emphasis on preserving flexibility (garrone, 15 Jan 2026). Fusion siting evaluates 220 U.S. coal power plant sites using 22 criteria in four groups—State Policies, Federal Policies, Risk & Hazard Metrics, and Connectivity & Spatial Factors—weighted by FUCOM/F-FUCOM and aggregated by WSM (Abdussami et al., 24 Jun 2025).

5. Empirical behavior, interpretability, and validation

A recurring empirical theme is that explicit criteria improve auditability. In LLM-based relevance judgment, per-criterion grades expose why a passage is considered relevant or merely related, and they allow manual inspection of borderline cases such as topical but non-answering passages (Farzi et al., 13 Jul 2025). RAD extends this principle by requiring every claim in the generated decision report to be linked to evidence passages, numeric scores, and weighting logic (Wu et al., 24 May 2025). Evidential reasoning preserves full belief distributions instead of collapsing all evidence into a single point estimate, which supports richer interpretation of uncertainty and ignorance (Barahona et al., 2016).

Validation regimes differ by domain. IR work uses leaderboard correlation metrics such as Spearman’s $\rightarrow$7 and Kendall’s $\rightarrow$8; software and sequence-labeling frameworks use ranking stability or F1; spatial and infrastructure studies examine overall scores, top-ranked sites, and criterion-level decomposition; probabilistic construction-product assessment reports PDFs, CDFs, and ranking probabilities (Farzi et al., 13 Jul 2025, Ke et al., 2020, Abdussami et al., 24 Jun 2025, Meng et al., 10 Apr 2025). This suggests that a multi-criteria framework is typically validated not only by final rankings but also by the stability and interpretability of intermediate criterion behavior.

Domain	Framework	Reported result
IR evaluation	Multi-Criteria judgments	On LLMJudge, Multi-Criteria with Llama-3-8B reached $\rightarrow$9 and $\rightarrow$0
Chinese word segmentation	Unified BERT MCCWS	Average F1 = 97.204; average OOV recall = 83.60
MMC products	Probabilistic MCDM	For S3, Sustainability = 0.6346 and Circularity = 0.6483
Fusion siting	FUCOM/F-FUCOM + WSM	Highest-ranked site: R. M. Schahfer; group weights: CSF 26.59%, FP 25.16%, RHM 25.09%, SP 23.15%

Interpretability can also be structural rather than textual. HRA for ranking metaheuristics applies rank-based normalization and robust TOPSIS hierarchically across functions, indicators, and dimensions, producing a global closeness score while retaining intermediate dimension-level summaries (Goula et al., 2024). In the CEC 2017 study, EBOwithCMAR obtained the highest global closeness coefficient, $\rightarrow$1, while the intermediate levels exposed dimension-specific differences such as DES ranking first at dimension 100 (Goula et al., 2024).

6. Limitations, controversies, and future directions

The main methodological tension concerns compensation. Weighted sums are transparent, but they allow strong performance on one criterion to offset weak performance on another. The score-design literature makes this tension explicit: improving a scalar score should improve all performance metrics, yet under mild assumptions scalarization with $\rightarrow$2 is generally impossible for the improvement objective unless the relevant cone is a ray (Kabra et al., 2024). By contrast, Pareto preservation alone is much weaker; one-dimensional positive scalarization can preserve Pareto-optimality without preventing misaligned incentives (Kabra et al., 2024). This is a substantive controversy rather than a notational one: whether a framework should summarize or protect criteria.

A second limitation is the dependence of outcomes on elicitation choices. Weighting bias, subjective categorical mappings, prompt sensitivity, and dataset-specific conventions recur across frameworks (Basciani et al., 2023, Farzi et al., 13 Jul 2025, Lamanna, 21 Oct 2025). In MCCWS, sharply conflicting criteria and imbalanced data may bias a shared decoder toward dominant datasets; in IR evaluation, decomposition reduces brittleness but does not eliminate leniency on borderline topical content; in low-code selection and agricultural planning, the choice of rubrics, pairwise comparisons, and natural-break thresholds directly affects the output (Ke et al., 2020, Farzi et al., 13 Jul 2025, Lamanna, 21 Oct 2025, Haris et al., 2024).

A third challenge is structural realism. Additive models assume separability; fuzzy and Choquet models address interactions but introduce parameter-identification burdens (Mohebbi et al., 2020). Bayesian models can incorporate criteria correlation and heterogeneous decision-makers, but they require stronger modeling commitments and heavier inference (Mohammadi, 2022). Scenario theory provides sharper robustness certificates for simultaneous multi-criteria satisfaction, yet still assumes i.i.d. sampling across datasets and leaves dependence across criteria and datasets to future extensions (Garatti et al., 1 Apr 2026).

The surveyed work points toward several future directions already named in the sources: learnable aggregation and thresholding, human-in-the-loop review of ambiguous cases, dynamic or per-query criteria, multimodal decision models, richer sensitivity analysis, ensemble judges, additional MCDM schemes such as TOPSIS or VIKOR in automated pipelines, and updated validation on newer systems and datasets (Farzi et al., 13 Jul 2025, Wu et al., 24 May 2025, Wang et al., 17 Feb 2025, Meng et al., 10 Apr 2025, Lamanna, 21 Oct 2025). Taken together, these directions indicate that the multi-criteria framework is evolving from a static aggregation template into a broader paradigm for structured, uncertainty-aware, and auditable decision support.