Stakeholder-Oriented Performance Evaluation Framework

• The stakeholder-oriented performance evaluation framework is a multidimensional approach that integrates diverse stakeholder concerns into comprehensive performance assessments.
• It employs mixed methods and iterative designs, including Bayesian workflows and adaptive utility-weighted benchmarking, to translate stakeholder input into measurable metrics.
• Empirical studies show that active stakeholder engagement enhances project outcomes while highlighting the need for dynamic, context-aware evaluation mechanisms.

A stakeholder-oriented performance evaluation framework is an approach to assessment in which performance is specified, measured, and interpreted through the concerns, interests, values, expectations, or utility functions of multiple stakeholder groups rather than through a single fixed benchmark. Across the literature, such frameworks treat performance as multidimensional, context-dependent, and often temporally variable, with criteria that may differ between formal and informal evaluations and across stakeholder groups (McLeod et al., 2021). In empirical Monitoring and Evaluation (M&E) research on donor-funded projects in Kisumu Central Sub-County, stakeholder involvement in M&E influenced the performance of donor-funded projects; the study reported regularity of stakeholder interaction at mean $= 4.05$, $SD = 1.08$, contribution to organizational development at mean $= 3.79$, $SD = 0.94$, and incorporation of stakeholder perspectives into programming at mean $= 2.06$, $SD = 0.88$, indicating room for improvement (Amuni et al., 4 Sep 2025).

1. Conceptual basis

The central premise is that performance is not unitary but pluralistic. In project settings, evaluations may differ between stakeholders and across time because stakeholder expectations determine which evaluation criteria are appropriated, and those criteria are shaped by socioeconomic, historical, and organizational contexts (McLeod et al., 2021). This perspective shifts evaluation away from a purely objectivist reliance on the “iron triangle” of time, cost, and quality and toward a broader view in which process, product, and organizational dimensions coexist.

Several frameworks make this multidimensionality explicit. In software process improvement, five dimensions characterise the level of success achieved in SPI: project efficiency, impact on the process user, business success, direct operational success, and process improvement fit (Abrahamsson, 2013). In enterprise architecture, the proposed measurement framework consists of unique combination of higher order goals, non-functional requirement support and inputs-outcomes pair evaluation (Dube et al., 2011). In sustainability assessment, the reviewed design approaches distinguish Sustainability Indicator selection, relative importance assessment, and interdependency analysis, again resisting reduction to a single scalar criterion (Farahdel et al., 2024).

A recurring implication is that evaluation criteria are neither fixed nor purely technical. In recommendation settings with multiple stakeholders with possibly varying goals and interests, a single evaluation method or measure is not able to evaluate all relevant aspects in such a complex setting (Bauer et al., 2019). This same critique reappears in ML fairness and GenAI evaluation, where benchmark-driven, point-estimate comparisons are described as insufficient for uncertainty quantification, broader societal impacts, and stakeholder inclusion (Long, 21 Apr 2025).

2. Stakeholders, concerns, and representational units

Stakeholder-oriented frameworks differ primarily in how they identify stakeholders and what representational unit they assign to stakeholder input. In Kisumu Central Sub-County, the identified stakeholder categories included area administrators, project staff, beneficiaries, community members, funders, government, and religious leaders, with involvement spanning needs assessment, baseline assessment, monitoring, evaluation, meetings/workshops, and outreach (Amuni et al., 4 Sep 2025). In organizational-performance construct classification, stakeholder group-level performance is categorized with respect to investors, customers, employees, and society (Gopalakrishnan et al., 2021). In value-based assessment of algorithmic systems, the mapped stakeholders include development team, auditing team, data domain experts, and decision subjects, each with different insight needs and communication requirements (Yurrita et al., 2022). In digital archives, stakeholder groups were elicited as upstream, provider, system, consumer, and downstream actors (Atzenhofer-Baumgartner et al., 4 Jul 2025).

The theoretical lenses are equally varied. Stakeholder Engagement Theory guided the Kisumu study, emphasizing active, meaningful involvement of all stakeholder groups, and was supplemented by Resource-Based Theory to highlight the balance in organizational resources and priorities for optimal performance (Amuni et al., 4 Sep 2025). In project-success research, the framework links stakeholder interests, needs, values, beliefs, and experiences to stakeholder expectations and then to situated evaluation criteria (McLeod et al., 2021). In AI decision support, stakeholder preferences are formalized through context-dependent reward functions,

$$R_i : \mathcal{X} \times \mathcal{A} \times \mathcal{O} \to [0,1],$$

so that each actor’s preferences become explicit objects of optimization rather than informal background assumptions (Vineis et al., 12 Feb 2025).

This representational shift is technically consequential. Some frameworks map stakeholders to viewpoints and views, following IEEE 1471-2000 concepts of stakeholders, concerns, viewpoints, and views (Dube et al., 2011). Others map them to utility vectors, prior distributions, stress tests, or value manifestations. The result is not a single canonical formalism, but a family of methods for turning heterogeneous stakeholder positions into evaluable artifacts.

3. From stakeholder concerns to constructs, metrics, and tests

A core design problem is translating stakeholder concerns into measurable signals. CIRCLE formalizes this translation by starting with stakeholder concerns outside the AI stack and producing named constructs, indicators, scenarios, and monitoring plans through a six-stage lifecycle: Contextualize, Identify, Represent, Compare, Learn, and Extend (Schwartz et al., 27 Feb 2026). Its “construct systematization” turns a vague concern into a well-defined concept, while “construct operationalization” links that concept to observable behaviors, scenarios, and scoring rules.

Bayesian evaluation offers a different but related mechanism. In GenAI evaluation, stakeholder and expert knowledge is incorporated through prior elicitation, where beliefs about harms, bias, unfairness, or other values of interest are encoded as priors in parameter space or observable output space. The central inference step is written as

$$p(\bm{\theta}\mid \bm{y}) \propto p(\bm{y}\mid \bm{\theta})p(\bm{\theta}),$$

and stakeholder-provided prior data can directly inform posterior inference (Long, 21 Apr 2025). Here, stakeholder participation is not limited to post hoc interpretation; it enters the probabilistic model itself.

Other frameworks operationalize stakeholder input through utilities and weighted links. Adaptive utility-weighted benchmarking models benchmarking as a multilayer graph

$G = (V_T, V_M, V_H, E, W),$

with nodes for metrics, model components, and stakeholder groups, and edge weights from stakeholder nodes to metric nodes derived from conjoint analysis (Waggoner, 12 Feb 2026). In multi-stakeholder ML evaluation for job matching, participants’ utilities over seven metrics—Accuracy, Specificity, Sensitivity, Precision, Disparate Impact, Equalized Odds, and Counterfactual Fairness—were estimated through a discrete-choice design, then clustered into five preference groups (Yokota et al., 3 Mar 2025).

A more distributional formulation appears in stress-testing approaches to fairness. Rather than privileging a fixed fairness metric, each stakeholder curates one or more stress tests containing examples reflecting their interests, and the model passes or fails each test relative to a pre-defined threshold (Lopez-Paz et al., 2022). This relocates performance evaluation from metric selection alone to the curation of socially relevant distributions of examples.

4. Methodological architectures

Stakeholder-oriented evaluation frameworks commonly adopt mixed, iterative, or lifecycle-based research designs because no single instrument is taken to capture the full performance picture. The Kisumu study used a Convergent Parallel Mixed Methods Design, simultaneously integrating questionnaires and key informant interviews, with a sample size computed using Yamane’s formula,

$n = \frac{N}{1 + N(e)^2},$

where $SD = 1.08$0 and $SD = 1.08$1, yielding $SD = 1.08$2 community respondents plus 108 purposively sampled project staff (Amuni et al., 4 Sep 2025).

In recommender-system evaluation, the recommended designs are likewise Convergent Parallel Design and Sequential Design, combining offline system-centric metrics with user-centric and stakeholder-specific measures (Bauer et al., 2019). This is motivated by the claim that accuracy, profit, coverage, novelty, serendipity, diversity, and subjective feedback capture different aspects of multi-stakeholder performance and can expose blind spots when interpreted together.

Bayesian workflows make the iterative structure explicit. The stages are initial model building, prior predictive checks, model fitting, model diagnostics, posterior predictive checks, sensitivity analysis, and model comparison/refinement (Long, 21 Apr 2025). Stakeholder participation can occur at prior elicitation, model checking and validation, and interpretation and refinement. CIRCLE offers a parallel lifecycle with concrete work products—Context Brief, Evaluation Design Plan, Execution Plan, Findings Synthesis Report, Stakeholder Insights Brief, and Continuous Monitoring Plan—thereby formalizing traceability from stakeholder concern to downstream governance artifact (Schwartz et al., 27 Feb 2026).

This methodological pluralism also appears outside AI. In sustainability assessment, interviews, questionnaires, Delphi and Fuzzy-Delphi methods, stakeholder workshops, Pareto analysis, SMART approach, AHP, Fuzzy-AHP, DEMATEL, VIKOR, and causal models are all used to select, weight, and relate Sustainability Indicators (Farahdel et al., 2024). The consistent pattern is that stakeholder orientation tends to require more than one data source, more than one analytical lens, or more than one stage of validation.

5. Aggregation, benchmarking, and trade-off management

A major internal divide in stakeholder-oriented evaluation concerns aggregation. Some frameworks aggregate stakeholder-sensitive dimensions into weighted composite scores. In SPI, the proposed aggregate model is

$SD = 1.08$3

where the $SD = 1.08$4 reflect the relative importance of the five success dimensions for the relevant stakeholder(s) (Abrahamsson, 2013). In participatory multi-stakeholder decision-making, candidate strategies are ranked by a synthetic scoring mechanism,

$SD = 1.08$5

where the $SD = 1.08$6 encode user- or organization-prioritized metrics (Vineis et al., 12 Feb 2025).

Adaptive benchmarking generalizes the idea further by embedding stakeholder utilities into benchmark structure. Utility-derived edge weights are defined as

$SD = 1.08$7

and benchmark evaluation can be written as

$SD = 1.08$8

A human-in-the-loop adaptive rule then updates the weights while preserving stability, boundedness, and interpretability (Waggoner, 12 Feb 2026).

Other frameworks refuse single-number closure. In the stress-testing approach to fairness, results are presented as a vector of pass/fail outcomes per stress test rather than as a single “fairness number,” precisely to preserve the identity of the stakeholder concern and the distribution under which the claim is made (Lopez-Paz et al., 2022). In justice-based fair decision-making, performance-fairness trade-offs are represented in a two-dimensional utility space,

$SD = 1.08$9

and evaluated through Pareto-optimal trade-offs under deterministic or stochastic, shared or group-specific policies (Gupta et al., 15 Apr 2026).

These alternatives indicate a substantive methodological controversy rather than a merely technical choice. Aggregation can support ranking and deployment decisions, but vector- or Pareto-based reporting preserves contestation, minority concerns, and irreducible tensions among values.

6. Empirical patterns, tensions, and limitations

Empirical studies consistently report that stakeholder involvement is associated with richer evaluation and, often, better downstream performance, but they also show that the depth of influence matters. In Kisumu Central Sub-County, regular forums, workshops, and outreach were common, and 60.8% reported stakeholders were “often” engaged, yet least involvement was noted in project design, where only 3.9% indicated active stakeholder participation (Amuni et al., 4 Sep 2025). The same study reported that stakeholders generally affirmed they were consulted, but often not sufficiently in key programming steps, and that perceptions of tokenistic engagement undermined trust and project results.

Evidence from ML metric-preference elicitation complicates the assumption that “the stakeholder view” is homogeneous. In the job-matching study, 837 participants were clustered into five groups based on utility values for seven performance and fairness metrics; majority clusters did not particularly prioritize fairness metrics, while minority clusters expressed specific preferences for Equalized Odds, Counterfactual Fairness, or Disparate Impact (Yokota et al., 3 Mar 2025). The paper therefore warns that averaging preferences can suppress fairness-oriented minority perspectives.

Methodological limitations are also recurrent. The sustainability-assessment review identifies the static nature of many models, subjectivity and bias in participatory approaches, limited representation of marginalized stakeholder views, complexity of fuzzy hybrid models, and aggregation loss in composite indices (Farahdel et al., 2024). The enterprise-architecture framework was proposed partly because earlier efforts were concerned about only custom scales indicating the availability of a parameter in a range (Dube et al., 2011). In higher education, outcome-based quality assessment is constrained by the lack of an integrated data collection system and rich datasets, and by the treatment of “Students With No Data Available” as a zero-score category for simplicity (Khan et al., 2017).

A plausible implication is that stakeholder-oriented evaluation is moving toward dynamic, adaptive, and lifecycle-based forms. CIRCLE explicitly centers continuous monitoring, adaptive utility-weighted benchmarking uses a human-in-the-loop update rule, and Bayesian workflows support continuous learning from new data (Schwartz et al., 27 Feb 2026). Another plausible implication is that future work will continue to combine participatory elicitation with formal machinery—utilities, priors, stress tests, or network weights—so that stakeholder inclusion affects not only reporting but also the underlying evaluative model itself.