---
title: Contestability Assessment Score (CAS)
url: https://www.emergentmind.com/topics/contestability-assessment-score-cas
type: topic
---

# Contestability Assessment Score (CAS)

Contestability Assessment Score (CAS) is an emerging assessment construct for evaluating whether an AI system is not merely explainable but meaningfully contestable: open to challenge, responsive to disagreement, and equipped with procedural pathways for review, justification, traceability, and recourse. In the literature considered here, CAS is not yet a single canonical standard. One paper, “ConGaIT: A Clinician-Centered Dashboard for Contestable AI in Parkinson’s Disease Care,” presents an explicit weighted normalized CAS over eight criteria and reports a total score of \(0.970\) [2507.22300]. Other papers supply the conceptual, legal, organizational, and formal foundations that a broader CAS would need, including lifecycle-wide contestability, the distinction between explanation and justification, public-sector due process requirements, and formal models of decision-level contestability [2103.01774] [2408.01051] [2504.18236] [2406.10430] [2507.11323] [2605.16041]. A terminological clarification is necessary: in “CAS-IQA: Teaching Vision-Language Models for Synthetic Angiography Quality Assessment,” “CAS” does not mean “Contestability Assessment Score,” but “Contrast-free Angiography Synthesis” [2505.17619].

## 1. Definition, scope, and disambiguation

In the explicit CAS usage documented in ConGaIT, CAS is described as a score from prior work, cited there as Moreira et al. (2025), for evaluating “how well systems support such engagement across eight criteria spanning four pillars: human-centered, technical, legal, and organizational” [2507.22300]. Within that framing, CAS is broader than standard explainability metrics. It is tied to procedural justice, oversight, recourse, and user empowerment, and it is used to assess whether a socio-technical system supports meaningful challenge rather than merely presenting explanations [2507.22300].

This scope matters because several neighboring literatures use “contestability” in a wider sense than a single dashboard or appeal interface. “Conceptualising Contestability: Perspectives on Contesting Algorithmic Decisions” characterizes contestability as “the ability to contest decisions,” often via post-hoc review or appeal, but also as an institutional safeguard and, less commonly, a design property of systems [2103.01774]. “From Stem to Stern: Contestability Along AI Value Chains” extends the unit of analysis still further, defining contestable AI as AI systems “open and responsive to human dispute and intervention throughout their lifecycle,” thereby shifting attention from a single decision output to the full AI value chain [2408.01051].

The acronym is therefore context-sensitive. In contestability research, CAS denotes a score or assessment framework for contestable AI. In the angiography-quality paper, by contrast, CAS denotes “Contrast-free Angiography Synthesis,” and the work does not introduce a contestability score at all [2505.17619].

## 2. Conceptual foundations of contestability

The literature consistently separates contestability from explanation-only accounts. In the Australian ethics-framework analysis, explainability is treated as necessary but not sufficient: explanation provides grounds for contest, whereas contestability requires actual avenues and procedures for challenge and review [2103.01774]. The same distinction is sharpened in the public-sector regulation study, which reports the formulation “contesting presupposes understanding” and distinguishes descriptive explainability from normative explainability. Descriptive explainability concerns how a system works; normative explainability concerns reasons, institutional embedding, justification, accountability, and legitimacy [2504.18236]. This distinction is central to CAS because a system can disclose model logic yet still fail to support meaningful challenge.

A second recurring distinction is between explanation and justification. ConGaIT states this directly: explanation is how the model arrived at a result, whereas justification is why that result is normatively valid or acceptable in context [2507.22300]. The public-sector literature converges on the same point by emphasizing that contestability depends on reason-giving, not only technical intelligibility, and that contestation channels must be connected to review and remedy rather than stopping at disclosure [2504.18236] [2406.10430].

A third conceptual axis concerns the level at which contestation occurs. Several papers reject a purely individual, post-decision framing. The value-chain workshop paper explicitly raises individual versus collaborative contestability, contestability within versus from outside the supply chain, and governance or public-sphere challenge to infrastructure, data practices, and downstream societal impacts [2408.01051]. The public-sector regulation paper similarly distinguishes judicial and non-judicial channels of contestation and individual and collective contestation action [2504.18236]. The government-AI workshop report treats contestability as a due-process-inflected capability requiring notice, understandable explanation, access to relevant information, meaningful human review, accessibility, reproducibility, procurement controls, and ongoing monitoring [2406.10430].

The most explicit formal contrast with recourse appears in “Explainable AI Isn’t Enough! Rethinking Algorithmic Contestability.” There, recourse assumes the decision is valid and helps a person change features to obtain a better outcome, whereas contestability starts from the presumption that the decision may be incorrect and seeks evidence to challenge and potentially overturn it [2605.16041]. This suggests that CAS, when used rigorously, should not be a generic transparency score. It is an assessment of whether challenge is possible, intelligible, procedurally supported, and substantively capable of altering outcomes.

## 3. Scoring architecture in explicit CAS practice

The clearest formal CAS rubric in the current literature appears in ConGaIT. The paper states that CAS spans eight criteria under four pillars and provides a table from which the aggregation can be reconstructed as a weighted normalized sum [2507.22300]:

\[
\mathrm{CAS} = \sum_{p} \lambda_p \cdot \frac{s_p}{s^{\max}_p}
\]

where \(p\) indexes each criterion, \(\lambda_p\) is its weight, \(s_p\) is the achieved score, and \(s^{\max}_p\) is the maximum possible score. The per-criterion contribution is

\[
\mathrm{CAS}_p = \lambda_p \cdot \frac{s_p}{s^{\max}_p}
\]

and the total CAS is the sum of these contributions [2507.22300].

| Criterion | Weight \(\lambda_p\) | Score / Max |
|---|---:|---:|
| Explainability | 0.30 | 2 / 2 |
| Openness to Contestation | 0.12 | 2 / 2 |
| Traceability | 0.12 | 9 / 10 |
| Built-in Safeguards | 0.12 | 1 / 1 |
| Adaptivity | 0.10 | 2 / 2 |
| Auditing | 0.10 | 2 / 2 |
| Ease of Contestation | 0.07 | 9 / 10 |
| Explanation Quality | 0.07 | 42 / 50 |

Using these values, ConGaIT reports a total CAS of \(0.970\), which is a weighted normalized composite on a \(0\)–\(1\) scale rather than an unweighted average or model-performance measure [2507.22300]. The weighting scheme is asymmetric: explainability carries \(0.30\), nearly one-third of the total, while traceability, built-in safeguards, and openness to contestation each carry \(0.12\), and ease of contestation and explanation quality each carry \(0.07\) [2507.22300].

The paper also makes clear what remains unspecified. It does not provide a standalone equation in the main text, rubric-level subcriteria for each component, thresholds for acceptable CAS, inter-rater agreement statistics, or a full evaluation checklist [2507.22300]. The aggregation is therefore reproducible from the table, but the assignment of raw scores is only partially transparent.

## 4. Decision-level formalizations and evidentiary grounds

Beyond system-level rubric design, recent work develops formal accounts of what makes a particular decision contestable. The strongest example is “Explainable AI Isn’t Enough! Rethinking Algorithmic Contestability,” which defines two central notions. A decision is normatively contestable if

\[
\hat{d}(\tilde{x}^{(i)}) \neq d^*(i),
\]

and epistemically contestable relative to a feature set \(X\) if

\[
\hat{d}(\tilde{x}^{(i)}) \neq d(x^{(i)}).
\]

Here \(d^*(i)\) is the normatively correct decision, \(d(x^{(i)})\) is the epistemically correct decision given the individual’s true features, and \(\hat{d}(\tilde{x}^{(i)})\) is the actual decision based on measured features and an estimated model [2605.16041].

That paper argues that standard local XAI methods such as counterfactuals, LIME, and Anchors generally establish only weaker forms of evidence—“somewhere contestability” or “somewhere inaccuracy”—because they reveal errors in a neighborhood around the individual rather than sufficient grounds to overturn the focal decision itself [2605.16041]. It identifies three stronger evidence types for reversal: predictive multiplicity, incorrect feature values, and neglected overruling evidence. Predictive multiplicity concerns disagreement among equally justified models in a Rashomon set; incorrect feature values concern decisions based on wrong personal data; neglected overruling evidence concerns relevant additional evidence outside the original feature set that would change the decision [2605.16041]. A plausible implication is that a mature CAS would need to distinguish weak anomaly signals from decision-overturning evidence.

A different formalization appears in “Contestability in Quantitative Argumentation,” where contestability is modeled in Edge-Weighted Quantitative Bipolar Argumentation Frameworks. The contestability problem is to modify edge weights so that a topic argument \(\alpha\) attains a desired strength \(s\), that is, to find \(w'\) such that

\[
\sigma_{\mathcal{Q}'}(\alpha) = s.
\]

The paper defines the attainable set \(S(\alpha)\), develops gradient-based relation attribution explanations (G-RAEs) as sensitivity measures over relations, and provides an iterative algorithm for adjusting edge weights toward a target outcome [2507.11323]. This is not a published CAS, but it supplies formal ingredients—attainability, sensitivity, and intervention guidance—that could support a decision-level contestability assessment.

Taken together, these papers suggest two analytically distinct CAS layers. One layer scores whether institutions enable contestation procedurally. Another scores whether a specific output is challengeable on evidentiary or formal grounds. The literature does not yet unify these layers into a single validated standard.

## 5. System-level implementation contexts

ConGaIT provides the most concrete application of CAS as a system-level evaluation instrument. The dashboard is positioned as clinician-centered contestable AI in Parkinson’s disease care and uses a “Contest & Justify” interaction pattern supported by visual explanations, role-based feedback, and traceable justification logs [2507.22300]. The workflow links a CNN prediction of Hoehn and Yahr stage, Layer-wise Relevance Propagation (LRP), GPT-4o-generated justification, clinician-selected argument types—Factual Error, Normative Conflict, and Reasoning Flaw—and immutable audit logs [2507.22300]. In CAS terms, the paper presents this design as supporting explainability, openness to contestation, traceability, auditing, adaptivity, and ease of contestation.

The public-sector literature broadens these implementation requirements. “Challenging the Machine: Contestability in Government AI Systems” treats contestability as the ability to effectively challenge a decision made in whole or part using an automated system and develops a due-process-oriented bundle of conditions: notice, understandable explanation, access to evidence, reproducibility, legal fidelity, meaningful human review, accessibility, procurement controls, and ongoing monitoring [2406.10430]. The same report argues that some AI/ML techniques may be incompatible with contestability in a given context and that agencies should retain an off-ramp, including the option not to deploy or to decommission a system [2406.10430].

In public-sector AI regulation, contestability is further mediated by translation across technical, legal, and organizational dimensions. The interview study on explainability and contestability identifies three translation processes: alignment of top-down and bottom-up regulation, assignment of responsibility for interpreting regulations, and establishment of interdisciplinary collaboration [2504.18236]. It also emphasizes that contestability must extend beyond judicial channels to non-judicial mechanisms such as mediation, ombudspersons, human intervention, tools for scrutiny, and annual assessments [2504.18236]. The value-chain workshop paper extends implementation concerns to upstream infrastructure, data collection, model development, oversight, deployment, downstream societal impacts, and governance or public debate [2408.01051].

This broader literature indicates that CAS, if generalized beyond a single dashboard, must assess more than interface design. It must consider who can contest, what can be contested, when contestation can occur, how evidence and responsibility are surfaced, and whether institutional actors can respond meaningfully across the AI lifecycle [2408.01051] [2406.10430].

## 6. Interpretation, limitations, and unresolved questions

A recurrent misconception is to treat CAS as a synonym for explanation quality, model accuracy, or safety. The literature does not support that reading. ConGaIT explicitly presents CAS as a contestability score, not a clinical effectiveness or predictive-performance measure, and its reported \(0.970\) should not be interpreted as proof of clinical safety, proof of clinician trust in real practice, or a validated real-world human-factors outcome [2507.22300]. More generally, the public-sector papers warn that formal availability of appeal or explanation can coexist with weak practical contestability when review is inaccessible, responsibility is diffuse, or remedy is absent [2103.01774] [2406.10430].

The evidence base is also uneven. ConGaIT’s observable criteria were scored through direct inspection, while subjective criteria such as Explanation Quality were assessed using simulated feedback from three clinician personas generated via OpenAI GPT-4 and Microsoft TinyTroupe rather than a real clinical user study [2507.22300]. Several of the most influential CAS-relevant papers are conceptual, agenda-setting, or workshop-oriented and do not provide validated scoring models, reliability studies, or benchmark thresholds [2408.01051] [2406.10430]. Even where formulas are reconstructed or formal mechanisms are defined, the literature frequently leaves weighting choices, minimum thresholds, and cross-domain calibration unresolved [2507.22300] [2507.11323] [2605.16041].

A further limitation is domain dependence. The public-sector studies repeatedly stress context sensitivity, especially in welfare, health, and other high-stakes settings involving vulnerable populations [2504.18236] [2406.10430]. The value-chain paper adds geographic and cultural variation, collective harms, environmental impacts, and the “many hands problem” of distributed responsibility [2408.01051]. These considerations suggest that a single universal CAS may be less appropriate than a family of sector-sensitive profiles or sub-scores.

The present state of the literature therefore supports a precise but limited conclusion. CAS is best understood not as a settled universal metric, but as an emerging assessment framework for contestable AI. Its explicit quantitative realization currently appears most clearly as a weighted normalized composite over eight criteria in ConGaIT [2507.22300]. Its broader theoretical content is supplied by work on contestability as due process, lifecycle-wide dispute and intervention, the distinction between descriptive explanation and normative justification, and formal models of decision reversal or outcome adjustment [2103.01774] [2408.01051] [2504.18236] [2406.10430] [2507.11323] [2605.16041].

Source: https://www.emergentmind.com/topics/contestability-assessment-score-cas