Contestability Assessment Score (CAS)

• Contestability Assessment Score (CAS) is an emerging framework for evaluating if AI systems enable meaningful challenge via transparent explanations, justifications, and procedural recourse.
• CAS applies a weighted set of criteria across human-centered, technical, legal, and organizational dimensions to assess contestation beyond standard explainability metrics.
• Current implementations, such as the ConGaIT dashboard, use a normalized composite score on a 0–1 scale, though full standardization and validation remain ongoing challenges.

Contestability Assessment Score (CAS) is an emerging assessment construct for evaluating whether an AI system is not merely explainable but meaningfully contestable: open to challenge, responsive to disagreement, and equipped with procedural pathways for review, justification, traceability, and recourse. In the literature considered here, CAS is not yet a single canonical standard. One paper, “ConGaIT: A Clinician-Centered Dashboard for Contestable AI in Parkinson’s Disease Care,” presents an explicit weighted normalized CAS over eight criteria and reports a total score of $0.970$ (Nguyen et al., 30 Jul 2025). Other papers supply the conceptual, legal, organizational, and formal foundations that a broader CAS would need, including lifecycle-wide contestability, the distinction between explanation and justification, public-sector due process requirements, and formal models of decision-level contestability (Lyons et al., 2021, Balayn et al., 2024, Schmude et al., 25 Apr 2025, Landau et al., 2024, Yin et al., 15 Jul 2025, Freiesleben et al., 15 May 2026). A terminological clarification is necessary: in “CAS-IQA: Teaching Vision-LLMs for Synthetic Angiography Quality Assessment,” “CAS” does not mean “Contestability Assessment Score,” but “Contrast-free Angiography Synthesis” (Wang et al., 23 May 2025).

1. Definition, scope, and disambiguation

In the explicit CAS usage documented in ConGaIT, CAS is described as a score from prior work, cited there as Moreira et al. (2025), for evaluating “how well systems support such engagement across eight criteria spanning four pillars: human-centered, technical, legal, and organizational” (Nguyen et al., 30 Jul 2025). Within that framing, CAS is broader than standard explainability metrics. It is tied to procedural justice, oversight, recourse, and user empowerment, and it is used to assess whether a socio-technical system supports meaningful challenge rather than merely presenting explanations (Nguyen et al., 30 Jul 2025).

This scope matters because several neighboring literatures use “contestability” in a wider sense than a single dashboard or appeal interface. “Conceptualising Contestability: Perspectives on Contesting Algorithmic Decisions” characterizes contestability as “the ability to contest decisions,” often via post-hoc review or appeal, but also as an institutional safeguard and, less commonly, a design property of systems (Lyons et al., 2021). “From Stem to Stern: Contestability Along AI Value Chains” extends the unit of analysis still further, defining contestable AI as AI systems “open and responsive to human dispute and intervention throughout their lifecycle,” thereby shifting attention from a single decision output to the full AI value chain (Balayn et al., 2024).

The acronym is therefore context-sensitive. In contestability research, CAS denotes a score or assessment framework for contestable AI. In the angiography-quality paper, by contrast, CAS denotes “Contrast-free Angiography Synthesis,” and the work does not introduce a contestability score at all (Wang et al., 23 May 2025).

2. Conceptual foundations of contestability

The literature consistently separates contestability from explanation-only accounts. In the Australian ethics-framework analysis, explainability is treated as necessary but not sufficient: explanation provides grounds for contest, whereas contestability requires actual avenues and procedures for challenge and review (Lyons et al., 2021). The same distinction is sharpened in the public-sector regulation study, which reports the formulation “contesting presupposes understanding” and distinguishes descriptive explainability from normative explainability. Descriptive explainability concerns how a system works; normative explainability concerns reasons, institutional embedding, justification, accountability, and legitimacy (Schmude et al., 25 Apr 2025). This distinction is central to CAS because a system can disclose model logic yet still fail to support meaningful challenge.

A second recurring distinction is between explanation and justification. ConGaIT states this directly: explanation is how the model arrived at a result, whereas justification is why that result is normatively valid or acceptable in context (Nguyen et al., 30 Jul 2025). The public-sector literature converges on the same point by emphasizing that contestability depends on reason-giving, not only technical intelligibility, and that contestation channels must be connected to review and remedy rather than stopping at disclosure (Schmude et al., 25 Apr 2025, Landau et al., 2024).

A third conceptual axis concerns the level at which contestation occurs. Several papers reject a purely individual, post-decision framing. The value-chain workshop paper explicitly raises individual versus collaborative contestability, contestability within versus from outside the supply chain, and governance or public-sphere challenge to infrastructure, data practices, and downstream societal impacts (Balayn et al., 2024). The public-sector regulation paper similarly distinguishes judicial and non-judicial channels of contestation and individual and collective contestation action (Schmude et al., 25 Apr 2025). The government-AI workshop report treats contestability as a due-process-inflected capability requiring notice, understandable explanation, access to relevant information, meaningful human review, accessibility, reproducibility, procurement controls, and ongoing monitoring (Landau et al., 2024).

The most explicit formal contrast with recourse appears in “Explainable AI Isn’t Enough! Rethinking Algorithmic Contestability.” There, recourse assumes the decision is valid and helps a person change features to obtain a better outcome, whereas contestability starts from the presumption that the decision may be incorrect and seeks evidence to challenge and potentially overturn it (Freiesleben et al., 15 May 2026). This suggests that CAS, when used rigorously, should not be a generic transparency score. It is an assessment of whether challenge is possible, intelligible, procedurally supported, and substantively capable of altering outcomes.

3. Scoring architecture in explicit CAS practice

The clearest formal CAS rubric in the current literature appears in ConGaIT. The paper states that CAS spans eight criteria under four pillars and provides a table from which the aggregation can be reconstructed as a weighted normalized sum (Nguyen et al., 30 Jul 2025):

$$\mathrm{CAS} = \sum_{p} \lambda_p \cdot \frac{s_p}{s^{\max}_p}$$

where $p$ indexes each criterion, $\lambda_p$ is its weight, $s_p$ is the achieved score, and $s^{\max}_p$ is the maximum possible score. The per-criterion contribution is

$$\mathrm{CAS}_p = \lambda_p \cdot \frac{s_p}{s^{\max}_p}$$

and the total CAS is the sum of these contributions (Nguyen et al., 30 Jul 2025).

Criterion	Weight $\lambda_p$	Score / Max
Explainability	0.30	2 / 2
Openness to Contestation	0.12	2 / 2
Traceability	0.12	9 / 10
Built-in Safeguards	0.12	1 / 1
Adaptivity	0.10	2 / 2
Auditing	0.10	2 / 2
Ease of Contestation	0.07	9 / 10
Explanation Quality	0.07	42 / 50

Using these values, ConGaIT reports a total CAS of $0.970$, which is a weighted normalized composite on a $0$–$$\mathrm{CAS} = \sum_{p} \lambda_p \cdot \frac{s_p}{s^{\max}_p}$$0 scale rather than an unweighted average or model-performance measure (Nguyen et al., 30 Jul 2025). The weighting scheme is asymmetric: explainability carries $$\mathrm{CAS} = \sum_{p} \lambda_p \cdot \frac{s_p}{s^{\max}_p}$$1, nearly one-third of the total, while traceability, built-in safeguards, and openness to contestation each carry $$\mathrm{CAS} = \sum_{p} \lambda_p \cdot \frac{s_p}{s^{\max}_p}$$2, and ease of contestation and explanation quality each carry $$\mathrm{CAS} = \sum_{p} \lambda_p \cdot \frac{s_p}{s^{\max}_p}$$3 (Nguyen et al., 30 Jul 2025).

The paper also makes clear what remains unspecified. It does not provide a standalone equation in the main text, rubric-level subcriteria for each component, thresholds for acceptable CAS, inter-rater agreement statistics, or a full evaluation checklist (Nguyen et al., 30 Jul 2025). The aggregation is therefore reproducible from the table, but the assignment of raw scores is only partially transparent.

4. Decision-level formalizations and evidentiary grounds

Beyond system-level rubric design, recent work develops formal accounts of what makes a particular decision contestable. The strongest example is “Explainable AI Isn’t Enough! Rethinking Algorithmic Contestability,” which defines two central notions. A decision is normatively contestable if

$$\mathrm{CAS} = \sum_{p} \lambda_p \cdot \frac{s_p}{s^{\max}_p}$$4

and epistemically contestable relative to a feature set $$\mathrm{CAS} = \sum_{p} \lambda_p \cdot \frac{s_p}{s^{\max}_p}$$5 if

$$\mathrm{CAS} = \sum_{p} \lambda_p \cdot \frac{s_p}{s^{\max}_p}$$6

Here $$\mathrm{CAS} = \sum_{p} \lambda_p \cdot \frac{s_p}{s^{\max}_p}$$7 is the normatively correct decision, $$\mathrm{CAS} = \sum_{p} \lambda_p \cdot \frac{s_p}{s^{\max}_p}$$8 is the epistemically correct decision given the individual’s true features, and $$\mathrm{CAS} = \sum_{p} \lambda_p \cdot \frac{s_p}{s^{\max}_p}$$9 is the actual decision based on measured features and an estimated model (Freiesleben et al., 15 May 2026).

That paper argues that standard local XAI methods such as counterfactuals, LIME, and Anchors generally establish only weaker forms of evidence—“somewhere contestability” or “somewhere inaccuracy”—because they reveal errors in a neighborhood around the individual rather than sufficient grounds to overturn the focal decision itself (Freiesleben et al., 15 May 2026). It identifies three stronger evidence types for reversal: predictive multiplicity, incorrect feature values, and neglected overruling evidence. Predictive multiplicity concerns disagreement among equally justified models in a Rashomon set; incorrect feature values concern decisions based on wrong personal data; neglected overruling evidence concerns relevant additional evidence outside the original feature set that would change the decision (Freiesleben et al., 15 May 2026). A plausible implication is that a mature CAS would need to distinguish weak anomaly signals from decision-overturning evidence.

A different formalization appears in “Contestability in Quantitative Argumentation,” where contestability is modeled in Edge-Weighted Quantitative Bipolar Argumentation Frameworks. The contestability problem is to modify edge weights so that a topic argument $p$0 attains a desired strength $p$1, that is, to find $p$2 such that

$p$3

The paper defines the attainable set $p$4, develops gradient-based relation attribution explanations (G-RAEs) as sensitivity measures over relations, and provides an iterative algorithm for adjusting edge weights toward a target outcome (Yin et al., 15 Jul 2025). This is not a published CAS, but it supplies formal ingredients—attainability, sensitivity, and intervention guidance—that could support a decision-level contestability assessment.

Taken together, these papers suggest two analytically distinct CAS layers. One layer scores whether institutions enable contestation procedurally. Another scores whether a specific output is challengeable on evidentiary or formal grounds. The literature does not yet unify these layers into a single validated standard.

5. System-level implementation contexts

ConGaIT provides the most concrete application of CAS as a system-level evaluation instrument. The dashboard is positioned as clinician-centered contestable AI in Parkinson’s disease care and uses a “Contest & Justify” interaction pattern supported by visual explanations, role-based feedback, and traceable justification logs (Nguyen et al., 30 Jul 2025). The workflow links a CNN prediction of Hoehn and Yahr stage, Layer-wise Relevance Propagation (LRP), GPT-4o-generated justification, clinician-selected argument types—Factual Error, Normative Conflict, and Reasoning Flaw—and immutable audit logs (Nguyen et al., 30 Jul 2025). In CAS terms, the paper presents this design as supporting explainability, openness to contestation, traceability, auditing, adaptivity, and ease of contestation.

The public-sector literature broadens these implementation requirements. “Challenging the Machine: Contestability in Government AI Systems” treats contestability as the ability to effectively challenge a decision made in whole or part using an automated system and develops a due-process-oriented bundle of conditions: notice, understandable explanation, access to evidence, reproducibility, legal fidelity, meaningful human review, accessibility, procurement controls, and ongoing monitoring (Landau et al., 2024). The same report argues that some AI/ML techniques may be incompatible with contestability in a given context and that agencies should retain an off-ramp, including the option not to deploy or to decommission a system (Landau et al., 2024).

In public-sector AI regulation, contestability is further mediated by translation across technical, legal, and organizational dimensions. The interview study on explainability and contestability identifies three translation processes: alignment of top-down and bottom-up regulation, assignment of responsibility for interpreting regulations, and establishment of interdisciplinary collaboration (Schmude et al., 25 Apr 2025). It also emphasizes that contestability must extend beyond judicial channels to non-judicial mechanisms such as mediation, ombudspersons, human intervention, tools for scrutiny, and annual assessments (Schmude et al., 25 Apr 2025). The value-chain workshop paper extends implementation concerns to upstream infrastructure, data collection, model development, oversight, deployment, downstream societal impacts, and governance or public debate (Balayn et al., 2024).

This broader literature indicates that CAS, if generalized beyond a single dashboard, must assess more than interface design. It must consider who can contest, what can be contested, when contestation can occur, how evidence and responsibility are surfaced, and whether institutional actors can respond meaningfully across the AI lifecycle (Balayn et al., 2024, Landau et al., 2024).

6. Interpretation, limitations, and unresolved questions

A recurrent misconception is to treat CAS as a synonym for explanation quality, model accuracy, or safety. The literature does not support that reading. ConGaIT explicitly presents CAS as a contestability score, not a clinical effectiveness or predictive-performance measure, and its reported $p$5 should not be interpreted as proof of clinical safety, proof of clinician trust in real practice, or a validated real-world human-factors outcome (Nguyen et al., 30 Jul 2025). More generally, the public-sector papers warn that formal availability of appeal or explanation can coexist with weak practical contestability when review is inaccessible, responsibility is diffuse, or remedy is absent (Lyons et al., 2021, Landau et al., 2024).

The evidence base is also uneven. ConGaIT’s observable criteria were scored through direct inspection, while subjective criteria such as Explanation Quality were assessed using simulated feedback from three clinician personas generated via OpenAI GPT-4 and Microsoft TinyTroupe rather than a real clinical user study (Nguyen et al., 30 Jul 2025). Several of the most influential CAS-relevant papers are conceptual, agenda-setting, or workshop-oriented and do not provide validated scoring models, reliability studies, or benchmark thresholds (Balayn et al., 2024, Landau et al., 2024). Even where formulas are reconstructed or formal mechanisms are defined, the literature frequently leaves weighting choices, minimum thresholds, and cross-domain calibration unresolved (Nguyen et al., 30 Jul 2025, Yin et al., 15 Jul 2025, Freiesleben et al., 15 May 2026).

A further limitation is domain dependence. The public-sector studies repeatedly stress context sensitivity, especially in welfare, health, and other high-stakes settings involving vulnerable populations (Schmude et al., 25 Apr 2025, Landau et al., 2024). The value-chain paper adds geographic and cultural variation, collective harms, environmental impacts, and the “many hands problem” of distributed responsibility (Balayn et al., 2024). These considerations suggest that a single universal CAS may be less appropriate than a family of sector-sensitive profiles or sub-scores.

The present state of the literature therefore supports a precise but limited conclusion. CAS is best understood not as a settled universal metric, but as an emerging assessment framework for contestable AI. Its explicit quantitative realization currently appears most clearly as a weighted normalized composite over eight criteria in ConGaIT (Nguyen et al., 30 Jul 2025). Its broader theoretical content is supplied by work on contestability as due process, lifecycle-wide dispute and intervention, the distinction between descriptive explanation and normative justification, and formal models of decision reversal or outcome adjustment (Lyons et al., 2021, Balayn et al., 2024, Schmude et al., 25 Apr 2025, Landau et al., 2024, Yin et al., 15 Jul 2025, Freiesleben et al., 15 May 2026).