CATS: Conflict-Aware Trust-Score

• CATS is a trust evaluation framework that models contradictions and mutual reinforcement using graph propagation and information-theoretic principles.
• It applies to both retrieval-augmented generation in NLP and cooperative autonomy in vehicles, improving coherence and safety.
• Empirical evaluations show significant performance gains, reducing misclassification and enhancing system robustness with scalable trust metrics.

Conflict-Aware Trust-Score (CATS) is a class of trust evaluation frameworks that integrate information-theoretic and graph-propagation principles to assess reliability in environments characterized by conflicting, incomplete, or adversarial data. Originally formalized in the context of both retrieval-augmented generation (RAG) for natural language systems and cooperative perception for autonomous vehicles, CATS methodologies explicitly model and penalize contradictions as well as reward mutual reinforcement among sources. These metrics provide principled, scalable mechanisms for diagnosing and increasing the trustworthiness of information-centric and autonomy-centric systems (Qian et al., 12 Mar 2025, Mishra et al., 18 Dec 2025, Asavisanu et al., 1 Mar 2025).

1. Core Principles and Conceptual Overview

CATS frameworks are unified by explicit modeling of conflict. In RAG systems, CATS scores reflect how well a generated or retrieved answer is grounded, correct, and behaviorally consistent amidst contradictory evidence, thus quantifying trustworthiness beyond semantic similarity. In autonomous vehicle (AV) networks, CATS blends direct reputation with majority-view detection to rapidly detect, isolate, and penalize sources whose messages conflict with the consensus.

Key principles include:

• Representation of entities and their interactions as graphs (documents in RAG, vehicles in AVs).
• Explicit identification, scoring, and downstream penalization of contradictions.
• Hybridization of propagation mechanisms (e.g., PageRank variants, local voting) with reputation aggregation.
• Emphasis on interpretable, actionable trust outputs—often supported by sub-metrics for different behavioral expectations.

2. Mathematical Foundations and Propagation Models

RAG Document Graphs

A typical document-centric CATS graph is defined as $G = (V, E)$ where each node $d \in V$ is a document (e.g., news article), and directed edges $(d' \to d) \in E$ encode supporting or contradicting factual claims. Edge weights are stored as matrices $W^+ \in \mathbb{R}^{|V| \times |V|}$ (support) and $W^- \in \mathbb{R}^{|V| \times |V|}$ (contradiction). Trust scores $t^{(k)} \in [0,1]^{|V|}$ are iteratively updated by propagating supporting and penalizing contradicting influences:

$$P_d^{(k)} = \frac{\sum_{d'} t_{d'}^{(k)} w^+_{d',d}}{\sum_{d'} w^+_{d',d}} \quad N_d^{(k)} = \frac{\sum_{d'} t_{d'}^{(k)} w^-_{d',d}}{\sum_{d'} w^-_{d',d}}$$

$$I_d^{(k)} = P_d^{(k)} - N_d^{(k)}, \quad f(I_d^{(k)}) = \frac{I_d^{(k)} + 1}{2}$$

$$t_d^{(k+1)} = (1-\alpha)t_d^0 + \alpha f(I_d^{(k)}), \;\; \alpha \in (0, 1)$$

Normalization, convergence criteria, and damping are tuned for stability and interpretability (Qian et al., 12 Mar 2025).

Cooperative Autonomy CATS

In V2X settings, CATS combines local, majority-based conflict detection and global, reputation-based aggregation. Each vehicle $i$ maintains a reputation score:

$$\mathrm{rep}_i = \sum_{\text{up-votes}} +1 - \sum_{\text{down-votes}} 1$$

Status transitions (Trusted, Untrusted, Banned) are thresholded on $\mathrm{rep}_i$. Passive majority filtering applies: any sensor message contradicted by $> \lfloor m/2 \rfloor$ Trusted peers is flagged and penalized. Rate-limited windows and double-flagging schemes ensure resilience to vote spam and support rapid, privacy-preserving isolation of misbehaving sources (Asavisanu et al., 1 Mar 2025).

3. Metrication and Behavioral Sub-Metrics

In open-domain QA and RAG, CATS is operationalized via four core normalized sub-metrics, aggregated (typically by uniform weighting):

• Grounded Refusal (F1–GR): F1 score for the model’s ability to refuse answer generation in the absence of supporting evidence.
• Answer Correctness (AC): Fraction of answered queries matching gold standard answers (strict Single-Truth Recall).
• Grounded Citation (GC): Proportion of cited documents that actually entail the associated model claims, judged via entailment-style NLI.
• Behavioral Adherence (BA): Agreement with expected behavior types (e.g., neutrality in “conflicting opinions,” recency prioritization in “outdated” scenarios), as assessed by an LLM-as-a-Judge.

The aggregated score is

$$\mathrm{CATS} = \frac{1}{4}(\mathrm{F1\!-\!GR} + \mathrm{AC} + \mathrm{GC} + \mathrm{BA})$$

or a parametrized weighted sum using appropriately normalized $w_{1...4}$ (Mishra et al., 18 Dec 2025).

4. Conflict Detection and Relationship Extraction

Extracting Conflict in RAG

Documents are processed using LLMs (e.g., Gemma2-7B) to extract explicit factual claims, resolving pronouns and temporal/contextual ambiguity for maximal precision. Embeddings (e.g., mxbai-embed-large-v1, 512 dimensions) are computed for claims. Cosine similarity and curated thresholds select candidate pairs for in-context classification, which assigns relationships as supporting (+1), contradicting (–1), or unrelated (discarded). This builds the $W^+, W^-$ structure for graph propagation (Qian et al., 12 Mar 2025).

Conflict and Majority in Autonomy

Upon receiving a signed message, vehicles and central authorities verify trust status, aggregate neighborhood views, and apply consistency checks. Conflicting minority reports trigger local down-voting and, if repeated in distinct assessment windows, result in bans and certificate revocation. All trust actions are privacy-preserving by design, leveraging pseudonym certificates and non-location-based voting.

5. Empirical Evaluation and Practical Impact

RAG and QA Systems

Integration of CATS via graph-propagation-based trust in retrieval re-ranking yields improved output coherence as measured by LLM-based human alignment scores. For political news QA over 814 articles, ClaimTrust raises the mean LLM response quality by ≈11.2% (+0.3648 to +0.4055) compared to vanilla RAG ranking, without changing substring-match retrieval accuracy (Qian et al., 12 Mar 2025). In “reasoning-trace-augmented RAG,” supervised fine-tuning produces substantial gains in all sub-metrics for Qwen-2.5-7B:

Metric	Baseline	SFT-improved
F1–GR	0.167	1.000
AC	0.069	0.883
GC	0.111	0.648
BA	0.074	0.722

These results confirm that CATS incentivizes not only correct answers but proper refusal and behavioral adherence (Mishra et al., 18 Dec 2025).

Autonomous Vehicles

CATS protocols reduce false-negative rates by ≈230× and significantly lower false positives in realistic city-scale SUMO simulations with up to 27,000 vehicles. Average time to ban a misbehaving vehicle is under 58 seconds. Communication overhead remains within 0.7 KB/s per vehicle and per-message in-situ checks remain sub-10ms (Asavisanu et al., 1 Mar 2025).

6. Limitations and Open Challenges

• In RAG frameworks, the accuracy and granularity of CATS rest on LLM-driven claim extraction and relationship classification, making the pipeline sensitive to model performance, prompt design, and domain coverage.
• Parameter selection (e.g., damping factor $\alpha$, claim similarity thresholds, majority windowing, ban triggers) requires careful tuning and domain adaptation.
• Claims extraction generalization across heterogeneous sources (e.g., news, science, social media) may require schema and module redesign.
• All current CATS implementations rely on robust empirical evaluation; standardized, domain-agnostic trustworthiness metrics and benchmarks remain undeveloped.

A plausible implication is that CATS methodologies will require continual co-evolution with upgrades in language modeling, knowledge extraction, and adversarial robustness.

7. Future Directions

Immediate research trajectories include:

• Domain-adaptive fine-tuning of LLM-based claim extraction/classification (e.g., adapting disease-specific claim schemas).
• Automated parameter optimization (e.g., Bayesian search for damping or threshold parameters).
• Extension to multi-way or graded conflict semantics, supporting nuanced propagation models.
• Deep integration with standardized trustworthiness evaluators for end-to-end RAG and autonomous perception systems spanning diverse real-world datasets (Qian et al., 12 Mar 2025, Mishra et al., 18 Dec 2025).

Continued development and rigorous, interpretable evaluation of CATS frameworks are poised to underpin advances in conflict-aware, high-stakes information systems.