Evidence-Anchored Reward Attribution (EARA)

• EARA is an evidence-based framework that transforms global outcomes into verifiable, localized, and signed reward signals.
• It leverages cooperative game theory and process-reward modeling to accurately distribute team-level scores to individual actions or messages.
• By enforcing auditability, repair-awareness, and strict credit conservation, EARA ensures robust, interpretable training in multi-agent, language, and multimodal systems.

Evidence-Anchored Reward Attribution (EARA) denotes a family of evaluation-aligned training methodologies in which the reward signals guiding learning are explicitly constructed to depend on evidence, process traces, or causal attributions that link observed system outcomes to the intermediate states, actions, or representations of a model or agent. This approach aims to enforce local faithfulness, credit conservation, and verifiable auditing throughout the full chain from system-level performance down to agent or message-level learning signals. EARA arose in response to inadequacies of outcome-only or post-hoc attribution methods, enabling principled credit assignment, correction-aware blame, and robust supervision in multi-agent, language, and multimodal systems.

1. Theoretical Foundations and Core Principles

EARA is operationally defined by the transformation of global evaluation metrics—typically scalar team outcomes or final task scores—into local, evidence-supported credit assignments and update signals. Its canonical formalization in multi-agent LLM systems leverages cooperative game theory (specifically, Shapley value credit assignment) together with process-reward modeling (PRM/OmegaPRM) to yield message-level, credit-conserving, signed rewards. The key unifying principle is that every local learning signal must be justified by and traceable to verifiable, auditable evidence sampled from the agent’s own computational trace or interaction record. This ensures:

• Locality: Learning signals are attached to individual messages, actions, or intermediate representations, not just to final outcomes.
• Signage: Rewards are signed, enabling both credit and blame (i.e., positive and negative reinforcement).
• Credit Conservation: The sum of all local rewards matches the global evaluator score, ruling out reward inflation or leakage.
• Direct Compatibility: Signals are plug-in ready for both RL (e.g., policy gradient, PPO, GRPO) and preference-based objectives (e.g., DPO, GRPO).
• Auditable Pathway: The full reward attribution chain can be reconstructed and justified at each level (Yang et al., 11 Nov 2025).

EARA’s conceptual formalism extends to Bayesian decision-making with KL-regularized control, where evidence corresponds to information-theoretic updates (conditional pointwise mutual information), and the reward proxy is precisely the information conveyed by an observed variable (Ortega, 2 Feb 2026).

2. Methodological Frameworks in Multi-Agent and Sequential Systems

The prototypical EARA pipeline for multi-agent LLM systems consists of:

1. System-level Evaluation: Compute the overall team or system score $R_{\mathrm{sys}}$, e.g., accuracy, F1, or custom task metric.
2. Shapley-based Agent Credit Assignment: For $\mathcal{A} = \{1,\ldots,n\}$ agents, define for each coalition $S \subset \mathcal{A}$:

• $v(S) = \mathrm{score}(E_S(x, y_S))$
• For agent $i$, Shapley value:

  $$\phi_i = \sum_{S \subset \mathcal{A} \setminus \{i\}} \frac{|S|! (n - |S| - 1)!}{n!} [v(S \cup \{i\}) - v(S)]$$

• The agent’s reward allocation $r_i = \phi_i$.

1. Process-Reward Modeling (Step-Level Attribution): Each agent’s credit $\phi_i$ is distributed across its messages $m_{i,t}$ using a local PRM judge that issues signed labels $s_{i,t} \in \{-1, 0, +1\}$. The per-message reward:

$\mathcal{A} = \{1,\ldots,n\}$0

with normalization constraints ensuring $\mathcal{A} = \{1,\ldots,n\}$1 and total conservation $\mathcal{A} = \{1,\ldots,n\}$2.

2. Failure Case (Blame and Preference Attribution): When $\mathcal{A} = \{1,\ldots,n\}$3, Shapley credits collapse to zero. EARA applies first-error localization via binary search to identify the earliest harmful message. Contrastive preference pairs are constructed for preference-based training. Corrective steps post-error are rewarded, enabling repair-aware learning.

Crucially, EARA guarantees that duplicative or sabotaging behavior is discouraged (as duplication yields marginal credit near zero and sabotage never increases credit) and that process-level audits are possible via reconstruction of the message-level reward chain (Yang et al., 11 Nov 2025).

3. Evidence-Anchored Reward Attribution in Language and Multimodal Systems

EARA is instantiated in a variety of contexts beyond multi-agent LLMs:

a) Table Reasoning and Structured Attribution (RSAT).

EARA defines a composite reward for small LLMs (SLMs), integrating answer accuracy, citation validity, entailment-based faithfulness, parsimony, and output format constraints. Faithfulness is assessed by sampling evidence (e.g., table cell values) cited in each reasoning step and scoring their entailment of the step’s claim using a cross-encoder NLI model. Only the integration of attribution into the generation process (not post-hoc annotation) yields high faithfulness and verifiable outputs; post-hoc recall is sharply limited (<13%) (Gajjar et al., 30 Apr 2026).

b) Retrieval-Augmented Generation (LEAR).

Evidence-anchored rewards in retrieval settings are computed independently for reasoning, extraction, and full context using hard masking to enforce verifiable support for answers. The RL reward aggregates answer correctness (via F1), length (promoting concise yet justifying evidence), and format adherence, all strictly grounded in the model’s cited, extracted content. This approach robustly compresses and denoises retrieved context (Zhao et al., 21 Jul 2025).

c) Causal Attribution in Chain-of-Thought Explanations.

EARA augments RLHF-type reward models with causal attributions (e.g., Integrated Gradients or Shapley value over chain-of-thought tokens), revealing which aspects of the explanation truly influenced the model’s answer. The reward model observes $\mathcal{A} = \{1,\ldots,n\}$4 and can penalize explanations not supported by what actually drove the LLM’s computation, thus mitigating reward hacking and fabricated rationales (Ferreira et al., 7 Apr 2025).

d) Multimodal Visual Reasoning and Attention Supervision.

In vision-language tasks, EARA is operationalized by directly supervising the spatial attention of the model over annotated evidence regions. For example, EASE (Evidence-Anchored Spatial Attention) converts bounding box annotations into soft visual token distributions and adds a KL penalty between model attention and these distributions, but only for rollouts rewarded by high answer correctness. This approach consistently increases pointing accuracy, evidence mass, and overall task performance across visual reasoning and multimodal math benchmarks (Hu et al., 29 May 2026, Liu et al., 15 Nov 2025).

4. Empirical Properties and Theoretical Guarantees

EARA realizes several desirable statistical and operational properties across domains:

Property	Description	Source
Locality	Rewards are attached to low-level steps or message units	(Yang et al., 11 Nov 2025)
Signed, Credit-Conserving	Local signals can be positive or negative; sum exactly matches outcome	(Yang et al., 11 Nov 2025)
Boundedness	Each local reward is tightly bounded, ensuring stability	(Yang et al., 11 Nov 2025)
Cooperation-Promoting	Agents/steps copying others or acting redundantly yield no marginal reward	(Yang et al., 11 Nov 2025)
Repair-Awareness	Both error identification and subsequent corrections receive targeted labels	(Yang et al., 11 Nov 2025)
Post-hoc Attribution Failure	Integrating attribution into the generation process is required; post-hoc mapping fails in SLMs	(Gajjar et al., 30 Apr 2026)
Information-Theoretic Identification	In special cases, EARA is the unique incentive signal aligning Bayesian posterior updates	(Ortega, 2 Feb 2026)

Ablations confirm that the faithfulness/evidence component is both necessary and not automatically learned from format or outcome signals alone. Removal of the evidence-aligned signal leads to sharp drops in faithfulness, while answer accuracy may remain unchanged or degrade slightly (Gajjar et al., 30 Apr 2026).

5. Practical Implementations and Domains

EARA has been successfully deployed in sequential decision-making, multi-agent LLM systems, evidence-based QA, multimodal document processing, and RL with verifiable rewards. Key ingredients and workflow stages typical to EARA implementations include:

• Judges and Evaluators: Automated local judges for message/step evaluation; NLI-based models for entailment; prefix/failure alignment judges for error localization.
• Plug-in RL Algorithms: Compatibility with GRPO, PPO, DAPO, DPO, and policy-gradient updates, often circumventing the need for explicit critics by group-relative advantage computation.

• Structured Output Constraints: JSON or specially tagged outputs (e.g., > , <answer>, <extract>) for explicit process traceability.

  • Supervised Fine-Tuning and RL Stages: SFT often provides initial format and citation fluency; EARA reward-driven RL is essential for evidence-based fidelity improvements. For instance, in RSAT, SFT alone boosts citation validity, but RL is required for faithfulness (Gajjar et al., 30 Apr 2026).

  Applications in multimodal regimes extend EARA to train vision-LLMs to both answer accurately and spatially ground their justifications on privileged evidence supervision during RL (without requiring such evidence at inference), yielding measurable improvements in attention mass and evidence coverage (Hu et al., 29 May 2026).

  6. Open Problems and Future Directions

  Despite empirical successes, EARA remains subject to limitations. Challenges include the cost of computing precise attributions (especially IG/Shapley methods), train–eval circularity (especially when faithfulness is both trained and scored using the same automatic proxy), generalization to out-of-domain or noisy evidence, and extension to on-line, human-in-the-loop RLHF. The identification theorem in Bayesian control highlights that EARA only fixes rewards up to a context-dependent baseline, motivating new approaches for reward normalization and cross-task coherence (Ortega, 2 Feb 2026). Future work spans human-supervised faithfulness models, cross-domain evaluations (e.g., finance, medicine), and improved interfaces for presenting evidence attribution to human overseers (Gajjar et al., 30 Apr 2026, Yang et al., 11 Nov 2025).

  7. Connections and Distinctions from Related Paradigms

  EARA differs from outcome-only RL by enforcing verifiable, evidence-based process supervision at every level of granularity. It stands in contrast to post-hoc or globally distributed reward schemes that cannot distinguish justifications supported by evidence from those that arise by chance or model bias. Compared to earlier step-level reward tagging schemes (PRM), EARA provides precise credit conservation and formal fairness guarantees via Shapley theory, supports repair-awareness, and integrates naturally with preference optimization pipelines. In multimodal and retrieval-augmented settings, the evidence-anchoring constraint is necessary both for empirical faithfulness and for user-verifiable outputs; retrofitted or post-hoc attribution fails to achieve comparable reliability or interpretability (Yang et al., 11 Nov 2025, Gajjar et al., 30 Apr 2026, Hu et al., 29 May 2026).