AgenticScore: Evaluation of Agentic Systems

• AgenticScore is a broad concept defining diverse evaluation methods that prioritize trajectory, workflow, and architectural fidelity over mere final outcomes.
• Benchmarks such as AgentRecBench and AgentArch exemplify its application by measuring task success, process correctness, and enterprise workflow efficiency.
• Recent studies expand agentic scoring to include privacy, safety, and tool integration, emphasizing comprehensive performance assessments in dynamic contexts.

Searching arXiv for papers on agentic evaluation and scoring frameworks. “AgenticScore” is best understood as a family resemblance rather than a single standardized metric. In recent literature, some papers explicitly note that they do not define a metric literally called “AgenticScore”; instead, they operationalize agentic evaluation through benchmark-specific scoring protocols that assess how an agent acts in an environment, uses tools, follows workflows, preserves privacy, or coordinates multi-step decisions. The unifying idea is a shift from outcome-only evaluation toward trajectory-aware, workflow-aware, and architecture-aware assessment of agentic behavior (Shang et al., 26 May 2025, Bogavelli et al., 13 Sep 2025, Huang et al., 7 May 2026, Ngong et al., 5 Mar 2026, Liu et al., 2 Dec 2025).

1. Terminological status and scope

The term has no single canonical definition across the cited literature. In "AgentRecBench: Benchmarking LLM Agent-based Personalized Recommender Systems" (Shang et al., 26 May 2025), the authors state that the paper does not define a metric literally called “AgenticScore”; the benchmark’s scoring protocol is instead the evaluation framework of AgentRecBench, centered on ranking-based Hit Rate@N. Other works adopt different domain-specific names and formalisms: "AgentArch: A Comprehensive Benchmark to Evaluate Agent Architectures in Enterprise" (Bogavelli et al., 13 Sep 2025) uses Acceptable Score for end-to-end enterprise workflow success; "Beyond Task Success: Measuring Workflow Fidelity in LLM-Based Agentic Payment Systems" (Huang et al., 7 May 2026) introduces Agentic Success Rate (ASR); "AgentSCOPE: Evaluating Contextual Privacy Across Agentic Workflows" (Ngong et al., 5 Mar 2026) evaluates Leak Rate (LR), Pipeline Violation Rate (PVR), and Violation Origin Rate (VOR); and "Process-Centric Analysis of Agentic Software Systems" (Liu et al., 2 Dec 2025) proposes graph-derived process metrics without collapsing them into a single scalar.

This dispersion of terminology is substantive rather than cosmetic. The literature evaluates different objects: some scores judge final task correctness under tool-use constraints, some judge whether the agent followed the correct ordered workflow, some judge privacy-preserving information flow at each boundary of the pipeline, and some judge structural or behavioral properties of the trajectory itself. A plausible implication is that “AgenticScore” functions more as a shorthand for agentic evaluation methodology than as the name of a universally agreed scalar.

2. Task-level and benchmark-level scoring protocols

One major usage evaluates whether an agent solves a task under realistic interaction constraints. AgentRecBench formalizes an agentic recommender as an LLM-based agent interacting with an environment

$$\mathcal{E} = (\mathcal{U}, \mathcal{I}, \mathcal{H}),$$

where $\mathcal{U}$ is the user feature space, $\mathcal{I}$ the item feature space, and $\mathcal{H}=\{(u,i,c,\tau)\}$ historical interactions with contextual metadata and timestamps. At each step, the agent observes $s_t \in \mathcal{S}$ and selects $a_t$ via $\pi_\theta$, with action space

$\mathcal{A} = \mathcal{I} \cup \mathcal{A}_{seek}.$

The benchmark evaluates classic, evolving-interest, and cold-start recommendation in a textual simulator built from Yelp, GoodReads, and Amazon. Each test instance contains 1 positive item and 19 negative items, so the agent ranks 20 candidates, and performance is measured by

$$\text{HR@}N = \frac{1}{|\mathcal{T}|}\sum_{t\in\mathcal{T}} \mathbb{I}(p_t \in \mathcal{R}_t^N),$$

with $N \in \{1,3,5\}$. Within this protocol, the “score” is the fraction of test cases in which the positive item appears in the top-$\mathcal{U}$0 list. The benchmark reports that agentic methods, especially Baseline666, RecHackers, and DummyAgent, substantially outperform MF and LightGCN across classic, evolving-interest, and cold-start settings (Shang et al., 26 May 2025).

AgentArch defines a stricter enterprise workflow score. It evaluates 18 agentic configurations across 6 LLMs on two workflows—Requesting Time Off and Customer Request Routing—using deterministic mock tools and human-annotated tool order, arguments, and decisions. The primary metric is Acceptable Score,

$\mathcal{U}$1

where success requires correct tool choice, correct tool arguments, and the correct final decision simultaneously. This explicitly rejects the notion that a correct final answer alone is sufficient for enterprise agentic success. The benchmark’s headline results are modest: the best score on the complex Customer Request Routing task is 35.3%, and the best score on the simpler Time Off task is 70.8%; moreover, $\mathcal{U}$2 peaks at only 0.0634, indicating low reliability across repeated trials (Bogavelli et al., 13 Sep 2025).

Controlled environments also use task-level accuracy to isolate agentic reasoning. GSM-Agent transforms GSM8K into a retrieval-first agentic benchmark in which the model sees only the question and must retrieve missing premises using Search(x) and NextPage(\cdot). Success is exact-match numerical accuracy, and even frontier models remain far from perfect: GPT-5 reaches 66.78% and o3 68.46% on the reported setting, while the paper highlights that even GPT-5 achieves only about 67% accuracy in the abstract. The benchmark’s significance lies less in the scalar itself than in its isolation of proactive search, tool use, and revisit behavior as distinct from static chain-of-thought reasoning (Zhu et al., 26 Sep 2025).

Long-horizon agentic scaling adds a different task-level perspective. "Agentic Aggregation for Parallel Scaling of Long-Horizon Agentic Tasks" (Lee et al., 13 Apr 2026) studies $\mathcal{U}$3 parallel rollouts and defines aggregation quality through Metric@K, the average benchmark metric after combining subsets of trajectories of size $\mathcal{U}$4. Here, the relevant “score” is task-dependent—gold-answer comparison on some benchmarks, exact answer-set match on DeepSearchQA, and rubric-based LLM judging on deep research tasks—but the core move is to score the final output $\mathcal{U}$5 produced by an aggregator that can inspect trajectories rather than only final answers. AggAgent improves over the strongest baseline by 2.4 to 5.3 points on average, by as much as 10.3 points on two deep research tasks, with aggregation overhead around 5.7% over rollout cost at $\mathcal{U}$6.

3. Workflow fidelity and process quality

A second major usage scores whether the agent followed the correct process, not merely whether it reached a correct endpoint. ASR in the HMASP payment setting is the clearest formalization. Let the expected trajectory be $\mathcal{U}$7 and the observed trajectory be $\mathcal{U}$8, and let $\mathcal{U}$9 and $\mathcal{I}$0 be the transition multisets of consecutive agent pairs. The paper defines transition recall and precision as

$\mathcal{I}$1

and then

$\mathcal{I}$2

ASR therefore scores ordered transition fidelity. It detects skipped checkpoints that are invisible to Task Success Rate and Agent Handoff F1. Across 18 models and 90,000 instances, 10 of 18 models exhibit a payment-processing ASR gap, while 8 models achieve perfect 100% on T3 TSR, HF1, and ASR. Notably, GPT-4.1 is reported to have perfect TSR and HF1 yet hidden workflow shortcuts, whereas GPT-5.2 achieves perfect ASR (Huang et al., 7 May 2026).

Agentic task decomposition work extends process scoring to graph structure. "Advancing Agentic Systems: Dynamic Task Decomposition, Tool Integration and Evaluation using Novel Metrics and Dataset" (Gabriel et al., 2024) evaluates DAG construction and tool selection through Node F1 Score, Structural Similarity Index (SSI), and Tool F1 Score. SSI is defined as

$\mathcal{I}$3

The empirical conclusion is topology-dependent: for sequential tasks, SSI is the strongest predictor of Answer Score with $\mathcal{I}$4 and $\mathcal{I}$5; for parallel tasks, Tool F1 has the strongest reported correlation with Answer Score at $\mathcal{I}$6 and $\mathcal{I}$7. This directly supports the view that an agentic score is not topology-invariant.

Process-centric software analysis goes further by representing trajectories as graphs. Graphectory models a run as a cyclic directed graph

$\mathcal{I}$8

with temporal edges $\mathcal{I}$9 for chronological succession and structural edges $\mathcal{H}=\{(u,i,c,\tau)\}$0 for navigation in task space. From this representation, the paper derives Node Count, Temporal Edge Count, Loop Count, Average Loop Length, Structural Edge Count, and Structural Breadth, and compresses phase sequences into LANGUTORY strings over localization, patching, and validation. Over 4000 trajectories from SWE-agent and OpenHands on SWE-bench Verified, the analysis shows that richer prompts and stronger LLMs produce more complex Graphectory, resolved issues often follow coherent localization-patching-validation patterns, unresolved ones are more chaotic and repetitive, and even successful trajectories frequently remain inefficient (Liu et al., 2 Dec 2025).

Behavioral LLM-judge scoring introduces another process-centric variant. In the stock-prediction framework, five-day episodes are scored along six dimensions—regime detection, routing, adaptation, risk calibration, strategy coherence, and error recovery—by an ensemble of GPT 5.4, Claude 4.6 Opus, and Gemini 3.1 Pro. Perturbation-based validation over 420 episodes yields targeted score drops from $\mathcal{H}=\{(u,i,c,\tau)\}$1 to $\mathcal{H}=\{(u,i,c,\tau)\}$2 on intended dimensions versus an average of $\mathcal{H}=\{(u,i,c,\tau)\}$3 on the remaining five, with Krippendorff’s $\mathcal{H}=\{(u,i,c,\tau)\}$4 up to 0.85. The composite score correlates with realized 20-day Sharpe ratio at $\mathcal{H}=\{(u,i,c,\tau)\}$5, and the per-dimension shortfalls are fed back into SAC reward shaping (Ridhawi et al., 7 May 2026).

4. Architecture and orchestration as scoring targets

Some work treats architecture itself as the object of evaluation. AgentArch varies four design dimensions—orchestration strategy, agent prompt implementation, memory architecture, and thinking-tool integration—and evaluates 18 configurations across GPT-4.1, GPT-4o, GPT-4.1-mini, o3-mini, LLaMA 3.3 70B, and Claude Sonnet 4. Its central conclusion is that no one-size-fits-all architecture exists: function calling generally outperforms ReAct, especially in multi-agent settings; thinking tools help simpler tasks more than complex ones; and model-specific preferences are pronounced. Claude Sonnet 4 with single-agent function calling peaks on Customer Request Routing at 35.3%, while GPT-4.1 with single-agent function calling, summarized memory, and thinking tools peaks on Time Off at 70.8% (Bogavelli et al., 13 Sep 2025).

The orchestration literature makes these trade-offs explicit. "Design and Implementation of Agentic Orchestrations and Orchestration of Agents" (Rinderle-Ma et al., 30 Jun 2026) classifies realizations into OO1–OO4 depending on whether the agent is process-aware and whether an explicit process frame exists. It then proposes quantitative metrics for realization properties: task specificity through cyclomatic complexity

$\mathcal{H}=\{(u,i,c,\tau)\}$6

and ABC complexity

$\mathcal{H}=\{(u,i,c,\tau)\}$7

correctness through precision, recall, and $\mathcal{H}=\{(u,i,c,\tau)\}$8 against a rule set $\mathcal{H}=\{(u,i,c,\tau)\}$9; reactivity through false negative rate

$s_t \in \mathcal{S}$0

and traceability through log correctness relative to the executed trace. In the predictive light sensing case study, OO1 yields $s_t \in \mathcal{S}$1, OO2 $s_t \in \mathcal{S}$2, OO3 $s_t \in \mathcal{S}$3, and OO4 $s_t \in \mathcal{S}$4, while deterministic OO5 and OO6 achieve perfect correctness.

A control-theoretic perspective supplies a deeper formal taxonomy of what counts as agency. "A Control-Theoretic Foundation for Agentic Systems" (Eslami et al., 11 Mar 2026) models an agentic controller as runtime authority over variables such as $s_t \in \mathcal{S}$5, $s_t \in \mathcal{S}$6, $s_t \in \mathcal{S}$7, $s_t \in \mathcal{S}$8, $s_t \in \mathcal{S}$9, and $a_t$0 within a closed-loop dynamical system, and organizes agency into five levels ranging from reactive rule-based control to governed synthesis of goals and controller architectures. The paper does not define a numeric score, but it provides a rigorous basis for interpreting “agenticness” as hierarchical decision authority over the control stack.

Typed workflow frameworks imply yet another evaluation axis. "Agentics 2.0: Logical Transduction Algebra for Agentic Data Workflows" (Gliozzo et al., 4 Mar 2026) treats an LLM call as a transducible function $a_t$1 that must satisfy typing, explainability, local evidence, and provenance requirements. This suggests an AgenticScore-like evaluation rooted in schema validity, locality of evidence, semantic observability, and benchmarked utility rather than in raw outcome correctness alone.

5. Privacy, safety, and information-flow scoring

Privacy-oriented agentic evaluation rejects the idea that a clean final answer implies a safe execution. AgentSCOPE introduces the Privacy Flow Graph, grounded in Contextual Integrity, which decomposes agentic execution into information flows at four boundaries: instruction, query, response, and output. Each edge is annotated by sender, recipient, subject, data type, and transmission principle. The benchmark evaluates 62 multi-step scenarios across eight regulatory domains and reports Task Success Rate, Leak Rate, Pipeline Violation Rate, and Violation Origin Rate. Across seven state-of-the-art LLMs, privacy violations occur in over 80% of scenarios, final-output leakage is about 24–40% depending on the model, and PVR is about 82–94%; most violations arise at the tool-response stage, where APIs return sensitive data indiscriminately. The central claim is that output-level evaluation alone substantially underestimates the privacy risk of agentic systems (Ngong et al., 5 Mar 2026).

AgentSocialBench extends privacy scoring to human-centered agentic social networks in which teams of personalized agents coordinate across domains and users. It formalizes a directed social graph $a_t$2, uses seven scenario categories spanning dyadic and multi-party interactions, and evaluates privacy with Leakage Rate

$a_t$3

along with category-specific metrics such as CDLR, MLR, CULR, MPLR, HALR, CSLR/CER, and ACS. Utility is measured by Information Abstraction Score and Task Completion Quality. The paper’s main findings are that cross-domain coordination creates the strongest leakage pressure and that “privacy instructions” can paradoxically increase discussion of sensitive information, a result termed the abstraction paradox. In the reported aggregates, dyadic leakage falls from 0.36 at L0 to 0.32 at L2 while IAS rises from 0.76 to 0.92, but in the multi-party group leakage rises from 0.11 to 0.13 as IAS rises from 0.76 to 0.89 (Wang et al., 1 Apr 2026).

These privacy benchmarks materially broaden the meaning of agentic scoring. They show that in regulated or socially mediated settings, the target of evaluation is often not “Can the agent finish the task?” but “Which information crossed which boundary, under what norm, and where did the violation originate?” This reorients AgenticScore-like thinking toward boundary-wise auditing rather than endpoint success.

6. Related concepts and terminological ambiguities

Several similarly named constructs are distinct from the benchmark-oriented usage of “AgenticScore.” "AgentScore: Autoformulation of Deployable Clinical Scoring Systems" (Estévez et al., 29 Jan 2026) concerns the automatic construction of bedside clinical checklists, not a general metric for agentic systems. It searches for a small rule subset $a_t$4 and builds a unit-weighted score

$a_t$5

followed by a thresholded decision. The framework uses LLM proposal plus deterministic verification, but its object is a deployable clinical guideline rather than the evaluation of agentic behavior per se.

Older strategic-scoring theory is also conceptually adjacent but operationally different. "Scoring Strategic Agents" (Ball, 2019) studies how an intermediary should score a strategic sender whose observable features are manipulable at cost. The optimal score underweights some features and overweights others to deter gaming while preserving calibration on average. This is a theory of predictive scoring under strategic distortion, not a benchmark for LLM agents, but it is relevant wherever the scored entity can adapt to the metric.

A more theoretical reinterpretation appears in "Probabilistic Modeling of Latent Agentic Substructures in Deep Neural Networks" (Lee et al., 8 Sep 2025), which models subagents as distributions combined by weighted logarithmic pooling and defines welfare through log score. In that framework, strict unanimity is impossible under linear pooling and impossible in binary outcome spaces, but possible with three or more outcomes. This is not an empirical benchmarking metric, yet it offers a formal way to think about whether composed substructures constitute a coherent higher-level agent.

Taken together, these works suggest that no single scalar adequately captures agentic quality across domains. The literature instead converges on a multidimensional view in which agentic evaluation may involve task success, tool correctness, ordered transition fidelity, structural coherence, privacy-boundary compliance, provenance, and architectural suitability. “AgenticScore,” in this broader scholarly sense, denotes the attempt to measure those properties at the level of trajectories, workflows, and control structures rather than at the level of final text alone.