TraceCard: Cost & Redundancy in LLM Tracing

• TraceCard is a compact, deterministic YAML summary that encodes every LLM call, tool use, and agent spawn, capturing detailed cost and redundancy signals.
• It employs a structured schema with fields like total_cost_usd and top_cost_spans to provide actionable insights for pruning and error localization in skill distillation.
• Serving as a bridge between raw traces and efficient pipeline analysis, TraceCard enables counterfactual reasoning and cost attribution for improved LLM performance.

Searching arXiv for papers on “TraceCard” and closely related trace-representation systems. TraceCard is a compact, deterministic YAML summary of a single LLM agent session in the ClawTrace tracing platform. It is designed as a machine-consumable intermediate representation that captures the full execution path, per-step cost in USD with cache-aware pricing, token counts by type, and signals about redundancy and failures or repairs. Within the ClawTrace and CostCraft pipeline, TraceCards serve as the bridge between raw traces and skill distillation: they translate raw traces into a small, uniform representation that CostCraft, or other pipelines, can reason over, especially about cost and redundancy (Yuan et al., 26 Apr 2026).

1. Definition and rationale

TraceCards were introduced to address a specific limitation in skill-distillation pipelines for LLM agents: existing trajectory-mining methods such as Trace2Skill see structure and outcome, but not how expensive each step was. Without per-step cost, a pipeline cannot distinguish adding a missing step to fix a bug from removing an expensive step that never affected the outcome. TraceCard therefore records every LLM call, tool use, and sub-agent spawn during an agent session, then compiles the session into a compact YAML object that an LLM analyst can ingest within a single context window (Yuan et al., 26 Apr 2026).

The immediate setting is the ClawTrace plus CostCraft workflow. ClawTrace instruments the agent and logs events; each session is then compiled into a TraceCard; CostCraft reads TraceCards and produces preserve, prune, and repair patches that are merged into an evolved SKILL.md. In this architecture, TraceCard is neither a raw transcript nor a dashboard-oriented observability artifact. The paper explicitly distinguishes it from observability tools that track per-span tokens and cost for humans but do not provide a compact, structured object suitable for downstream distillation (Yuan et al., 26 Apr 2026).

A recurrent misconception is to treat TraceCard as a generic trace dump. The design is instead selective and analytic. It preserves “what happened” together with rich cost signals, but it does so in summarized form: the paper reports a typical size of 1.2–1.8 kB and emphasizes compactness rather than transcript completeness. This suggests that TraceCard is intended less as archival storage than as an inference-ready representation for cost-aware agent analysis (Yuan et al., 26 Apr 2026).

2. Schema, fields, and cost attribution

The core schema is organized around whole-session aggregates and a bounded set of high-value local signals. The principal fields are summarized below (Yuan et al., 26 Apr 2026).

Field	Content	Notes
total_cost_usd	Cache-aware USD cost for the session	Sum over all spans
total_tokens	input, output, cacheRead, cacheWrite	Session aggregate
top_cost_spans	Top-5 spans sorted by cost_usd	Includes kind, role_hint, tokens, args_sample
redundant_tool_calls	Clusters of similar repeated tool calls	Heuristic field
sub_agents	Child-session summaries	Includes output_used_in_final
failed_or_repaired	Error-pattern matches in tool results	Heuristic field

Several fields are explicitly heuristic. role_hint is assigned from message behavior and turn position, with categories such as tool_call and final_reply. redundant_tool_calls groups tool calls with the same tool name and argument similarity at or above 0.8 by normalized Levenshtein distance; clusters of size at least two are flagged as redundant. Audited on 10 traces, this heuristic achieved 100% precision, ~80% recall. sub_agents.output_used_in_final is defined through Jaccard overlap between sub-agent output and the parent’s final message, although the paper notes that it was not exercised in experiments because the tested backbone did not spawn sub-agents. failed_or_repaired marks tool calls whose results match known error patterns such as placeholder outputs or exceptions (Yuan et al., 26 Apr 2026).

Cost attribution is central to the representation. ClawTrace computes per-span cost from input, output, cache-read, and cache-write token counts using provider-specific per-token USD rates, and the resulting cache-aware cost feeds both total_cost_usd and top_cost_spans. The paper gives an April 2026 example for openai-codex/gpt-5.4: $2.00 / 10^6` input tokens, `$8.00 / 10^6 output tokens, $0.50 / 10^6` cache-read tokens, and `$2.00 / 10^6 cache-write tokens. The importance of cache awareness is explicit: cache-read tokens are billed at about 25% of the fresh input rate; in the studied runs, cache-read tokens are 30–50% of input volume; and mis-pricing cache reads as full input overstates cost by 1.6–2.0× and distorts rankings of expensive steps (Yuan et al., 26 Apr 2026).

The published example illustrates the intended granularity. A session with session_id: "sb-task-47484" and model "openai-codex/gpt-5.4" has total_cost_usd: 0.068, token totals broken down by category, two expensive read_file('input.xlsx') calls inside top_cost_spans, and a redundancy cluster ["span-3", "span-7"] with similarity: 0.94. The example makes explicit that TraceCard is designed to preserve ranked cost structure and evidence of likely redundancy in the same object (Yuan et al., 26 Apr 2026).

3. Function in CostCraft skill distillation

TraceCards are the primary input to the Success Analyst and Error Analyst in CostCraft. CostCraft defines three patch actions. Preserve patches keep a behavior that contributed to success. Prune patches remove an expensive step that did not affect the outcome; each must name a specific high-cost span, provide a natural-language counterfactual, and phrase the rule as a behavior to avoid rather than as a numeric cost cap. Repair patches fix failures using oracle evidence such as rubric failures or golden outputs. This reorganizes skill distillation around correctness versus efficiency rather than the conventional success versus error split (Yuan et al., 26 Apr 2026).

For successful sessions, the Success Analyst uses TraceCard structure to derive both preserve and prune patches. role_hint and span structure help identify final-reply patterns and effective workflows, while top_cost_spans and redundant_tool_calls identify expensive, repeated operations. A prune patch is only admitted if it names a target span, supplies a counterfactual explanation for why removing that step would not change the outcome, and states the learned rule as avoidance behavior. The paper’s abbreviated JSON example encodes exactly this logic: "Read each input file once and cache its contents; avoid redundant re-reads." with target_span: "span-7" and a counterfactual that the second read_file('input.xlsx') returned identical content and did not influence the final answer (Yuan et al., 26 Apr 2026).

For failed sessions, TraceCard is used mainly for localization. failed_or_repaired can highlight suspect steps, while top_cost_spans and role_hint help locate key decisions. The Error Analyst then combines TraceCard context with oracle tools such as inspect_mismatches and read_golden_snippet to emit a repair patch containing an action: "repair", a failure taxonomy label such as T4 placeholder mismatch, and evidence grounded in both the trace and the oracle. TraceCard thus supplies causal neighborhood and error localization, while the oracle supplies ground truth (Yuan et al., 26 Apr 2026).

In the merge stage, TraceCard-derived patches are ordered by priority: repair first, prune second, preserve third, with preserve singletons dropped unless they appear in at least two trajectories. Conflicts are resolved by the precedence relation repair overrides prune and prune overrides preserve. The final SKILL.md has sections for Trigger, Workflow, Stop rules, Artifact checklist, and Cost control; the Cost control section is entirely derived from prune patches, and therefore directly from the cost-aware insights encoded in TraceCards (Yuan et al., 26 Apr 2026).

4. Counterfactual pruning and empirical findings

The paper places unusual emphasis on counterfactual reasoning. Mechanically, the Success Analyst inspects top_cost_spans, chooses a candidate span such as a repeated file read, checks whether that span appears in a redundancy cluster, and asks whether the step introduces content unique to the final output. If the step is high-cost, part of a redundancy cluster, and apparently non-unique with respect to final content, the analyst encodes the claim that omitting it would still have preserved success. This counterfactual is stored explicitly in the prune patch rather than left implicit in the pruning decision (Yuan et al., 26 Apr 2026).

Experiments on SpreadsheetBench tie TraceCards directly to quality and cost outcomes. The setup uses 50 sampled professional-subset tasks from SpreadsheetBench, with a split of 10 evolve, 30 held-out, and 10 dev; each run yields one TraceCard. The principal ablations are Full CostCraft, No-prune, No-cost-attribution, and No-CF. On the 30 held-out SpreadsheetBench tasks, removing cost fields from TraceCards more than doubles median cost uplift on successful tasks from +22% to +49%, regressions rise from 4 to 6, and 5 of those regressions are catastrophic with Q=0. Discarding prune patches triples regressions from 4 to 13, even though median cost remains similar at +15% versus +21%; 8 of the 13 regressions produce no output because the agent “runs but never writes.” Quality preservation drops monotonically as signals are removed, with Full CostCraft at 86.7% preservation and No-prune at 56.7% (Yuan et al., 26 Apr 2026).

A second experimental result is the cross-benchmark asymmetry on SkillsBench. The SpreadsheetBench-trained skill is applied to 30 non-spreadsheet SkillsBench tasks spanning data processing, scientific computing, document analysis, and code generation. Cost decreases on 16 of 27 valid task pairs, and median cost falls from $0.105` to `$0.071, approximately a 32% decrease. Quality changes are mixed: 2 tasks improve and 3 regress. The interpretation given in the paper is asymmetric transfer. Prune rules transferred well because they target “universal waste patterns,” such as reading each input file once and caching it. Preserve rules transferred poorly because they encode SpreadsheetBench-specific conventions and caused regressions on unrelated task types including PDF editing, energy pricing, and fraud detection (Yuan et al., 26 Apr 2026).

These findings sharpen the conceptual role of TraceCard. It is not merely a container for observability metadata; it is the vehicle by which cost attribution, redundancy evidence, and counterfactual admissibility become operational in skill learning. A plausible implication is that TraceCard’s strongest contribution lies in separating efficiency knowledge from benchmark-specific behavioral conventions (Yuan et al., 26 Apr 2026).

5. Instrumentation, compilation, and design decisions

TraceCards are produced by ClawTrace instrumentation. ClawTrace is implemented as an OpenClaw plugin that registers 8 event hooks: session_start, session_end, llm_input, llm_output, before_tool_call, after_tool_call, subagent_spawning, and subagent_ended. Events are batched in memory and flushed on shutdown as JSON to POST /v1/traces/events. The ingest API is not framework-specific: any agent harness that posts conformant JSON can produce valid TraceCards. Downstream, a graph lakehouse pipeline built from PuppyGraph and Iceberg materializes “silver tables” per event type, and sub-agent call graph reconstruction is handled by a childSessionKey → parentSpanId map (Yuan et al., 26 Apr 2026).

Compilation from trace events to TraceCard is deterministic. The compiler computes per-span and total token counts, computes cache-aware per-span and total USD cost using provider pricing, ranks spans by cost to select top_cost_spans, applies Levenshtein-based clustering to tool calls to produce redundant_tool_calls, computes Jaccard overlap for sub-agent usage heuristics, and detects failure or repair signals in tool results. The output is a compact YAML card rather than a full transcript (Yuan et al., 26 Apr 2026).

Three design decisions are made explicit. First, compactness: cards are about 1.2–1.8 kB, and the paper states that dozens fit in an LLM context. Second, extensibility: the schema has no CostCraft-specific fields, so any distillation pipeline can consume it; heuristic fields can be extended to include latency, energy, or new role hints. Third, standardization: the representation follows OpenTelemetry-like notions such as spans and parent-child structure but adds cost-aware fields, while the JSON ingest API is intended to standardize trace capture across agent frameworks (Yuan et al., 26 Apr 2026).

The practical workflow presented in the paper follows directly from this architecture. An agent is instrumented with ClawTrace or a conformant event emitter, one YAML TraceCard is generated per session, practitioners inspect cost and redundancy signals, successful sessions yield prune candidates while failed sessions yield repair candidates, and the resulting patches are merged into SKILL.md. The evolved skill is then redeployed, re-instrumented, and compared against baseline TraceCards. This places TraceCard at the center of an iterative, trace-driven skill-evolution loop (Yuan et al., 26 Apr 2026).

6. Related trace-card formulations in adjacent work

Although TraceCard is defined operationally in ClawTrace, adjacent research uses closely related “card” or card-like trace abstractions for other domains. OpenTracer, a dynamic transaction trace analyzer for Ethereum, captures every executed EVM instruction, constructs a function-level invocation tree, decodes storage accesses, and supports invariant extraction from transaction histories. Its accompanying technical synthesis explicitly proposes per-transaction or per-contract “trace card” views built from call trees, decoded parameters and returns, storage changes, and invariant summaries, suggesting a blockchain analogue of the TraceCard idea (Chen et al., 2024).

Budgeted Dynamic Trace Structures introduces a data-structural framework for maintaining rooted trace graphs and append-only histories under an explicit byte or token budget. Its central compaction mechanism is summary-plus-suffix replacement: summarize older history, retain the longest possible recent suffix under budget, and preserve valid truncation boundaries. The paper’s synthesis explicitly presents BDTS as a blueprint for a TraceCard system in which traces remain queryable while fitting strict token budgets, and reports compaction from 350k–2.71M approximate tokens to 1,048–4,120 approximate tokens, as well as tokenizer-measured reductions from 3,359–3,360 tokens to 432–433 tokens (Alpay et al., 20 May 2026).

TraVista, a tool for debugging single-request performance issues in distributed systems, is not named TraceCard, but it develops a complementary design pattern: a single selected trace is contextualized with aggregate metric, temporal, and structure data. Its extended Gantt view overlays latency distributions, temporal “molehills” for contention, and rarity encodings for events and edges. This suggests a performance-debugging interpretation of trace cards in which a single trace becomes intelligible only when aggregate context is visually attached to it (Anand et al., 2020).

TRACE uses the term in yet another way. Its technical synthesis treats “TraceCard” as a conceptual provenance or attribution mechanism for LLM-agent trajectories, realized through a two-channel behavioral watermark that is distortion-free in action choices, self-synchronizing under deletion, and invariant under rewriting in its tally channel. In that formulation, the “card” is not a YAML session summary but a verifiable provenance profile embedded in the trajectory itself (Gao et al., 9 Jul 2026).

Taken together, these neighboring formulations indicate that “TraceCard” has become a useful organizing idea for compact, structured, and task-specific representations of execution traces. The concrete ClawTrace TraceCard is cost-aware and distillation-oriented; the adjacent formulations emphasize invariants, budgeted summarization, performance debugging, or provenance. This suggests that the term now denotes a family resemblance rather than a single universal schema.

7. Limitations and future directions

The limitations of TraceCard in its original formulation are explicit. All reported results use a single backbone, openai-codex/gpt-5.4, with seed=0; the paper provides no multi-seed variance analysis and notes that regressions may be seed-sensitive. The evolve set is small, with only 10 evolve tasks, and only 2 prune rules were learned: skipping workspace memory files when the task is self-contained, and reading each input file once and caching it. Those rules match only 2 of 17 successful held-out tasks. Heuristic fields are also unevenly validated: sub_agents.output_used_in_final was not exercised because no sub-agents were spawned, and failed_or_repaired was used but not deeply analyzed across diverse failure types. The plugin implementation is currently “OpenClaw-only,” and portability of the ingest API to other harnesses is not fully validated. Finally, latency, energy consumption, and other operational metrics are absent from the current schema, and the cost model must be reconfigured when provider pricing changes (Yuan et al., 26 Apr 2026).

The stated research directions follow naturally from those constraints. The paper identifies scaling the evolve set as a route to richer prune rules and stronger cost compression, multi-seed evaluation as a way to test whether the protective effect of prune rules is stable, and broader use of TraceCards in other pipelines such as SkillRL, ReasoningBank, CoEvoSkills, and AutoSkill. It also proposes closed-loop self-evolution, in which TraceCards generated from skill-equipped agent runs feed new waste patterns and failure modes back into CostCraft. Extended instrumentation to include latency, energy, or other resource metrics is another direct extension, as is validating sub-agent usage heuristics on multi-agent workloads (Yuan et al., 26 Apr 2026).

A final misconception is that TraceCard already constitutes a mature universal standard. The paper instead presents it as open infrastructure for cost-aware agent research. That phrasing is important. It indicates a generalizable intermediate representation, but not yet a settled one. The combination of deterministic compilation, compact YAML encoding, explicit cost attribution, and redundancy-aware skill distillation makes TraceCard a concrete and influential proposal; its broader standardization remains a live research problem (Yuan et al., 26 Apr 2026).