TraceLab: Coding-Agent Serving Workloads

• TraceLab is a trace-driven analysis platform that records full coding-agent sessions by capturing multi-step, tool-invoking loops and dynamic context growth.
• The platform quantifies key workload properties such as context expansion, cache hit rates, tool-call latencies, and cost trade-offs for coding agents.
• Its unified, step-level schema supports simulation, cache analysis, and targeted serving optimizations to improve LLM-driven coding-agent systems.

TraceLab is a trace-driven analysis platform and dataset built to expose, for the first time, the end-to-end serving workload of real coding agents—Claude Code and OpenAI Codex—as they are used “in the wild” (Zhu et al., 29 Jun 2026). It records full sessions rather than conversational or single-turn completion traffic, capturing multi-step, tool-invoking loops with growing context and human pacing. The resulting analysis characterizes coding-agent workloads in terms of context growth, tool behavior, cache hit and miss patterns, and cost and latency breakdowns, and it highlights concrete opportunities for optimizing LLM serving for agentic code tooling (Zhu et al., 29 Jun 2026). The name should be distinguished from "SonoTraceLab," which is an acoustic modelling system for biosonar and in-air sonar simulation rather than a coding-agent serving trace (Jansen et al., 2024).

1. Motivation, scope, and goals

TraceLab is motivated by the observation that coding agents execute multi-step “agentic loops” in which an LLM reads its accumulated context, emits reasoning and natural-language output, invokes tools such as shell commands, file read/write, and custom agents, appends tool results to the context, and repeats until a final answer is returned (Zhu et al., 29 Jun 2026). These loops can generate very large contexts, measured in hundreds of thousands of tokens, while emitting only short outputs, measured in hundreds of tokens, per step. They also interleave LLM calls with tool calls whose latencies range from milliseconds to hours, and human users take breaks between requests, causing long idle gaps.

Within that setting, TraceLab defines four explicit goals: to collect a large, anonymized, cross-provider trace of day-to-day coding-agent sessions; to normalize it into a unified, step-level schema suitable for serving-system analysis; to characterize key workload properties, including context growth, tool behavior, cache hit and miss patterns, and cost and latency breakdowns; and to highlight actionable opportunities for optimization while sharing the data, trace-collection pipeline, and analysis code with the community (Zhu et al., 29 Jun 2026).

A central implication of this framing is that coding-agent serving is not adequately represented by existing public traces and benchmarks that capture conversational or code-completion traffic. TraceLab treats long-horizon, tool-heavy, human-paced execution as the primary workload of interest. This suggests that serving bottlenecks for coding agents are likely to differ materially from those of chat and single-turn completion systems.

2. Dataset composition and normalized schema

TraceLab collects logs from 43 developers over eight months, from Sep 2025 to Jun 2026, spanning 2,676 Claude Code sessions and 1,589 Codex sessions, for 4,265 total sessions (Zhu et al., 29 Jun 2026). In aggregate, the trace contains 357,161 LLM steps and 432,510 tool calls, reading 52.56 B “prefix” (cached) tokens, appending 2.34 B “append” (fresh) tokens, and generating 186.9 M output tokens. At public list prices, this corresponds to roughly \$40.4 K of API spend.

Quantity	Value
Developers	43
Time span	Sep 2025–Jun 2026
Claude Code sessions	2,676
Codex sessions	1,589
Total sessions	4,265
LLM steps	357,161
Tool calls	432,510

The normalized step schema uses one row per LLM invocation plus its tool calls. It records prefix tokens $P$, append tokens $A$, output tokens $O$, a time-ordered list of sub-events, and tool-execution latency (Zhu et al., 29 Jun 2026). For Claude, the schema defines

$$P_{\text{Claude}} := \text{cache\_read\_input\_tokens}$$

and

$$A_{\text{Claude}} := \text{input\_tokens} + \text{cache\_creation\_input\_tokens}.$$

For Codex, it defines

$P_{\text{Codex}} := \text{cached\_input\_tokens}$

and

$$A_{\text{Codex}} := \text{input\_tokens\_total} - \text{cached\_input\_tokens}.$$

The time-ordered sub-events include user messages, tool calls, reasoning, and text output with timestamps. Tool-execution latency is directly reported by Codex or derived by Claude logs. Pricing breakdown further categorizes cost into prefix, append, and output tokens (Zhu et al., 29 Jun 2026).

This schema is significant because it places token accounting, temporal ordering, and tool execution in a single step-level representation. A plausible implication is that TraceLab is intended not merely as a descriptive corpus, but as a workload substrate for serving-system simulation, cache analysis, and latency decomposition.

3. Autonomous loops and context dynamics

TraceLab reports that a session averages 9.2 requests, with $p99 = 137$, 8.9 user-initiated steps, 73.6 tool-initiated steps, and 101.4 tool calls (Zhu et al., 29 Jun 2026). To resolve a single request, the agent takes on average 8.8 LLM invocations and 10.8 tool calls. Most steps are tool-initiated, so the loop runs without further user input once started.

Context growth is correspondingly persistent. On more than 99% of steps, the total-input length, defined as prefix plus append, grows. Negative changes occur on 0.4% of Claude steps and 3.4% of Codex steps, and these are broken down into “micro” ($\leq 1$ k tokens), “ordinary” (1 k–64 k), and “major” ($> 64$ k) reductions (Zhu et al., 29 Jun 2026).

TraceLab also identifies “context compaction” events, defined as near-limit ($A$0 of session’s max) sudden drops of at least 64 k tokens that do not recover for at least 3 steps. These occur in 9.7% of sessions and average 3.7 compactions per affected session. Codex sessions compact more often, at 18.4%, than Claude sessions, at 4.5% (Zhu et al., 29 Jun 2026).

These results characterize coding-agent sessions as long autonomous loops with monotone or near-monotone context accumulation interrupted by occasional structural resets. This suggests that serving policies tuned for short conversational turns may mischaracterize both the temporal granularity and the memory behavior of coding-agent execution.

4. Long-context, short-output structure

One of TraceLab’s most prominent findings is the “long context, short output” pattern. Claude steps read a median of 126 k prefix tokens and append 857 fresh tokens; Codex reads 115 k and appends 886. Outputs are short: the median is 252 tokens for Claude and 184 for Codex, and the $A$1 remains under 1.7 k tokens (Zhu et al., 29 Jun 2026). Frequent tool calls break reasoning into short bursts.

The input structure is also bimodal. Very small prefixes, below 1 k, correspond to cache misses or session starts and carry large appends, with median greater than 70 k tokens. Once the prefix passes approximately 32 k tokens, the append collapses to below 1 k tokens in more than 90% of steps (Zhu et al., 29 Jun 2026).

These properties matter for serving because they separate two distinct regimes: session-start or cache-miss steps with large append bursts, and steady-state steps with large cached prefixes and very small fresh appends. TraceLab explicitly connects this distributional structure to append-length-aware prefill scheduling. A plausible implication is that the efficiency frontier for coding-agent serving depends less on maximizing output throughput and more on balancing prefill cost, cache reuse, and low-overhead execution for short incremental steps.

5. Tool-call behavior and latency structure

Tool activity is both diverse and skewed. Claude uses approximately 54 distinct tools, while Codex uses approximately 31. In both cases, the top 3 tools account for more than 80% of calls: examples given are Bash, Read, and Edit for Claude, and exec_command, write_stdin, and apply_patch for Codex (Zhu et al., 29 Jun 2026).

Latency is heavily tailed. Calls below 1 s constitute approximately 70% of calls but less than 1% of total tool-time, whereas calls above 1 min constitute approximately 4% of calls but account for more than 85% of tool time (Zhu et al., 29 Jun 2026). For Codex specifically, tool calls show non-execution overhead. Comparing end-to-end latency $A$2 and runner-reported internal time $A$3, the residual

$A$4

averages 1.11 s and has $A$5 s, driven by security- or approval-heavy commands.

TraceLab uses these measurements to motivate lower-overhead tool calling and semantic-aware tool-latency prediction (Zhu et al., 29 Jun 2026). The paper’s observation is that eviction policies based solely on tool name or static histograms are insufficient because tool latencies vary heavily within each tool type. This suggests that serving stacks for coding agents may need latency models conditioned on arguments, command patterns, and recent per-session or per-tool online latency observations rather than only on coarse tool identity.

6. Prefix cache behavior, TTL trade-offs, and API spend

TraceLab reports an overall token-weighted prefix hit rate of approximately 95.7% (Zhu et al., 29 Jun 2026). User-initiated steps have lower hit, at 84%, due to human idle gaps, while tool-initiated steps hit at approximately 97.5%. Idle gaps greater than 5 min begin to incur significant misses; beyond 1 h almost no prefix survives.

Only 19% of append tokens are truly “fresh”; the remainder is redundant re-prefill. Prefill amplification, defined as total appends divided by fresh tokens, is 5.3× overall, 8.1× for Claude, and 3.9× for Codex (Zhu et al., 29 Jun 2026). A simple eviction-time sweep shows that increasing cache TTL from 1 min to 1 h raises the achievable hit rate from 85.4% to 98.6%, but increases the KV-storage ratio

$A$6

from 0.74 to 5.07.

An upper-bound consumer view estimates that avoiding human-gap misses on user-steps could save 45.9% of append tokens and 12.8% of cost (Zhu et al., 29 Jun 2026). The same analysis provides operator-side guidance: a 5-minute TTL captures approximately 94% of tokens with less than 2× KV storage overhead, while a 1-h TTL reaches approximately 99% hit at approximately 5× storage.

A common misconception is that high cache hit rates eliminate prefill inefficiency. TraceLab shows the opposite: even with approximately 95.7% token-weighted prefix hit, redundant re-prefill remains substantial because only 19% of append tokens are truly fresh. The practical issue is therefore not merely whether cache hits occur, but how misses are induced by human pacing, tool delays, and eviction-time policy.

7. Serving optimizations, release, and research role

TraceLab does not ship a full end-to-end serving system; its emphasis is workload characterization and the quantification of upper-bound savings and trade-offs (Zhu et al., 29 Jun 2026). The paper identifies several concrete optimization opportunities. For lower-overhead tool calling, it points to lower-latency auto-approval, fusing multiple tool calls into a single RPC, and co-locating LLM and tool runtimes to eliminate scheduling overheads. For append-length-aware prefill scheduling, it proposes routing long-append steps to high-throughput, throughput-optimized prefill engines and routing short-append steps to low-latency, low-parallelism in-memory prefill paths.

For KV management, the paper proposes semantic tool-latency prediction, TTL tuning, gap-duration predictors, and an “agent harness” that automatically pings the cache or issues a no-op LLM call before a long user break to avoid eviction (Zhu et al., 29 Jun 2026). It also raises an output-cache policy question: some providers resend prior outputs as append, while others cache them, and TraceLab frames this as a dynamic choice based on output length, cache pressure, and network or storage bandwidth.

The release includes a sanitized trace with anonymized session, user, and tool IDs, token counts, timestamps, and tool latencies, together with the end-to-end collection, normalization, and analysis pipeline at https://github.com/uw-syfi/TraceLab.git, and a browsable project website with summary dashboards and usage instructions at https://tracelab.cs.washington.edu (Zhu et al., 29 Jun 2026). Users can replay the trace, generate normalized step-level CSVs, reproduce all tables and figures in the paper, and extend the analysis or build serving-system simulations and optimizations on this realistic coding-agent workload.

In the arXiv record, TraceLab is therefore positioned as the first large-scale, cross-provider look into real coding-agent serving workloads (Zhu et al., 29 Jun 2026). Its distinctive contribution is not a new model architecture, but a workload characterization substrate: long autonomous loops, long contexts with short outputs, diverse and heavily tailed tool calls, high but imperfect prefix cache hit rates, and human-gap effects. This suggests that future LLM serving stacks for coding agents will be shaped as much by trace realism and systems co-design as by model-side improvements alone.