Total Cost of Agency: Exact Attribution of Memory Injection Cost in Multi-Agent LLM Workflows
This presentation examines a novel cost-accounting framework for multi-agent language model workflows. The authors isolate memory injection—tokens retrieved from earlier workflow steps and fed back into downstream prompts—as a distinct, measurable cost component that conventional observability systems hide within aggregate input-token counts. Using exact tokenizer-based attribution and a benchmark of 200 enterprise tasks spanning five workflow depths, the work demonstrates that memory injection accounts for up to 27.6% of billed cost at depth six, grows nearly linearly with workflow depth under bounded retrieval, and responds predictably to retrieval-capacity tuning, offering a new lever for cost optimization separate from model-tier routing.Script
When you chain language model agents together, every downstream node pays again for the outputs generated upstream. Providers bill those tokens twice: once when they're produced, and again when they're injected as context into the next specialist's prompt.
The authors propose Total Cost of Agency, a decomposition that separates memory injection from other input costs. They measure it exactly by tokenizing each node's prompt twice: once with just the base components, and again after adding retrieved memory. The difference is the injection volume, captured with the provider's actual tokenizer rather than word-count proxies.
Across 200 enterprise tasks, injection grows from 143 tokens at depth two to 754 at depth six, following a nearly linear trend with an R-squared above 0.99. At the deepest workflows, memory injection accounts for more than a quarter of total billed cost, even though inference remains the dominant expense.
Reducing retrieval capacity from 32 entries to 2 cuts injected tokens by nearly 29 percent and lowers cost by 6.7 percent in a single-seed comparison. This establishes memory capacity as a controllable cost lever, separate from model-tier routing, though the accuracy impact and the optimal operating point remain open questions.
Graph rewrites—node fusion, reordering, shared namespaces—are effectively cost-neutral in this harness. The full optimized system actually costs more per task because the tier assigner escalates accuracy-sensitive nodes from the small tier, demonstrating that routing dominates total cost and that memory optimization is a complement, not a substitute.
The strongest result is an accounting claim: memory-generated context is billed, depth-dependent, and invisible unless instrumented separately. The authors leave open how much of this cost survives prompt caching and where the capacity-performance frontier lies. To explore these methods yourself and create videos like this one, visit EmergentMind.com.
