Can AI Agents Really Complete RTL-to-GDS? Lessons from Benchmarking Tool-Interactive EDA Workflows

Abstract: LLM-driven agent systems have emerged as a promising paradigm for electronic design automation (EDA), demonstrating strong potential for automating complex design workflows. However, existing evaluations primarily examine individual LLMs on isolated EDA tasks, providing limited insight into how different agent systems perform across complete EDA flows. In this work, we present FluxBench, a systematic evaluation of AI agents on end-to-end EDA workflows under unified prompts, tool environments, and technology library settings. Our evaluation covers representative scenarios, including RTL generation with open-source toolchains and an RTL-to-GDS flow using closed-source commercial EDA tools for industrial applications. Through these workflows, we assess agents' capabilities in RTL code generation, iterative repair, tool-feedback utilization, logic synthesis, placement and routing (P&R), and Engineering Change Order (ECO) automation. To further characterize the efficiency of agent systems, we introduce Token ROI, a cost-efficiency metric that measures effective improvements in EDA artifacts relative to token usage and runtime cost. Experimental results show that, even when built on the same foundation model, different agent system architectures can exhibit performance gaps of up to 86.27%. Moreover, among systems with comparable task performance, Token ROI can differ by as much as $105.92\times$. In the RTL-to-GDS flow using PicoRV32 as a case study, FluxEDA achieves an end-to-end score of up to 97.94, outperforming Claude Code equipped with domain-specific EDA skills by up to $8.39\times$. These results indicate that domain-specific skills alone are insufficient to improve agent performance in large-scale EDA scenarios. Instead, both agent system design and foundation model capability play critical roles in enabling effective automated EDA workflows.

• The paper introduces FluxBench, a process- and cost-aware benchmark spanning RTL generation and industrial-style RTL-to-GDS flows, with Token ROI combining task quality, token cost, and execution time.
• The paper finds that FluxEDA’s MCP-based architecture completes all RTL-to-GDS stages across nine model and timing configurations, reaching scores of 97.94 under loose timing and 90.99 under tight timing.
• The paper shows that agent architecture can create performance gaps of up to 86.27%, Token ROI can vary 105.92× among similarly performing systems, and EDA skills alone cannot reliably resolve tool-interaction failures.

Motivation and scope

LLM-based agents have moved from passive code assistants toward autonomous participants in electronic design automation (EDA) flows, but existing benchmarks such as VerilogEval, RTLLM, RealBench, and CVDP primarily evaluate static, single-turn RTL generation outcomes (2305.15412, Zhong et al., 2023, Jin et al., 22 Jul 2025, Pinckney et al., 17 Jun 2025). They provide little support for measuring multi-round tool interaction, token consumption, or process-level behavior in long-horizon flows. This paper introduces FluxBench, a systematic benchmark for evaluating AI agents on end-to-end EDA workflows under unified prompts, containerized tool environments, and consistent technology library settings. The evaluation spans two regimes: RTL generation from natural-language specifications using open-source toolchains (spec-to-RTL subsets of RealBench and CVDP), and an industrial-style RTL-to-GDS flow on PicoRV32 at a 55 nm node using closed-source commercial tools, covering logic synthesis, place-and-route (P&R), engineering change order (ECO), and final GDS/report generation.

A central motivation is cost. The paper quantifies how agentic EDA inflates resource consumption: moving from single-turn RTL reasoning to multi-turn RTL chat to agentic RTL design raises the input/output token ratio from 1.80 to 12.38, average token usage from 1.02K to 213.82K, and average task cost from $0.007 to $0.734. This motivates a process-aware, cost-aware evaluation methodology rather than final-outcome scoring alone.

Benchmark construction

All tasks execute in independent agent sessions within a containerized environment that provides a complete open-source EDA toolchain; commercial tools and restricted PDKs are mounted read-only from the user environment rather than redistributed, balancing reproducibility with licensing compliance. Agents see only task descriptions, public inputs, and editable files; hidden tests, reference solutions, and scoring data are withheld until after completion. Missing tools or process data are recorded as explicit environment failures rather than masked by substitute flows.

The RTL generation suite draws easy/hard subsets from RealBench (AES, e203_hbirdv2, SDC families) and 112 CVDP tasks (94 easy, 18 hard). The RTL-to-GDS task uses PicoRV32 under two SDC timing constraints — loose (350 MHz) and tight (700 MHz) — with a fixed die size.

Token ROI metric

The paper's principal methodological contribution is Token Return on Investment (Token ROI), defined as the normalized task score $S_k \in [0,1]$ divided by the product of total token-based cost $C_{\mathrm{token}}$ (summing input, output, cached, reasoning, and tool-observation tokens weighted by unit price) and wall-clock execution time $C_{\mathrm{time}}$. For cross-task comparability, both investments are clipped to task-specific best/worst reference bounds and linearly mapped to $[1,100]$ (tokens) and $[1,5]$ (time), yielding a normalized ROI. Task returns are structured: RTL generation scores weight functional pass rate at 80% plus compile/synthesis pass rates; RTL-to-GDS uses sequential gates over synthesis (clock gating, netlist generation), P&R (I/O placement, standard-cell placement, CTS, routing), physical-aware ECO, and a final result score averaging area, setup/hold WNS/TNS, and power against fixed normalization references. This gated structure means later stages contribute only when all preceding gates pass, penalizing agents that fail mid-flow.

Main results

Three agent architectures are evaluated: FluxEDA-Agent (an MCP-based, EDA-oriented framework), Claude Code (a general-purpose CLI coding agent), and CC+EDA Skills (Claude Code augmented with domain-specific IC/EDA skills), each paired with DeepSeek-V4 Flash, GLM-5.2, and Claude Sonnet 5.

RTL generation. CVDP-Easy is nearly saturated (six of nine configurations score 100.00), while RealBench remains difficult — the best scores are 57.10 (easy) and 53.58 (hard), both from FluxEDA with GLM-5.2, which is also the sole configuration achieving 100.00 on CVDP-Hard. Efficiency varies dramatically: DeepSeek-V4 consistently yields costs below ~18 RMB with far higher ROI than the other backends, whereas architectures with comparable scores differ in normalized ROI by up to 105.92×. FluxEDA with DeepSeek-V4 improves vanilla Claude Code's RealBench-Hard score from 22.00 to 32.74 while reducing runtime across all four subsets. Notably, even on identical foundation models, architecture choice alone produces performance gaps up to 86.27%.

RTL-to-GDS. FluxEDA dominates: with DeepSeek-V4 it achieves 97.94 (loose) and 90.99 (tight) with the highest ROIs (39.91 and 48.06); with GLM-5.2 it reaches 93.70 under the tight constraint. All three backends paired with FluxEDA exceed 90 points under both constraints, indicating robustness to model choice. By contrast, baselines are erratic: Claude Code with DeepSeek-V4 scores only 13.33 despite lowest cost/runtime, and CC+EDA Skills collapses to 11.67 with Claude Sonnet 5 despite that backend scoring 87.05 under vanilla Claude Code. FluxEDA outperforms Claude Code with domain-specific EDA skills by up to 8.39× end-to-end.

Observations

Skills have limited impact. Domain-specific EDA skills improve performance on easy RTL tasks and the 350 MHz flow but offer no advantage — sometimes negative — under the 700 MHz constraint. The paper concludes that skills alone are insufficient for realistic industrial flows; foundation-model capability and system architecture dominate. This is a notable claim against the current practice of augmenting agents with curated domain knowledge.

Tcl compatibility is the dominant PR failure mode. Innovus Tcl command compatibility errors account for 31.7% of all P&R-stage error events, concentrated in Claude Code runs (48.7% with DeepSeek-V4, 28.0% with GLM-5.2), reduced to 10.0%/7.7% with skills, and eliminated entirely (0.0%) under FluxEDA. Structured workflows and wrapped tool interfaces therefore suppress low-level interface mismatches, though failures shift toward higher-level API session and design-convergence issues rather than disappearing.

MCP-based architecture improves process stability. FluxEDA completes every stage of the RTL-to-GDS flow — including physical-aware ECO — across all nine model/constraint combinations. CLI-based configurations frequently fail at I/O placement, CTS, or ECO, and skills do not fix these structural weaknesses attributable to weak state management, unstructured feedback, and limited error recovery. The paper acknowledges that MCP incurs additional token overhead through structured tool descriptions and feedback, but argues this is acceptable given long execution chains and low error tolerance in physical implementation.

Limitations and open questions

Several caveats bear directly on the results. The RTL-to-GDS evaluation rests on a single design (PicoRV32) at a single technology node (55 nm) with one commercial toolchain, so conclusions about architectural robustness may not transfer to larger SoCs, advanced nodes, or other vendors' tools. The Token ROI normalization depends on task-specific reference bounds ($C^{\mathrm{best}}$, $C^{\mathrm{worst}}$, and their time analogues), whose provenance and sensitivity are not analyzed; different bounds could reorder efficiency rankings. The finding that skills fail under tight timing constraints is established empirically for one skill set and one constraint pair, leaving open whether better-curated or tool-version-aware skills could close the gap. Finally, the observation that FluxEDA merely shifts failure modes from Tcl compatibility to API-session and convergence issues suggests the deeper question of whether any current agent architecture can reliably handle design-convergence failures, which remain unsolved here.

Conclusion

FluxBench provides the first systematic, cost-aware benchmarking of AI agents across realistic end-to-end EDA flows, combining open-source RTL generation tasks with closed-source-tool RTL-to-GDS implementation and introducing Token ROI as a joint measure of quality, token cost, and runtime. Its central empirical findings — that agent architecture alone induces gaps up to 86.27%, that Token ROI can vary 105.92× among equally performing systems, and that domain-specific skills do not substitute for strong models or structured MCP-based tool interfaces — collectively argue that process-level stability, not just final QoR, should be the primary target for advancing automated EDA.