ORFS-agent: Tool-Using Agents for Chip Design Optimization (2506.08332v1)

Abstract: Machine learning has been widely used to optimize complex engineering workflows across numerous domains. In the context of integrated circuit design, modern flows (e.g., going from a register-transfer level netlist to physical layouts) involve extensive configuration via thousands of parameters, and small changes to these parameters can have large downstream impacts on desired outcomes - namely design performance, power, and area. Recent advances in LLMs offer new opportunities for learning and reasoning within such high-dimensional optimization tasks. In this work, we introduce ORFS-agent, an LLM-based iterative optimization agent that automates parameter tuning in an open-source hardware design flow. ORFS-agent adaptively explores parameter configurations, demonstrating clear improvements over standard Bayesian optimization approaches in terms of resource efficiency and final design metrics. Our empirical evaluations on two different technology nodes and a range of circuit benchmarks indicate that ORFS-agent can improve both routed wirelength and effective clock period by over 13%, all while using 40% fewer optimization iterations. Moreover, by following natural language objectives to trade off certain metrics for others, ORFS-agent demonstrates a flexible and interpretable framework for multi-objective optimization. Crucially, RFS-agent is modular and model-agnostic, and can be plugged in to any frontier LLM without any further fine-tuning.

• The paper introduces an LLM-based iterative optimization agent that automates parameter tuning in chip design workflows.
• It leverages natural language instructions and statistical techniques to optimize performance, power, and area metrics over fewer iterations than traditional methods.
• The approach is model-agnostic and cost-efficient, enabling flexible, constrained, and multi-objective optimizations within open-source EDA toolchains.

ORFS-agent (2506.08332) introduces an LLM-based iterative optimization agent designed to automate parameter tuning within an open-source integrated circuit (IC) design flow. The paper tackles the practical challenge of optimizing complex electronic design automation (EDA) workflows, where the large number of configuration parameters significantly impacts key design metrics like power, performance, and area (PPA). Unlike traditional Bayesian optimization (BO) methods which can be context-limited and require explicit objective functions, ORFS-agent leverages the reasoning and tool-use capabilities of LLMs to navigate this high-dimensional optimization space.

The core idea is to integrate an LLM agent into the OpenROAD-flow-scripts (ORFS) environment [ORFS]. OpenROAD is an open-source, end-to-end RTL-to-GDSII toolchain, making it a suitable platform for open research due to its transparency, reproducibility, and extensibility via APIs. ORFS provides JSON/Tcl wrappers around OpenROAD, simplifying large-scale experiments. The agent's objective is to iteratively modify configuration parameters (like core utilization, clock period, placement/routing settings) to improve target design metrics.

The ORFS-agent operates through an iterative loop, conceptualized in Figure 2. Each iteration involves several stages:

1. RUN: Launch multiple parallel ORFS runs, each with a distinct set of parameters suggested by the agent.
2. READ: Gather logs and metrics from the completed (or timed-out) runs. This includes final metrics like routed wirelength (WL) and effective clock period (ECP), as well as surrogate metrics from earlier stages (e.g., clock tree synthesis - CTS) that are available even if the run times out.
3. COLLATE: Consolidate the data from all runs into a unified dataset.
4. INSPECT: Analyze the collated data. The agent can use INSPECT tools (function calls) to examine distributions, correlations, or other statistical properties of the results and parameters.
5. OPTIMIZE: Based on the analysis, propose new parameter sets for the next iteration. The agent can use OPTIMIZE tools (like a Gaussian Process model with Expected Improvement) to guide the search for promising configurations.
6. AGGLOMERATE (optional): If the OPTIMIZE step generates more candidate parameter sets than the number of parallel runs available, AGGLOM tools are used to select a diverse or high-quality subset of candidates.
7. ALTER: Modify the ORFS configuration file (config.mk) and SDC file with the selected new parameter sets for the next batch of runs.

This loop continues until a stopping criterion, typically a total number of iterations (and thus, total ORFS runs), is met. The LLM maintains a GLOBALCONTEXT which is initialized by the user prompt (specifying PDK, circuit, objective, parameter domains, etc.) and updated iteratively with summarized results.

A key implementation aspect is the reliance on the LLM's function-calling ability to interact with external tools. The paper categorizes these tools into:

• INSPECT tools: Analyze data (e.g., InspectDistribution to check relationships between parameters and metrics).
• OPTIMIZE tools: Suggest new parameter points (e.g., GP using Gaussian Processes and acquisition functions like Expected Improvement).
• AGGLOM tools: Select a subset of points from a larger pool (e.g., EntropySelect for diversity).

These tools are implemented using standard libraries (like scikit-learn, numpy) wrapped in functions that the LLM can call by providing arguments based on its current context and reasoning. This modular approach allows the LLM to leverage established numerical methods without needing to perform complex calculations internally.

The practical application demonstrated is optimizing PPA metrics for various circuits (IBEX, AES, JPEG) across two technology nodes (SKY130HD and ASAP7) using the OpenROAD flow. The agent optimizes a set of 4 or 12 parameters influencing stages like placement, CTS, and routing (Table 3).

The experimental results highlight several practical benefits:

• Improved Design Metrics: ORFS-agent improves routed wirelength and effective clock period significantly compared to the default ORFS flow parameters.
• Resource Efficiency: It achieves results comparable to or better than OR-AutoTuner (a BO-based method for ORFS) while using substantially fewer optimization iterations (e.g., 600 iterations for ORFS-agent vs. 1000 for OR-AutoTuner in the 12-parameter setting).
• Flexible Objectives: The agent can handle single-objective optimization (minimizing WL or ECP) and multi-objective optimization (minimizing a weighted sum, like $\frac{WL}{WL_{\alpha}} + \frac{ECP}{ECP_{\alpha}}$) simply by changing the natural language instruction in the prompt.
• Constrained Optimization: It can perform constrained optimization based on natural language commands, such as optimizing WL while ensuring other metrics (Area, Count, Power, PDP) do not worsen beyond a specified tolerance (e.g., 2% or 4% relative to baseline, shown in Table 5). This is a significant practical advantage over methods that require formal constraint specification.
• Model Agnosticism: The framework is designed to be model-agnostic, allowing it to leverage different LLM providers (e.g., Claude, OpenAI) without fine-tuning. This enables immediate adoption of newer, more capable models as they become available, potentially leading to further performance gains. The paper notes preliminary checks with newer models show promise (Appendix D).
• Lower Training Cost: By avoiding fine-tuning on domain-specific data, ORFS-agent bypasses the significant data collection, training, and maintenance costs associated with fine-tuned LLMs like ChatEDA or ChipNeMo.

Implementation considerations include managing the parallel execution of ORFS runs, parsing potentially incomplete JSON logs from timed-out runs, and effectively managing the LLM's context window across iterations. The use of surrogate metrics from earlier flow stages (CTSWL, CTSECP) helps the agent learn even when runs don't complete to the final stage, improving robustness to timeouts. Resource requirements involve orchestrating multiple parallel ORFS instances (each using vCPUs and RAM) and paying for LLM API calls. The paper mentions the API costs are relatively low compared to the potential cost of fine-tuning.

Trade-offs exist between ORFS-agent's approach and fine-tuning methods. While fine-tuning can yield highly specialized models, it incurs substantial upfront and ongoing costs for data curation, training, and deployment. ORFS-agent trades potential peak performance from deep specialization for flexibility, lower cost, faster adoption of model advancements, and the ability to leverage general-purpose reasoning for tasks like natural language constraints.

Future work aims to enhance the agent by enabling dynamic tool generation, integrating Retrieval Augmented Generation (RAG) to access external documentation, and evaluating performance on larger designs and parameter spaces.

In summary, ORFS-agent presents a practical and flexible framework for applying LLMs to automate and optimize complex engineering workflows like chip design. Its model-agnostic, tool-using architecture, combined with its ability to handle natural language objectives and constraints, offers a promising direction for making advanced EDA accessible and efficient.