LLM4RTL: Tool-Assisted LLM for RTL Generation

Abstract: LLMs have facilitated impressive progress in software engineering, code generation, tooling, and systems. Concurrently, a significant body of research has developed which explores a growing variety of methods and systems for applying LLMs to hardware and chip design (e.g., systems for RTL code generation based on functional description). However, when it comes to open Verilog/RTL code-generation, we need high-quality training samples to build specialized and more effective LLM systems through fine-tuning or low-rank adaptation. Here, we propose a ``judge-renew-check-renew-check'' (JRCRC) pipeline which updates a current public dataset using a hierarchy of state-of-the-art commercial LLM models differing in their costs and capabilities in RTL code generation. This approach achieves a cost-effective mechanism for filtering and refining code-generation samples into a higher-quality training dataset. Our experiments also identify some common weaknesses of LLMs in rule-based reasoning and logic, and consequently, in RTL code-generation. Having identified these weaknesses, we develop an architecture for incorporating pre-processing tools to dynamically assist the LLMs in inferring logical relationships from tabular data formats. With our tools-assisted architecture for RTL code generation, we achieve significant overall performance gains in the VerilogEval benchmark and outperform many state-of-the-art methods. Our LLM4RTL system achieves performance comparable to that of GPT-4O using a significantly much smaller LLM.

• The paper introduces a hierarchical screening process (JRCRC) that refines training datasets for more accurate RTL synthesis.
• It integrates rule-based toolchains to extract logical relations, addressing LLM weaknesses in handling tabular and combinatorial data.
• Tool augmentation improves PASS@1 accuracy from ~0.53 to 0.60, nearly matching larger commercial LLMs on logic-intensive tasks.

LLM4RTL: Tool-Assisted LLMs for RTL Generation

Problem Context and Motivation

LLMs have shown increasing capability in code generation tasks spanning from high-level programming languages to Hardware Description Languages (HDLs) such as Verilog. Despite this, extant open datasets for (text, Verilog) pairs possess persistent issues in correctness, representational coverage, and syntactic/semantic alignment. Existing literature demonstrates that LLM-generated code for hardware lags behind that for software, primarily due to the poor quality and scarcity of verified training samples, and the known limitations of LLMs in handling rule-based logical reasoning, in particular over tabular or combinatorial data representations.

The paper "LLM4RTL: Tool-Assisted LLM for RTL Generation" (2606.15500) systematizes improvements in two critical directions: (1) dataset curation via a hierarchical LLM pipeline to extract high-quality training samples under resource constraints, and (2) the explicit integration of rule-based toolchains to mitigate logic-driven failures of pure LLM approaches, especially in translation from tabular representations (e.g., truth tables, waveforms, Karnaugh maps) to Verilog modules.

Hierarchical Data Screening and Refinement

The authors introduce a cost-effective sample refinement workflow, termed "judge-renew-check-renew-check" (JRCRC), integrating multiple LLMs with complementary cost-performance profiles (specifically DeepSeek-V3-671B and GPT-5). The pipeline aims to iteratively screen, renew, and validate samples from an initial corpus (OriGen: 222K samples) without discarding legacy, high-quality examples produced by prior LLM generations.

Key steps include:

• Data Screening: An efficient but strong LLM (DeepSeek-V3) acts as a 'judge,' filtering out low-quality or corrupt samples. A significant overlap with top-tier commercial LLMs (like GPT-5) in identifying invalid samples was empirically validated, reducing computational cost.
• Data Renewal: Rejected samples are regenerated using the same LLM and compiled with IVerilog for syntax validation; residual failures are then escalated to GPT-5 for both solution and testbench synthesis.
• Quality Preservation: All renewed, syntax-correct, and semantically diverse solutions are admitted to the fine-tuning pool, even those not passing the testbench, to maintain coverage and robustness.

This pipeline enables practical curation with a total cost below \$300 for full dataset processing, demonstrating that state-of-the-art data generation for code synthesis does not necessitate brute-force use of frontier LLMs at scale.

LLM Weaknesses in Logical/Tabular Reasoning

Empirical evaluation confirms an intrinsic limitation: even well-finetuned 7B class models show consistent failures across tasks demanding deduction from non-textual/tabular specifications (e.g., waveform translation to logic). This is clearly illustrated by task-level performance bubbles comparing model and dataset combinations, which reveal a static cluster of unsolved tasks regardless of model improvements, thus indicating irreducible logical/chain-of-thought failures.

Figure 1: Empirical task performance comparing DS-Coder-7b-Instruct-V1.5 with Qwen2.5-Coder-7B-Instruct, demonstrating non-trivial learning bias and consistent hard-negative clusters for logic-intensive tasks.

Tool-Augmented Generation: Architecture and Mechanism

To address these observed failures, the authors present a modular toolchain to preprocess input tasks with tabular or logic-specific structures. The toolchain parses task descriptions and non-textual representations to extract explicit logical relations, which are then supplied to the LLM as context for code generation.

• Combinational Logic Tasks: Tools analyze state-invariant tabular representations, perform SOP/POS extraction, and derive canonical expressions without assumption of clock/state semantics.
• Sequential Logic Tasks: Tools identify state/output variables, infer clock or edge-triggered dependencies, and construct temporal logic mappings, leveraging structured algorithms for flip-flop/latch inference.

The final architecture delegates tool selection via heuristics (keywords), but the framework is extensible to agent-mediated or RL-driven tool dispatch. Post-processing iteratively validates generated code, correcting syntax errors until convergence.

Quantitative Results and Comparative Analysis

The efficacy of the approach is benchmarked on VerilogEval-human (156 tasks), utilizing PASS@1 and PASS@5 as metrics. Major findings include:

• Baseline 7B LLMs: Without adaptation, best performing models achieve ~0.36 PASS@1.
• Dataset Refinement: The JRCRC pipeline raises PASS@1 from 0.49 (original OriGen) to 0.53 (fully refined), with strong sensitivity to erroneous code shown—quality trumps quantity, as evidenced by the smallest dataset yielding the best performance.
• Tool Augmentation: Introducing tool-assisted logic extraction boosts PASS@1 from 0.528 to 0.60 (PASS@5 to 0.663), nearly saturating the performance gap with much larger, more expensive LLMs (e.g., GPT-4O).
• Task-Specific Impact: The most significant improvements are concentrated on tabular-driven logic tasks, where baseline models uniformly failed; tool assistance raises these from zero to perfect or near-perfect solution rates.

Discussion and Implications

The findings underscore that high-quality, logic-focused data and tool assistance can enable sub-10B parameter LLMs to closely match the performance of parameter- and cost-mismatched commercial LLMs on RTL synthesis. The effect size is most pronounced for tasks where inductive pattern learning is insufficient, and explicit deductive/algorithmic toolchains are necessary. Tool augmentation is model-agnostic and does not require retraining, thus serving as a plug-in solution for existing models and pipelines.

Several important implications arise:

• Data Curation: Curation with selective, hierarchical LLM pipelines can outperform naïve full-teacher distillation on both cost and quality axes, facilitating sustainable open-source dataset growth.
• Hybrid Reasoning Architectures: Model-mediated reasoning for logic synthesis cannot yet supplant procedural, rule-based tools—especially when tasked with combinatorial or temporal deduction from tables, a well-documented shortcoming in LLM reasoning [wolff2025well, cheng2024inductive, cai2024role].
• Extensibility: The modular toolchain can be generalized to more complex circuit topologies, additional non-textual description formats, and alternative HDLs.
• Tool Selection Policies: The present heuristic approach can benefit from meta-learning strategies (e.g., RL agents for optimal tool dispatch), as suggested for future work.

Conclusion

LLM4RTL demonstrates that pairing systematic data screening with pre-generation logical inference tools produces significant increases in RTL generation accuracy, pushing lightweight LLMs to near parity with state-of-the-art commercial alternatives. The work substantiates the core limitation of current LLMs in deductive logic tasks and provides a practical architecture for real-world, production-quality HDL synthesis pipelines under resource constraints. Future work should target adaptive tool selection, broader tool coverage, and the integration of post-synthesis validation circuits to further raise both semantic correctness and industrial applicability.