Interpretable and Verifiable Hardware Generation with LLM-Driven Stepwise Refinement

Abstract: LLMs have achieved remarkable success in software development. However, they are susceptible to hallucinations, meaning that they can introduce subtle semantic and logical errors. Due to the high stakes in chip design and manufacturing, hardware engineers are still reluctant to rely on LLMs for register-transfer level (RTL) generation. In this paper, we propose a hardware generation framework that combines the creativity and broad knowledge of LLMs with the explainability and mathematical rigor of formal methods. Specifically, we devise a set of transformation rules that cover various design decisions and hardware features. By iteratively applying these rules, an LLM agent can convert a design specification into an RTL program with guaranteed correctness. Experimental results demonstrate the effectiveness and efficiency of the framework.

• The paper demonstrates a novel agentic framework that integrates formal refinement calculus with LLMs to generate verifiable RTL designs.
• It employs iterative stepwise refinement with auto-formalization and incremental verification to ensure correctness-by-construction.
• Empirical evaluations on the VerilogEval V2 benchmark show a 92.3% pass@1 rate and efficient design synthesis, confirming scalability and reliability.

LLM-Driven Stepwise Refinement for Interpretable and Verifiable RTL Generation

Motivation and Background

LLMs have demonstrated substantial proficiency in software generation, yet hardware design remains notably resistant to monolithic LLM-driven approaches. RTL generation is challenged by the scarcity of high-quality, proprietary data, the catastrophic impact of subtle functional errors in hardware, and the incongruity between sequential LLM output modalities and inherently concurrent hardware semantics. Existing approaches such as agentic workflows, graph-based verification, and reinforcement learning with testbench feedback have improved reliability but tend to lack interpretable intermediate representations or structured stepwise design traces, inhibiting integration with formal verification, debugging, and downstream synthesis flows.

Formal program construction, grounded in refinement calculus, yields correctness-by-construction guarantees but relies extensively on human expertise for rule application and decision sequencing. This paper proposes a synergistic agentic framework leveraging both LLM creativity and formal methods’ rigor for interpretable, verifiable, and automated RTL generation, targeting the high correctness requirements of chip design.

Formal Hardware Specification and Refinement Calculus

Hardware requirements are translated from natural language and flow diagrams into a specification language, $\mathcal{L}_{spec}$, encompassing preconditions, during conditions, postconditions, and environment assumptions using bounded temporal operators. This representation is tractable for LLMs and supports semantically rich specification constructs needed for concurrent systems.

Hardware design is modelled as iterative successive refinement: an abstract specification is decomposed via a suite of refinement rules into concrete implementations. The calculus includes rule sets for process partitioning, control/data flow, assignment strategies, and compositional reasoning, each rule encapsulating legal application conditions and algebraic transformation logic. Importantly, parallel, piping, and bidirectional composition rules allow hierarchical system partitioning with explicit information flow and constraint propagation, while process-level initialization and invariant selection are designed for LLM compatibility, reducing developer reliance for loop invariant and synchronization discovery.

Agentic Architecture for Stepwise RTL Generation

The proposed agentic system orchestrates hardware generation in three stages:

1. Auto-Formalization: Translates specifications into formal statements via LLM, with syntax and functional validation loops using Dafny’s verification engine.
2. Stepwise Refinement: LLM agents plan high-level partitioning, iteratively select rule applications, record design steps, and backtrack on infeasible branches. Incremental verification is performed at each step, integrating assume/assert annotations for fine-grained correctness checking.
3. RTL Translation: Synthesizable Verilog code is generated from concrete Dafny implementations, using plan-derived auxiliary mappings for ports, registers, and process block types.

   Figure 1: Architecture of the agentic system for correct-by-construction RTL generation.

Verifiable reward signals and incremental correctness checks scaffold reinforcement learning and facilitate future improvements in search efficiency. The separation of planning, rule selection, and transformation mitigates context dilution and reduces token consumption.

Empirical Evaluation and Numerical Results

The system was evaluated on the VerilogEval V2 benchmark suite containing 156 design tasks with both combinational and sequential targets. The method was compared against baseline Claude Opus 4.6 and VeriMaAS, tested under equal conditions. Strong numerical results are reported:

• Pass@1 Rate: 92.3% (144/156), outperforming baselines, despite not being trained for top-down refinement.
• Average Tokens: 61.8k per instance, traded for correctness and incremental verifiability.
• Average Runtime: 221.1s per instance, reflecting stepwise agentic compositionality.
• Backtrack Rate: Only 2.6%, corroborating efficient search with structured rules.
• Average Search Depth: 7.1 steps.

Combinational designs showed slightly higher pass rates than sequential, and runtime/token statistics confirm that formal verification cost is negligible compared to LLM inference.

Figure 2: Number of tokens consumed in stage 2 vs. search depth on the refinement tree.

Figure 3: Total runtime of stage 2 vs. search depth on the refinement tree.

Both metrics scale linearly with search depth, supporting scalability and compositional refinement. The system demonstrates high cache hit rates (53%) and consistent recoverability from dead ends, indicating potential for more aggressive RL-driven agent search and larger design instantiations.

Practical and Theoretical Implications

By creating a framework where each design step is interpretable, verifiable, and recoverable, the paper provides a pathway to scalable, correctness-assured chip design. Integration of formal specification and incremental verification into the agent loop allows for early error detection, intermediate reward signals, and practical debugging, directly addressing the major risks in LLM-driven hardware generation.

Theoretically, the compositional refinement calculus and stepwise agent orchestration set a template for autonomous design in domains demanding high assurance. The approach highlights the feasibility of using LLMs for creative yet rigorous hardware synthesis through guardrailed iterative refinement, and suggests practical methodologies for synthetic data generation, toolchain scaffolding, and future RL-based optimization.

Future Directions

Key areas for extension include:

• Broader and more powerful refinement rule design.
• Scaffolding for transformation and verification of formal hardware languages.
• RL-enhanced agentic orchestration for efficient search and learning from intermediate rewards.
• Application to complex hardware designs, system-on-chip integration, and high-performance architectures requiring numerous interconnected refinement steps.

Conclusion

The paper introduces a formal agentic system for interpretable, verifiable, and correctness-assured RTL generation, leveraging LLMs in a stepwise refinement framework. Empirical results demonstrate substantial reliability and efficiency, with practical implications for industrial chip design automation. The formal-calculus-based methodology opens avenues for scalable, verifiable synthesis in AI-driven design, and future refinement in automation, rule design, and reinforcement learning integration is anticipated to further advance the field.