VeriThoughts: Verified RTL Synthesis

• VeriThoughts is a reasoning- and verification-centric framework that automates Verilog generation by integrating natural language prompts, chain-of-thought reasoning, candidate code synthesis, and formal equivalence checking.
• It employs a structured 5-tuple model (V, Q, R, V*, Lc) to benchmark and empirically analyze RTL synthesis quality via precise formal verification methods.
• The framework also serves as a GUIDE teaching unit, enabling students to generate, verify, and analyze RTL designs through reproducible labs and detailed performance logs.

VeriThoughts is a reasoning- and verification-centric framework for automated Verilog generation that appears in two closely related forms in the literature: as a large-scale dataset and benchmark for reasoning-based RTL synthesis, and as a GUIDE teaching unit on reasoning and formal-verification-backed RTL generation. In the research setting, it combines a natural-language question, a reasoning trace, generated Verilog, and a formal self-consistency label derived from equivalence checking against a golden design; in the educational setting, it operationalizes the same logic as a reproducible lab in which students generate, verify, and analyze candidate RTL against ground truth (Yubeaton et al., 16 May 2025, Xiao et al., 18 Mar 2026).

1. Definition, scope, and motivating problem

VeriThoughts was introduced to address several difficulties in automated hardware design. The Verilog-generation paper argues that Verilog data is scarce compared with Python, C++, or Java; that existing HDL datasets often lack paired natural-language prompts; that generating prompts automatically can lead to hallucinations or prompts that are too detailed, inconsistent, or reveal the answer; and that existing evaluations often rely on testbenches or unit tests, which may be unavailable, incomplete, or themselves buggy. The GUIDE paper formulates a closely related instructional problem: natural-language prompts alone can be noisy and unreliable, and code that “looks right” may still be subtly wrong because correctness in digital design depends on precise interface behavior, reset semantics, widths, and synthesis constraints (Yubeaton et al., 16 May 2025, Xiao et al., 18 Mar 2026).

Within this framing, VeriThoughts is not merely a corpus of prompt–code pairs. Its central claim is that RTL generation quality should be studied through a reasoning-oriented, formal-verification-based pipeline. The dataset paper describes VeriThoughts as a novel dataset designed for reasoning-based Verilog code generation and as a benchmark framework grounded in formal verification methods. The GUIDE paper presents the corresponding pedagogical objective more explicitly: to help students understand how verification-backed reasoning datasets are built, and how RTL generation quality can be analyzed through formal equivalence checking (Yubeaton et al., 16 May 2025, Xiao et al., 18 Mar 2026).

The resulting conception of VeriThoughts is therefore broader than code generation alone. It joins prompt construction, chain-of-thought-style reasoning traces, candidate RTL synthesis, and formal correctness checking into a single pipeline. This suggests a shift from prompt-only generation toward a workflow in which reasoning is treated as an inspectable artifact and formal equivalence checking acts as the arbiter of correctness.

2. Data model and construction pipeline

The core VeriThoughts representation is the 5-tuple

$\{V, Q, R, V^{*}, L_c\},$

where $V$ is the golden or original Verilog RTL, $Q$ is a natural-language question describing $V$, $R$ is the reasoning trace generated by a reasoning model, $V^{*}$ is the generated Verilog produced from $Q$, and $L_c$ is a self-consistency label indicating whether $V$ and $V^{*}$ are functionally equivalent (Yubeaton et al., 16 May 2025).

Symbol	Meaning
$V$0	Golden/original Verilog RTL
$V$1	Natural-language question/prompt
$V$2	Reasoning trace
$V$3	Generated Verilog
$V$4	Self-consistency label

VeriThoughts is built on top of MetRex, which contains about 25.8K synthesizable Verilog designs gathered from public sources, including human-written designs, machine-generated designs, and designs from VeriGen and RTL-Coder. The construction process discards existing text descriptions, keeps only RTL code, filters to samples shorter than 10,000 characters, and retains about 25K usable Verilog entries. For each design $V$5, the authors use Gemini-2.0-Flash-Thinking-Experimental to generate a prompt $V$6. The prompt template requires the module name, the input/output interface, preservation of the exact signal names, and a question that is not too detailed. The question must begin with QUESTION BEGIN and end with QUESTION [END](https://www.emergentmind.com/topics/eccentric-nuclear-disk-end) (Yubeaton et al., 16 May 2025).

Reasoning-trace generation is then performed with DeepSeek-R1. Given $V$7, the model produces both the reasoning trace $V$8 and the generated Verilog $V$9. The reported generation settings are temperature $Q$0, top_p $Q$1, and max length $Q$2 tokens. The prompt also appends instructions to keep interface names consistent and to delimit code and reasoning with [CODE](https://www.emergentmind.com/topics/confident-ordinary-differential-editing-code) BEGIN, CODE END, and > markers (Yubeaton et al., 16 May 2025).

The GUIDE version of VeriThoughts mirrors this structure in lab form. Students begin with a subset of ground-truth Verilog modules; for each module, an LLM generates a natural-language question corresponding to the design; a reasoning-capable LLM then produces a step-by-step reasoning trace and a candidate Verilog solution. The workflow is thus an instructional reduction of the research pipeline into a runnable subset suitable for analysis and grading (Xiao et al., 18 Mar 2026).

3. Formal verification and the self-consistency label

The decisive methodological element in VeriThoughts is the use of formal equivalence checking to define correctness. The dataset paper defines the self-consistency label as

$Q$3

where $Q$4 is a formal verification procedure. If the two designs are functionally equivalent, then $Q$5; otherwise $Q$6 (Yubeaton et al., 16 May 2025).

The reported toolchain uses Yosys with the following script:

$V$1

In the paper’s description, read_verilog loads both designs, prep; proc; opt; memory; prepares and optimizes the circuits, clk2fflogic converts sequential elements to a form suitable for verification, miter -equiv -flatten constructs an equivalence-checking miter circuit, and sat -seq 50 -verify ... uses SAT solving to prove equivalence over bounded sequential depth. For multi-module designs, the method performs submodule-level equivalence checking: each submodule pair $Q$7 is compared, and $Q$8 only if all submodules are equivalent (Yubeaton et al., 16 May 2025).

Both papers stress that this is stronger than simulation-only evaluation. The dataset paper states that formal verification checks all input combinations, avoids reliance on human-written unit tests, and eliminates the need to generate potentially buggy testbenches. The GUIDE paper operationalizes the same logic through match and mismatch labels assigned by a formal equivalence checker, which turns generated questions, reasoning traces, candidate code, and verification outcomes into a verification-backed reasoning dataset (Yubeaton et al., 16 May 2025, Xiao et al., 18 Mar 2026).

In the educational workflow, formal checking is also the basis for analysis. Students are asked to study how reasoning trace length, prompt style, and module complexity correlate with formal equivalence outcomes. This makes the checker not only a correctness oracle but also the center of an empirical design loop.

4. Benchmarking, ablations, and empirical findings

The paper evaluates models on two benchmarks. VerilogEval consists of 156 HDLBits-derived problems, uses the Human split, and is evaluated using human-designed test cases. The VeriThoughts benchmark is a held-out set of 291 questions sampled from VeriThoughts and evaluated using the paper’s formal verification framework. The reported metrics are pass@1, pass@5, and pass@10 over 20 trials (Yubeaton et al., 16 May 2025).

A central experimental question is whether reasoning helps. The reported answer is affirmative. Comparing reasoning and instruction-tuned models on self-consistent subsets, the paper reports the following results on the VeriThoughts benchmark: pass@1 of 75.5% versus 49.0%, pass@5 of 88.9% versus 69.0%, and pass@10 of 92.1% versus 73.7%. On VerilogEval, the corresponding comparison is pass@1 of 34.6% versus 21.9%, pass@5 of 47.2% versus 31.6%, and pass@10 of 50.7% versus 34.6%. The paper concludes that reasoning traces help, with especially strong gains on the VeriThoughts benchmark (Yubeaton et al., 16 May 2025).

The authors also construct six approximately 10K-sample subsets to analyze self-consistency, reasoning, and target choice. These subsets vary along three axes: whether the sample is self-consistent, whether the fine-tuning format is reasoning or instruct, and whether the Verilog target is the ground truth or the generated code. This design supports three reported research questions: whether reasoning helps, whether question–answer consistency improves accuracy, and whether hallucinated prompt–answer pairs still provide utility (Yubeaton et al., 16 May 2025).

For consistency analysis, the paper reports that self-consistent examples tend to be shorter, while inconsistent examples tend to have more lines of code, longer reasoning traces, and a higher fraction of sequential circuits. The authors explicitly suggest that inconsistent examples are often harder or more complex. Manual inspection of 100 samples sharpens this point. Among 50 inconsistent samples, 33 cases were classified as prompt incorrect, 8 as generated Verilog incorrect, and 9 as both incorrect. Among 50 self-consistent samples, the counts were 0 prompt errors, 0 code-generation errors, and 0 both errors. The paper’s main takeaway is that most mismatches arise because the generated question does not accurately summarize the original Verilog, rather than because the model fails to implement a correct prompt (Yubeaton et al., 16 May 2025).

The paper also reports that hallucinated prompt–answer pairs can still provide utility. When training on inconsistent examples where the target is the generated Verilog rather than the original ground truth, performance often improves on VeriThoughts and can remain competitive on VerilogEval. The authors do not claim a simple causal explanation, but highlight this as an empirical finding enabled by the dataset (Yubeaton et al., 16 May 2025).

For model comparison, the paper fine-tunes Qwen-2.5-Instruct-7B and Qwen-2.5-Instruct-14B using supervised fine-tuning with learning rate $Q$9, a cosine scheduler, bf16, 3 epochs, Llama-Factory, and 8×A100 GPUs. Its named model, VeriThoughts-14B, is based on Qwen-2.5-Instruct-14B and trained on the VeriThoughts reasoning subset. The reported results are 78.5% pass@1, 90.0% pass@5, and 92.1% pass@10 on VeriThoughts, and 43.7% pass@1, 52.2% pass@5, and 55.14% pass@10 on VerilogEval. The paper positions it as competitive among open-source models and as the best open-source pass@1 result on VerilogEval in the reported table (Yubeaton et al., 16 May 2025).

5. VeriThoughts as a GUIDE teaching unit

In GUIDE, VeriThoughts is one of the representative units for LLM-aided RTL generation, specifically under finetuned LLMs for RTL generation. Its educational goal is stated explicitly: to help students understand how verification-backed reasoning datasets are built, and how RTL generation quality can be analyzed through formal equivalence checking (Xiao et al., 18 Mar 2026).

The instructional workflow is presented as an end-to-end lab. Students select a subset of ground-truth Verilog modules from the VeriThoughts dataset; an LLM generates a natural-language question for each module; a reasoning-capable LLM produces a step-by-step reasoning trace and a candidate Verilog solution; a formal equivalence checker compares the candidate against the ground-truth design; and the checker assigns a label of match or mismatch. Students then analyze how reasoning trace length, prompt style, and module complexity correlate with formal equivalence outcomes (Xiao et al., 18 Mar 2026).

GUIDE standardizes this unit within a broader courseware architecture consisting of slides, a short video, a runnable Colab lab, and related papers. For VeriThoughts specifically, the lab is intended to be runnable from scratch, to include at least one end-to-end example, to provide evidence for grading, to use stable interfaces and file layouts, and to explain advanced terms clearly (Xiao et al., 18 Mar 2026).

The submission requirements formalize the pedagogical emphasis on reproducibility and verification artifacts. Students must provide: a selected subset configuration specifying which modules were chosen and what filtering criteria were used; a synthesized dataset containing generated natural-language questions, reasoning traces, candidate Verilog solutions, and formal equivalence labels; a formal equivalence checker log with at least one match example and at least one mismatch example; and a short analysis report discussing how pipeline or subset choices affect the match/mismatch distribution. The GUIDE paper presents this not as a benchmark report but as a structured laboratory exercise in which reasoning should be validated rather than merely admired (Xiao et al., 18 Mar 2026).

6. Significance, limitations, and future directions

VeriThoughts is significant because it reorganizes Verilog generation around verifiable artifacts. The dataset paper emphasizes four contributions in combination: paired prompt–code data, reasoning traces, a formal verification-based label for prompt–answer consistency, and a benchmark that avoids weak testbench-only validation. The GUIDE paper emphasizes the corresponding instructional benefits: it teaches that natural-language prompts alone are noisy, shows how formal verification can improve data quality by filtering candidate designs, and clarifies the connection among prompting, reasoning, RTL generation, and correctness checking (Yubeaton et al., 16 May 2025, Xiao et al., 18 Mar 2026).

The limitations are also clear in the source material. The GUIDE paper notes that the approach depends on access to a formal equivalence checker and a clean reference design; that it may be more appropriate for designs where ground-truth modules are available; that the quality of the reasoning trace is not guaranteed to imply correctness; that the GUIDE article itself does not provide detailed performance statistics for the unit; and that a lab designed around a subset of modules may be less representative of full-scale industrial RTL generation. The dataset paper, while not presenting a separate limitations section for VeriThoughts alone, reports that inconsistency is entangled with difficulty, longer reasoning traces, more lines of code, and a higher fraction of sequential circuits (Xiao et al., 18 Mar 2026, Yubeaton et al., 16 May 2025).

The future directions identified in the dataset paper are concrete. The authors explicitly suggest adding more data sources beyond MetRex for greater diversity, applying the same formal verification benchmarking methodology to other HDL benchmarks such as VerilogEval, expanding the use of reasoning and verification in dataset generation, and studying more deeply when hallucinated prompt–answer pairs are beneficial or harmful (Yubeaton et al., 16 May 2025).

Taken together, these works define VeriThoughts as both a research artifact and an instructional pattern. In the research paper, it is a formally verified, reasoning-augmented Verilog dataset centered on the tuple $V$0. In the GUIDE paper, it becomes a reproducible teaching unit that exposes the full reasoning-plus-verification loop. A plausible implication is that VeriThoughts marks a methodological preference for hardware-generation systems in which correctness is established by formal verification and reasoning is treated as analyzable evidence rather than as an unvalidated intermediate narrative.