---
title: 'VeriGRAG: Structure-Aware Verilog Code Generation'
url: https://www.emergentmind.com/topics/verigrag
type: topic
---

# VeriGRAG: Structure-Aware Verilog Code Generation

Searching arXiv for VeriGRAG and closely related papers to ground the article.
VeriGRAG is a framework for Verilog register-transfer-level code generation that augments large language models with retrieved structural representations of hardware designs. It addresses a specific limitation of text-centric code generation: Verilog encodes concurrent circuit structure, signal dependencies, datapaths, and interconnections that are not adequately captured by surface token statistics alone. The framework therefore combines graph extraction from Verilog, graph neural encoding, multimodal retrieval from natural-language specifications to graph embeddings, and transformer-based alignment that converts retrieved structural information into structure-aware soft prompts for a code LLM [2510.15914].

## 1. Definition and scope

VeriGRAG is introduced in the paper "VeriGRAG: Enhancing LLM-Based Verilog Code Generation with Structure-Aware Soft Prompts" as a method for improving the functional and syntactic correctness of LLM-generated Verilog by retrieving structural graph embeddings and injecting them into generation through soft prompts [2510.15914]. Its operating assumption is that Verilog differs from ordinary software code because correctness depends on circuit organization as much as on local syntax. The paper therefore treats the task as a cross-modal retrieval-and-generation problem: the input is a natural-language hardware description, whereas the retrieved prior knowledge is graph-structured Verilog circuitry.

The framework is not a verifier in the sense of claim checking or scientific auditing. This distinction matters because a similarly named system, VERIRAG, is a healthcare claim-verification framework built on retrieval-augmented generation and centered on methodological auditing, a Hard-to-Vary score, and a Dynamic Acceptance Threshold [2507.17948]. By contrast, VeriGRAG targets hardware code synthesis, and its core contribution is structure-aware retrieval plus prompt conditioning rather than evidence adjudication [2510.15914].

A plausible implication is that VeriGRAG occupies an intermediate position between code-oriented RAG and graph-conditioned multimodal generation. It does not simply append retrieved code snippets as text; instead, it retrieves graph embeddings derived from Verilog datapath structure and aligns them to the embedding space of the generation model [2510.15914].

## 2. Problem formulation and motivation

The motivating problem is LLM-based generation of Verilog RTL code from natural-language hardware specifications. The paper argues that existing approaches mainly treat Verilog as plain text and therefore underexploit topological structure, datapaths, and dependencies among hardware modules [2510.15914]. In this view, a Verilog fragment may appear locally plausible while still encoding an incorrect circuit because module interfaces, signal flow, sequential updates, or combinational dependencies are structurally inconsistent.

To address this, VeriGRAG treats structural information as a first-class retrieval target. The system first converts existing Verilog into graph form, then retrieves graph embeddings relevant to a new specification, and finally converts those embeddings into soft prompts that condition the LLM during code generation [2510.15914]. The paper explicitly states that standard retrieval-augmented generation is insufficient because the retrieval problem is cross-modal: the query is natural language, but the useful prior knowledge lies in graph-structured representations of Verilog circuits.

This motivation also differentiates VeriGRAG from other GRAG formulations. In GRAG for LiveRAG, for example, generation precedes retrieval through a hypothetical answer used as an extra search query, but there is no explicit verification of that intermediate content [2506.14516]. In DynaGRAG, the emphasis is enhanced subgraph representation, retrieval diversity, and hierarchical hard prompting for graph-grounded language understanding [2412.18644]. VeriGRAG instead uses graph retrieval to condition code synthesis with hardware-specific structural priors [2510.15914].

## 3. Structural representation and retrieval pipeline

VeriGRAG begins by extracting graphs from Verilog using Yosys. For each Verilog module, the paper states that it runs `"read_verilog; hierarchy -check; proc; opt; fsm"` to obtain the corresponding data path graph [2510.15914]. The representation is operational rather than fully axiomatized: each module is represented as a graph node, inter-module connections are defined by wires and modeled as edges, and in the paper’s illustrative description nodes represent hardware modules or signals while edges denote interconnections between them.

Node features include operation types, I/O types, I/O signal ports, and parameter lists, and these attributes are encoded into embeddings using a pretrained BERT model. Edge features are derived from normalized wire bit-widths and represent structural relationships among connected modules [2510.15914]. The graph encoder is GINEConv, chosen because the Verilog datapath graphs carry informative edge attributes. The node update is given as
\[
\mathbf{x}_i^{\prime} = h_{\Theta}\Big((1+\epsilon) \cdot \mathbf{x}_i + \sum_{j \in \mathcal{N}(i)} \mathcal{ReLU}(\mathbf{x}_j + \mathbf{e}_{j,i}) \Big).
\]
This update integrates the node’s own state, neighborhood information, and edge features [2510.15914].

Graph embeddings are learned with unsupervised contrastive learning. Two augmented views of a graph, produced by stochastic operations such as edge deletion and feature perturbation, are encoded independently:
\[
z_i^{(1)} = f_{\theta}(\widetilde{\mathcal{G}^{(1)}}), \quad z_i^{(2)} = f_{\theta}(\widetilde{\mathcal{G}^{(2)}}).
\]
Training uses an InfoNCE objective with cosine similarity and temperature \(\tau\) [2510.15914]. The resulting graph-level embeddings are stored and indexed for retrieval.

Retrieval itself is multimodal. The query is a natural-language hardware description, and the database items are graph embeddings extracted from Verilog modules. The paper discusses a cross-attention encoder as teacher and a dual encoder as student. The teacher improves positive-pair similarity through bidirectional interaction between language and graph modalities, but cannot pre-encode graph embeddings efficiently. The student independently encodes the language description and the graph-side representation, enabling pre-encoding and FAISS indexing, and is trained by combining its own InfoNCE objective with an MSE distillation loss from the teacher [2510.15914]. This suggests that the system is designed to retain cross-modal retrieval quality without sacrificing inference-time scalability.

## 4. VeriFormer and structure-aware soft prompts

The alignment module is VeriFormer, which bridges graph embeddings and the LLM embedding space. It consists of two transformer sub-modules that share the same self-attention layers: a graph transformer and a code transformer [2510.15914]. VeriFormer uses 96 learnable query tokens of dimension 768, initialized from CodeBERT weights, and these tokens interact with graph embeddings through cross-attention and with the code modality through the shared self-attention stack.

Training proceeds in two stages. In the first stage, the GNN is frozen while VeriFormer and the query tokens are trained to align graph information with Verilog code. The paper states three objectives here: a graph-code contrastive learning objective, a graph-code generation objective trained with cross-entropy loss, and a graph-code matching objective implemented as binary classification over graph-code pairs [2510.15914]. The purpose is to learn query-token representations that extract graph information most relevant to Verilog generation.

In the second stage, only the graph transformer is retained, and its outputs are projected into the input embedding dimension of the target LLM. These projected query outputs constitute the structure-aware soft prompts [2510.15914]. The paper introduces a KL-divergence-based distribution loss to reduce mismatch between the original LLM embedding distribution for the hardware description and the projected soft prompt distribution:
\[
Loss_{Dist} = \frac{1}{B\cdot N} \sum^{B\cdot N}_{i=1}\sum^{D}_{j=1} \mathbf{Z_i^{(j)}} \left(\log \mathbf{Z_i^{(j)}} - \log \mathbf{G_i^{(j)}} \right).
\]
The second-stage objective is
\[
Loss = Loss_{Gen} + \alpha \cdot Loss_{Dist}.
\]
The LLM remains frozen during this stage [2510.15914].

Functionally, the soft prompts are continuous prompt vectors generated from retrieved graph embeddings rather than textual exemplars. The paper states that the projected outputs are used as a soft prompt for the LLM, but does not explicitly specify whether they are prepended token embeddings, prefix states, or another insertion mechanism [2510.15914]. This suggests that VeriGRAG is best viewed as an instance-conditioned prompt-tuning system in which the prompt is synthesized from retrieved hardware structure.

## 5. Training procedure, benchmarks, and empirical results

The training corpus is built from OriGen and PyraNet. After preprocessing with MinHash and Jaccard similarity to remove duplicates and near-duplicates, and filtering empty or meaningless examples, the final dataset contains 276,627 examples [2510.15914]. The paper also states that QLoRA is used to train the LLM’s RTL generation capability, with backbone models DeepSeek-Coder-7B-Instruct, Qwen2.5-Coder-7B-Instruct, and Qwen2.5-Coder-14B-Instruct.

The multimodal retriever is trained in two phases: a cross-attention teacher trained for 15 epochs at learning rate \(5\times10^{-5}\), then a dual-encoder student trained for 100 epochs at learning rate \(5\times10^{-4}\) and global batch size 1024 [2510.15914]. VeriFormer stage 1 is trained for 3 epochs at learning rate \(1\times10^{-4}\) and stage 2 for 1 epoch at learning rate \(3\times10^{-6}\), with the paper reporting the corresponding GPU counts and batch sizes.

Evaluation uses VerilogEval v1, VerilogEval v2, RTLLM v1.1, and RTLLM v2. The main functional metric is \(pass@k\),
\[
pass@k := \mathbb{E}\left[ 1 - \frac{n-c \choose k}{n \choose k} \right],
\]
with \(n=20\) and reported \(k \in \{1,5\}\) [2510.15914]. Syntax pass rates are also reported where available.

Several headline results are explicit. On VerilogEval-Machine, VeriGRAG with Qwen2.5-Coder 14B achieves \(pass@1 = 82.9\) and \(pass@5 = 89.4\), the best result in that table [2510.15914]. On VerilogEval v2 Function, VeriGRAG Qwen2.5-Coder 14B attains the best reported \(pass@1\) at 62.2, while on RTLLM v1.1 Function the best \(pass@1\) is 57.6 with VeriGRAG Qwen2.5-Coder 7B and the best \(pass@5\) is 75.4 with VeriGRAG Qwen2.5-Coder 14B [2510.15914]. On RTLLM v2 Function, the best \(pass@1\) is 57.7 with VeriGRAG Qwen2.5-Coder 7B and the best \(pass@5\) is 71.5 with VeriGRAG Qwen2.5-Coder 14B [2510.15914]. The paper also emphasizes very strong syntax results, including \(pass@1 = 99.5\) and \(pass@5 = 100.0\) on VerilogEval v2 Syntax for VeriGRAG Qwen2.5-Coder 7B and VeriGRAG DeepSeek-Coder 7B [2510.15914].

The ablation study attributes the largest gain to the structure-aware soft prompts. For Qwen2.5-Coder-14B, VeriGRAG improves \(pass@1\) by 13.3% over the pretrained model on VerilogEval-Human and by 17.9% on VerilogEval v2, while the knowledge-distilled retriever yields an additional \(0.8\%\) \(pass@1\) on VerilogEval-Human and \(0.2\%\) on VerilogEval v2 [2510.15914]. The paper therefore presents the retriever distillation as helpful but secondary to the graph-conditioned prompting mechanism itself.

## 6. Relation to adjacent GRAG systems and terminological ambiguity

VeriGRAG belongs to a broader family of systems that combine retrieval and generation with graph or structure-aware components, but its technical identity is distinct. DynaGRAG emphasizes de-duplication, query-aware subgraph retrieval, Dynamic Similarity-Aware BFS, GCN-based pruning, and hierarchical hard prompting for general graph-grounded language understanding [2412.18644]. DGRAG addresses distributed edge-cloud deployment, using local knowledge graphs, Leiden-based subgraph partitioning, cloud-visible subgraph summaries, and a gate mechanism for query escalation [2505.19847]. G-RAG in material science builds entity-centric graphs augmented with Wikipedia knowledge and multimodal parsing of figures and tables [2411.14592]. These systems share the use of graph-structured external knowledge, but they do not target Verilog code generation through structure-aware soft prompts.

The term itself also requires disambiguation. The paper "VERIRAG: Healthcare Claim Verification via Statistical Audit in Retrieval-Augmented Generation" explicitly notes that a query for “VeriGRAG” is likely a naming confusion and that its own subject is VERIRAG, a biomedical-methodology-aware audit framework rather than a graph-based or Verilog-oriented generator [2507.17948]. For that reason, "VeriGRAG" should be reserved for the Verilog framework unless the context clearly refers to that healthcare claim-verification system.

A plausible implication is that the lexical similarity between "VeriGRAG" and "VERIRAG" can obscure a substantive methodological distinction. VeriGRAG injects retrieved graph embeddings into code generation, whereas VERIRAG scores scientific evidence quality using an 11-point methodological checklist and claim-specific thresholding [2510.15914; 2507.17948].

## 7. Limitations and significance

The paper states one visible limitation through the benchmark pattern: on VerilogEval-Human, VeriGRAG Qwen2.5-Coder 14B is slightly below VeriPrefer Qwen2.5-Coder 14B, which the authors attribute to VeriPrefer’s reinforcement-learning optimization for human preference-style prompts [2510.15914]. Beyond that, several implementation details remain unspecified in the text provided. The paper does not explicitly report the number of GINEConv layers, the precise graph-level pooling operator, a complete formula for the retriever’s distillation loss, or the exact mechanism by which the soft prompts are inserted into the LLM input stream [2510.15914].

Even with those omissions, the framework’s technical significance is clear from the design pattern it establishes. It shows that hardware-aware structural priors can be extracted automatically from existing RTL with Yosys, encoded by a GNN, retrieved with a multimodal retriever, aligned to code embeddings through a query-former-style transformer, and injected as soft prompts into a frozen or lightly adapted code LLM [2510.15914]. This suggests a broader methodological lesson: for code-generation domains where correctness depends on latent structure not readily expressed in surface tokens, retrieved non-textual representations may be more effective than prompt-only text exemplars.

In that sense, VeriGRAG is best understood as a structure-conditioned retrieval-augmented generation framework specialized for RTL synthesis. Its central claim is not that retrieval alone improves Verilog generation, but that retrieved graph embeddings become materially useful only after explicit modality alignment and prompt-space projection [2510.15914].

Source: https://www.emergentmind.com/topics/verigrag