TimingLLM: A Two-Stage Retrieval-Augmented Framework for Pre-Synthesis Timing Prediction from Verilog

Abstract: Early, tool-free prediction of post-synthesis timing remains a key obstacle to rapid RTL iteration. We introduce TimingLLM, a two-stage retrieval-augmented LLM pipeline that estimates worst negative slack (WNS) and total negative slack (TNS) directly from Verilog. Stage 1 is a fine-tuned LLM that acts as a compact post-synthesis timing oracle, producing path-level arrivals/required times that are summarized into lightweight structural-timing cues (e.g., bag-of-gates counts, critical-path depth, gate-type patterns). Stage 2 is an LLM-based regressor that predicts WNS/TNS and applies a learned diagonal steering vector at the last transformer block, computed from the k nearest timing-labeled modules in a disjoint retrieval bank. On VerilogEval, TimingLLM attains R_WNS = 0.91 (MAPE 12%) and R_TNS=0.97 (MAPE 16%) while running 1.3-1.6 times faster than prior methods. Training uses a new 60k-module Verilog corpus with synthesis reports, which we will release. After training once, TimingLLM can be adapted to new technology libraries and PVT corners by refitting only a small regression head on 1000 labeled modules per setting, consistently outperforming state-of-the-art baselines.

• The paper presents a novel two-stage LLM framework that predicts worst negative slack (WNS) and total negative slack (TNS) directly from Verilog without reliance on synthesis data.
• It employs a stage-one path-level timing analysis to generate a structural-timing fingerprint and a retrieval-augmented LLM-based regression model to refine predictions.
• Evaluation demonstrates strong performance with high Pearson R values (WNS=0.91, TNS=0.97) and faster runtimes compared to state-of-the-art EDA tools.

TimingLLM: Retrieval-Augmented Two-Stage LLM for RTL Timing Prediction

Introduction and Motivation

Early-stage timing prediction directly from register-transfer level (RTL) descriptions remains a bottleneck in ASIC and SoC design flows. Precise static timing analysis (STA) is only available after synthesis and detailed parasitic extraction, delaying the detection of timing violations and impeding rapid design iterations. Conventional machine learning approaches, including graph neural networks and regression-based methods, require extensive feature engineering and are hampered by poor generalization across libraries and process-voltage-temperature (PVT) corners. The emergence of LLMs capable of parsing hardware description languages presents a novel avenue to bypass these limitations. This work introduces TimingLLM, a two-stage LLM-based pipeline that predicts worst negative slack (WNS) and total negative slack (TNS) for RTL modules directly from Verilog, without tool-dependent post-synthesis inputs.

Dataset Construction and Stratification

A robust and granular dataset is central to the effectiveness of data-driven timing estimators. This work culls and processes a massive corpus derived from PyraNet, ultimately curating 60,000 synthesizable Verilog modules with associated synthesis-based timing reports. The selection pipeline incorporates RTL synthesis filtering, deduplication, and annotation with gate counts, flip-flop counts, path depths, and preliminary timing estimates. Stratification is performed along multiple axes: structural complexity, statefulness, timing difficulty, and functional domain. Embedding and k-means clustering on $\ell_2$-normalized bag-of-gates fingerprints ensure code-style diversity. Stratified sampling across clusters addresses rare blocks (e.g., bus arbiters, CRC engines), guaranteeing comprehensive coverage of architectural and timing-relevant patterns.

Figure 1: Dataset curation and stratified selection pipeline for the 60k-module corpus.

Two-Stage TimingLLM Architecture

TimingLLM employs a two-stage cascade: Stage 1 generates a structural–timing fingerprint via path-level timing analysis from Verilog, while Stage 2 leverages retrieval-augmented regression for final WNS/TNS prediction.

Stage 1: LLM-Based Path-Level Timing Analysis

Stage 1 fine-tunes a Llama 3.1-based LLM to emulate critical-path timing reasoning. Inputs are Verilog sources; targets are canonicalized STA reports enumerating arrivals, required times, and slack for each path. Output is a synthetic, interpretable timing report. Report statistics such as path slacks, gate and flip-flop counts, and critical-path topology are distilled into a $\ell_2$-normalized structural–timing fingerprint. This transformation allows inner-product similarity search over large repositories while abstracting away numeric timing labels.

Figure 2: Two-stage TimingLLM: fingerprint generation and retrieval-steered timing prediction.

Stage 2: Retrieval-Augmented LLM Regression

Stage 2 is an LLM-based regressor (DeepSeek-R1-7B backbone with LoRA adapters). For a test module, its fingerprint is used to retrieve the $k$ most similar modules from a disjoint retrieval bank. Activations (last-layer logits) for these neighbors are combined via cosine-weighted residual steering, biasing the query representation toward known timing priors. The regressed WNS/TNS values are output via a lightweight MLP head, which can be rapidly re-fit to new technology libraries and PVT corners using only a small sample of labeled modules.

Figure 3: Qualitative retrieval: feature-space steering leverages retrieval-bank neighbors with shared structural and timing characteristics.

The design is explicitly modular: the retrieval bank encodes priors without exposing training labels, facilitating robust cross-library adaptation and privacy preservation.

Evaluation and Ablation Studies

Experiments utilize three disjoint data splits: a 40k training set, 20k retrieval bank, and the public VerilogEval test set. Metrics reported are Pearson $R$ and MAPE on WNS/TNS.

Strong empirical results underscore the approach:

• On VerilogEval, TimingLLM achieves $R_{\text{WNS}}=0.91$ (MAPE 12%) and $R_{\text{TNS}}=0.97$ (MAPE 16%), outperforming state-of-the-art baselines such as CircuitFusion, RTL-Timer, and MasterRTL by 2–8 $R$ points and 2–14% MAPE.
• Runtime is 1.3–1.6$\times$ faster than prior methods (323s end-to-end vs. 411–514s for competitors on a comparable system).
• Memory footprint is held in check by freezing all base LLM weights; only LoRA adapters and the regression head are trained.

  Figure 4: Top-2 setup paths: Stage-1 LLM's path-level predictions match tool reports within 12 ps for WNS, 7 ps for TNS.

Category-wise breakdowns confirm robust performance across combinational, sequential, and control-centric designs, with the largest improvements on FSM/composite and stateful modules. TimingLLM demonstrates strong transferability: by retraining only the regression head with 1000 labeled modules, equivalent accuracy is maintained across Nangate45, Nangate15, and ASAP7 libraries as well as typical and worst-case corners.

Ablation studies dissect the contributions:

• Removing either Stage 1 (chain-of-thought) or Stage 2 (retrieval and regression) degrades accuracy—full pipeline is essential for optimal generalization.
• Diagonal feature-wise steering vectors at the last transformer block yield best prediction stability; scalar gain offers competitive, low-parameter alternatives.
• Optimal number of neighbors for retrieval is $k=3$, with diminishing returns beyond this value.

Theoretical and Practical Implications

TimingLLM establishes that LLMs, when guided by domain-specific synthetic reports and retrieval-based conditioning, can deliver high-fidelity quantitative predictions for design metrics that have traditionally proven resistant to text-based modeling. By separating timing reasoning (path-level explainability) from global estimation and relying on structure–timing fingerprints for retrieval, the architecture achieves both interpretability and flexibility. Portability across fabrication technologies, PVT corners, and Verilog coding styles is realized with minimal adaptation overhead.

Practical implications are significant: rapid, tool-free timing feedback becomes available during RTL development, reducing design-turnaround bottlenecks and accelerating architectural exploration. The public release of a stratified 60k-module corpus with ground-truth timing annotations will further stimulate research in RTL-aware LLMs and structural code retrieval.

Conclusion

TimingLLM demonstrates that a two-stage, retrieval-augmented LLM framework can achieve accurate, efficient, and portable post-synthesis timing prediction directly from Verilog code. Its ability to generalize across technology nodes and corners via lightweight head adaptation, coupled with strong empirical performance and runtime efficiency, marks a substantive advance in tool-free EDA methodologies. Future research may extend this paradigm to encompass additional metrics (power, area), broader HDL dialects, and memory-augmented architectures with dynamic retrieval policies, setting the stage for further integration of foundation models into the front-end VLSI design stack.