HLSDebugger: HLS Verification & Debug Paradigms

• HLSDebugger is a family of approaches that bridge the gap between software-level specifications and hardware-level implementations in high-level synthesis.
• The shift-left verification framework uses an autonomous agent, symbolic execution, and a heterogeneous knowledge graph to generate testbenches and verify dynamic behavior.
• An encoder-decoder LLM model and source-level debugging integrations enhance the system by locating and correcting HLS logic bugs and mapping hardware execution back to source code.

HLSDebugger designates a cluster of debugging and verification systems for high-level synthesis (HLS) workflows. In the cited literature, the name is used for a knowledge-augmented, agent-driven shift-left verification framework for checking functional consistency between a golden C specification and an HLS-oriented C implementation before synthesis; for an encoder-decoder large-language-model system that performs HLS logic-bug identification and correction; and, in related source-level debugging environments, for infrastructure that maps hardware execution back to source constructs in HLS or hardware-generator toolchains (Xiao et al., 15 Jun 2026, Wang et al., 29 Jul 2025, Goeders et al., 2015, Zhang et al., 2022). Taken together, these works indicate that “HLSDebugger” is best understood as a family of approaches spanning pre-synthesis equivalence checking, bug localization and repair, and source-level introspection over generated hardware.

1. Terminological scope and lineage

In the cited literature, the designation is attached to multiple technical artifacts rather than a single canonical implementation. One line of work describes HLSDebugger as a closed-loop verification tool built from the shift-left framework in “Shift-Left High-Level Synthesis Verification via Knowledge-Augmented LLM Agent,” with an Autonomous Verification Agent (AVA), Dual-Tier Consistency Checking, KLEE-guided refinement, gcov-based coverage evaluation, and a heterogeneous HLS Verification Knowledge Graph (Xiao et al., 15 Jun 2026). A second line uses the name for a customized LLM solution for HLS logic debugging, centered on a 300 K-sample dataset and a CodeT5-Large-based encoder-decoder model for bug location identification, bug type prediction, and bug correction (Wang et al., 29 Jul 2025). Earlier work in the LegUp ecosystem describes a debug framework also called HLSDebugger, or HLS-Scope, that gives software engineers a familiar source-level debugger interface while executing HLS-generated hardware on FPGA boards (Goeders et al., 2015). Related work on hardware generators connects modern software source-level debugging frameworks to generated RTL and reports an IDE experience for systems such as RocketChip in Chisel (Zhang et al., 2022).

Designation in the literature	Main function	Source
HLSDebugger as shift-left verification framework	Automated functional consistency checking between golden C and HLS-C implementations before synthesis	(Xiao et al., 15 Jun 2026)
HLSDebugger as LLM debugging model	Bug location identification, bug type prediction, and bug correction with the same model	(Wang et al., 29 Jul 2025)
HLSDebugger / HLS-Scope in LegUp	Source-level debugging of HLS-generated circuits on FPGA boards	(Goeders et al., 2015)
Source-level debugger for hardware generators	Breakpoints, source-variable recovery, and forward/backward debugging over generated RTL	(Zhang et al., 2022)

This terminological overlap reflects a recurring systems problem in HLS: the gap between software-facing source code, HLS-oriented transformations, and hardware-level behavior. The systems differ in intervention point—before synthesis, during model-based diagnosis, or during RTL/FPGA execution—but they all target that gap.

2. Shift-left verification architecture

The shift-left HLSDebugger is structured as a closed-loop agent-driven pipeline whose central controller is the Autonomous Verification Agent. AVA cycles through four stages: Testbench Generation, Coverage Evaluation and Stimulus Augmentation, Dual-Tier Consistency Checking, and Failure Diagnosis with Iterative Refinement (Xiao et al., 15 Jun 2026).

In Testbench Generation, AVA invokes an LLM interface such as GPT-5 or Gemini-3 with the golden C model $P_c$ and the HLS-C variant $P_h$. The prompt is augmented by topology-aware reasoning priors from the Knowledge Graph and KLEE-derived path constraints. The output is a candidate testbench pair $(TB_c, TB_h)$. In Coverage Evaluation and Stimulus Augmentation, AVA compiles and runs $TB_c$ against $P_c$ under gcov and measures statement, branch, and call coverage through the composite metric

$$\mathcal{C}_{total}=\omega_{stmt}\,\mathcal{C}_{stmt}+\omega_{branch}\,\mathcal{C}_{branch}+\omega_{call}\,\mathcal{C}_{call}.$$

If $\mathcal{C}_{total}$ falls below the threshold $\tau_{cov}$, uncovered branches are extracted, sent to KLEE for concrete corner-case inputs, and appended by the LLM to both testbenches before reevaluation.

The third stage imposes a static gate before dynamic comparison. AVA first evaluates the static structural alignment of $(TB_c, TB_h)$, using

$$\mathcal{S}_{static}=\omega_1\mathcal{S}_{IO}+\omega_2\mathcal{S}_{CFG}+\omega_3\mathcal{S}_{DDG},$$

and proceeds only when $P_h$0. If the threshold is not met, AVA prompts the LLM for testbench realignment. When static checks pass, AVA runs cycle-accurate co-simulation using Vitis HLS for $P_h$1 and a native simulator for $P_h$2 and collects input/output traces.

Failure diagnosis is classified by trace comparison. If inputs diverge, the framework labels the issue a testbench bug and returns to Testbench Generation. If inputs align but outputs differ, it reports a design bug, records the failing cycle and counterexample, and terminates. If no discrepancies are found and coverage remains at or above $P_h$3, equivalence is declared. The outer-loop pseudocode is described as following Algorithm 2 in the source paper, with a state memory $P_h$4 that stores coverage logs, static checks, and simulation outcomes to avoid repeating unproductive prompts.

3. Consistency checking, symbolic execution, and refinement

A defining mechanism of the shift-left framework is Dual-Tier Consistency Checking, which separates verification-harness defects from genuine functional mismatches. The static tier evaluates structural alignment along three axes, while the dynamic tier checks cycle-level behavioral equivalence (Xiao et al., 15 Jun 2026).

Input-stimulus similarity is defined over the linearized AST node sequences $P_h$5 and $P_h$6 of the paired testbenches:

$P_h$7

Control-flow-graph similarity is decomposed as

$P_h$8

where $P_h$9 measures control-node distribution overlap and $(TB_c, TB_h)$0 measures normalized nesting-depth difference. Data-dependence-graph similarity is

$(TB_c, TB_h)$1

These terms are then aggregated into $(TB_c, TB_h)$2.

The dynamic tier operates over per-cycle inputs and outputs. For each clock cycle $(TB_c, TB_h)$3, with input vectors $(TB_c, TB_h)$4 and output vectors $(TB_c, TB_h)$5, AVA asserts

$(TB_c, TB_h)$6

and

$(TB_c, TB_h)$7

Violation of the first condition is classified as a testbench bug; violation of the second is classified as a design bug. This decoupling is central to the framework’s claim that equivalence checking must distinguish harness inconsistency from implementation inconsistency.

Symbolic execution supplies the refinement signal. Inputs to $(TB_c, TB_h)$8 are declared symbolic in KLEE, path constraints accumulate as the CFG is traversed, and gcov identifies unvisited branches. Whenever such a branch is observed, AVA returns the corresponding constraint to KLEE, obtains a concrete model, and extends both testbenches with that vector. The loop continues until the total coverage threshold is reached; the source describes $(TB_c, TB_h)$9 as typically 0.95. This suggests a deliberate coupling of symbolic reasoning and LLM generation: KLEE narrows the search for corner cases, while the LLM maintains paired testbench consistency.

4. Knowledge-graph reasoning and empirical performance

The shift-left HLSDebugger includes a heterogeneous HLS Verification Knowledge Graph

$TB_c$0

constructed from a large corpus of verified 4-tuples $TB_c$1. The graph contains two subgraphs: a Coverage KG $TB_c$2 and a Semantics KG $TB_c$3 (Xiao et al., 15 Jun 2026).

In $TB_c$4, nodes represent structural idioms such as nested loops and pointer patterns, and edges of type “requires_input” connect these idioms to historical corner-case vectors. The intended encoding is described as “when you see pattern X, you need stimulus Y to cover branch Z.” In $TB_c$5, relation types include “semantic_equivalent_to,” “has_hls_pragma,” and “type_aligns_with,” and the graph encodes cross-domain consistency invariants such as

$TB_c$6

where $TB_c$7 extracts the AST substructure around data injections. At runtime, AVA builds a query subgraph $TB_c$8 from the new pair $TB_c$9, retrieves nearest neighbors using

$P_c$0

and turns the retrieved subgraphs into succinct reasoning rules in the LLM prompt.

The reported experiments cover 107 HLS benchmark pairs. When the combined KG + Agent framework is instantiated with Gemini-3-pro, the average composite coverage $P_c$1 is reported as 0.9826, the average static consistency $P_c$2 as approximately 0.7483, and the dynamic consistency rate as 95.33%. Under the same 3-round budget, the best prior iterative agent baseline reaches 94.31% coverage and 85.05% dynamic consistency. Ablation studies report that removing the Knowledge Graph drops dynamic consistency by approximately 8 pp and coverage by approximately 2 pp, while zero-shot LLM alone plateaus at less than 0.89 coverage and a 67% dynamic rate. The implementation blueprint further specifies a persistent Knowledge Graph server, a lightweight orchestrator that drives LLM prompts and external tools, and a final report that either certifies bit-exact equivalence before synthesis or pinpoints counterexamples for downstream RTL-level debugging.

5. HLSDebugger as an encoder-decoder LLM for HLS logic bugs

A separate use of the name denotes an LLM system specialized for logic-bug identification and correction in HLS code. The motivating challenges are stated as circuit-data scarcity, the complexity of hardware logic bugs relative to software bugs, and the need for multi-tasking in the absence of reliable testbenches (Wang et al., 29 Jul 2025).

The dataset is generated by a two-stage pipeline. In unsupervised collection, approximately 30 K real HLS modules are mined from GitHub and split on “module…end module,” and 10 K pseudo-HLS modules are synthesized from RTL via GPT-3.5; all are then cleaned or refined with GPT-3.5 to represent valid, bug-free HLS code. In supervised bug injection, GPT-3.5 first produces a “functionality sketch” for each correct module, then selects one bug type from a list of $P_c$3 common HLS logic bug-types and replaces a correct snippet with a buggy variant, and, on a small subset, additionally produces human-readable bug analysis and a high-level debugging strategy. The final supervised dataset has

$P_c$4

samples, with each sample including $P_c$5, $P_c$6, a bug type $P_c$7, a token mask for location, and optionally auxiliary analysis fields.

The model is built on CodeT5-Large, specified as 24 encoder and 24 decoder Transformer blocks, hidden size 1024, attention-head dimension 16, and approximately 770 M parameters. The input buggy HLS code is tokenized with CodeT5’s BPE and prepended with a special [CLS] token. The encoder outputs a global representation $P_c$8 and per-token embeddings $P_c$9. Two 3-layer MLP heads perform bug-location prediction and bug-type prediction, while the decoder, with causal self-attention and cross-attention to encoder outputs, generates a replacement sequence for the buggy snippet. Training uses a multi-term loss:

$$\mathcal{C}_{total}=\omega_{stmt}\,\mathcal{C}_{stmt}+\omega_{branch}\,\mathcal{C}_{branch}+\omega_{call}\,\mathcal{C}_{call}.$$0

with

$$\mathcal{C}_{total}=\omega_{stmt}\,\mathcal{C}_{stmt}+\omega_{branch}\,\mathcal{C}_{branch}+\omega_{call}\,\mathcal{C}_{call}.$$1

and typical weights $$\mathcal{C}_{total}=\omega_{stmt}\,\mathcal{C}_{stmt}+\omega_{branch}\,\mathcal{C}_{branch}+\omega_{call}\,\mathcal{C}_{call}.$$2, $$\mathcal{C}_{total}=\omega_{stmt}\,\mathcal{C}_{stmt}+\omega_{branch}\,\mathcal{C}_{branch}+\omega_{call}\,\mathcal{C}_{call}.$$3, $$\mathcal{C}_{total}=\omega_{stmt}\,\mathcal{C}_{stmt}+\omega_{branch}\,\mathcal{C}_{branch}+\omega_{call}\,\mathcal{C}_{call}.$$4, and $$\mathcal{C}_{total}=\omega_{stmt}\,\mathcal{C}_{stmt}+\omega_{branch}\,\mathcal{C}_{branch}+\omega_{call}\,\mathcal{C}_{call}.$$5.

Evaluation is reported on a 2 K-sample test set. At token level, HLSDebugger reaches 54.3% precision, 83.5% recall, 0.614 F1, and 0.935 AUC, compared with GPT-4 at 12.2%, 37.1%, 0.168, and 0.554, respectively. At line level, HLSDebugger achieves approximately 58% precision and 78% recall, versus GPT-4 at approximately 13% and 18%. Code-wise Top-1 correct line is 79.5% for HLSDebugger and 19.7% for GPT-4. For bug correction under strict substring match, HLSDebugger reports 37.6% accuracy, compared with 10.5% for GPT-4, 15.1% for Gemma-7B-SFT, and 23.7% for HLSEncoder + Gemma-7B-SFT. Even when given perfect locations, HLSDebugger’s fix rate is reported as 49.1%. The source also characterizes it as the first open-source LLM fine-tuned for both locating and fixing HLS logic bugs and notes limitations including the gap between injected bugs and industrial bug patterns, the current fix rate, and the absence of mixed C + pragma, SystemC, or full HLS-flow co-analysis.

6. Source-level debugging frameworks for generated hardware

The earlier HLS debugging literature emphasizes source-level observability and reversible execution over generated hardware. In the LegUp ecosystem, HLSDebugger, or HLS-Scope, is integrated directly into the open-source HLS flow. During synthesis, LegUp emits an instrumented RTL wrapper with trace buffers, breakpoints, and a communication UART, and also emits a MySQL debug database mapping C source lines to LLVM IR instructions, FSM states, and RTL signals or memories (Goeders et al., 2015).

That framework supports two execution modes. Live Mode provides clock-by-clock stepping, pausing of the on-chip FSM, and reading of on-chip variables from memories or registers. Replay Mode runs at full speed, records control-flow and data-write events into on-chip trace buffers over a finite replay window, and then replays the window with software-style step and rewind operations. Instrumentation includes a Trace Buffer Block, a Control-FSM Sampler, Variable-Write Watchers, Breakpoint Units, and a host link interface. Probes are inserted at each LLVM IR to FSM-state boundary and on every write port of local memories and C-style arrays, while replay relies on compressed control and data traces. The reported compression is approximately 40× over naïve full-width capture; 100 Kb of on-chip memory records approximately 4,300 C-level variable writes and control events; and typical replay windows are approximately 2,000–5,000 cycles. On a mid-sized quickSort accelerator, the reported resource overhead is +1.2% LUTs, +1.5% FFs, and +100 Kb BRAM for trace buffers, with a 5–10 MHz reduction in maximum frequency in Live Mode.

A related framework for high-level hardware generators generalizes source-level debugging beyond HLS-specific tools. Its architecture is divided into a Metadata Injector inside the hardware generator compiler, an RTL Simulator Adapter linked via the simulator’s VPI or FLI/PLI interface, and a Source-Level Debugger Frontend using protocols such as GDB remote or the VSCode Debug Protocol (Zhang et al., 2022). At its core is the mapping

$$\mathcal{C}_{total}=\omega_{stmt}\,\mathcal{C}_{stmt}+\omega_{branch}\,\mathcal{C}_{branch}+\omega_{call}\,\mathcal{C}_{call}.$$6

which associates a hierarchical module path, an RTL signal name, and simulation time with the original source filename, line number, and variable. The implementation relies on symbol-table lookup, low-form nomenclature extraction, and a runtime dictionary that resolves module-path and signal-name pairs back to SSA ids and source locations.

The simulator adapter registers a callback on every clock rising edge, queries active breakpoints, reads signal values, evaluates enable conditions, and, on a hit, pauses simulation, reconstructs stack frames, and communicates with the debugger. Reverse stepping is implemented either through a simulator time-set API or by replaying a saved VCD trace backwards. On a RocketChip suite of 20 RISC-V kernels, the reported overhead is at most 3.7% for optimized builds and at most 4.9% for debug builds, while the abstract summarizes the working prototype as offering forward and backward debugging with almost no simulation overhead, less than 5%. Limitations include dynamic loops that cannot be SSA-unrolled without a maximum-bound annotation, aggressive IR optimizations that may remove variables unless “DontTouch” debug mode is enabled, and the lack of source context for vendor-supplied IP without debug metadata.

These source-level systems address a different phase of the debugging stack than the shift-left verification framework or the encoder-decoder bug-fixing model. This suggests a layered interpretation of HLSDebugger in the literature: pre-synthesis equivalence checking aims to prevent erroneous designs from reaching RTL, model-based bug localization and correction aims to repair faulty HLS code, and source-level execution frameworks aim to make generated hardware behavior intelligible once synthesis or simulation is underway.