---
title: 'Wit-HW: Automated Hardware Bug Localization'
url: https://www.emergentmind.com/topics/wit-hw
type: topic
---

# Wit-HW: Automated Hardware Bug Localization

Wit-HW is an automated hardware bug localization framework that transforms hardware bug localization into a test generation problem. Instead of localizing faults from only one bug-triggering simulation trace, it aims to generate a set of witness test cases beyond the initial bug-triggering test case. Witness test cases are the cases that do not trigger the bug in the faulty design. By analyzing the execution differences between passing and failing test cases with spectrum-based method, the framework eliminates innocent design statements and ranks the buggy ones [2508.14414].

## 1. Problem formulation and localization objective

Wit-HW defines the buggy RTL design as \(b\), composed of \(z\) statements
\[
S = \{s_1, s_2, \ldots, s_z\},
\]
with a faulty subset \(S_b \subseteq S\). The design has \(n\) input signals, and a test case \(C\) is a sequence of input vectors over \(t\) cycles:
\[
C = \big((I_1^1,\ldots,I_n^1), (I_1^2,\ldots,I_n^2), \ldots, (I_1^t,\ldots,I_n^t)\big).
\]
Given a test case \(C\), the buggy design produces outputs \(O_b(C)\). A validator \(V\), such as a golden RTL design, determines whether \(O_b(C)\) is correct. If \(V\) deems the output incorrect, \(C\) is a failing test case.

The output of localization is a suspiciousness score list
\[
R = \{r_1, r_2, \ldots, r_z\},
\]
where \(r_i\) is the likelihood that statement \(s_i\) is buggy. The ideal ordering is stated as
\[
\forall s_i \in S_b,\ \forall s_j \notin S_b: \quad r_i > r_j.
\]
The input to the problem is the buggy design \(b\), the validator \(V\), and one failing test case \(C_b\); the output is the ranked list \(R\).

The framework is motivated by limitations of existing automatic hardware debugging techniques on large designs and deep bugs. Slicing-based approaches such as CirFix and RTL-Repair can produce very large suspicious regions because of concurrency and dense interconnections. Spectrum-based approaches such as Tarsel rely on a single failing test case, and in sequential hardware the bug may be triggered many cycles before its effect appears on outputs. Wit-HW addresses this by introducing multiple passing behaviors around the failing scenario, rather than analyzing a single trace in isolation.

## 2. Witness test cases and spectrum-based ranking

A witness test case is a test case that does not trigger the bug when run on the same buggy design \(b\). The key idea is that passing executions can exonerate statements that are repeatedly exercised without causing incorrect behavior. This converts bug localization into a contrastive problem between one failing execution and a generated set of passing executions.

Wit-HW uses statement-level hit spectra and the Ochiai suspiciousness coefficient. For a statement \(s\), let \(e_f(s)\) be the number of failing test cases in which \(s\) is executed, \(n_f(s)\) the number of failing tests in which \(s\) is not executed, and \(e_p(s)\) the number of passing witness tests in which \(s\) is executed. The suspiciousness score is
\[
\text{sus}(s) = \frac{e_f(s)}
{\sqrt{(e_f(s)+n_f(s)) \cdot (e_f(s)+e_p(s))}}.
\]

In Wit-HW there is exactly one failing test case \(C_b\), and the ranking focuses on statements touched by \(C_b\). For those statements,
\[
e_f(s)=1,\qquad n_f(s)=0,
\]
so the formula simplifies to
\[
\text{sus}(s) = \frac{1}{\sqrt{1 + e_p(s)}}.
\]
A statement executed in many passing witnesses receives a lower suspiciousness score, while a statement executed only in the failing run remains highly suspicious. This means that the quality of the witness set directly determines the quality of the localization.

A common misconception is that any passing test case is useful. Wit-HW explicitly rejects that view. Passing tests must be effective witnesses: they should remain close to the failing execution so that they exercise relevant logic, but they should also be diverse enough to avoid repeating the same uninformative behavior.

## 3. Similarity, diversity, and execution-distance criteria

Wit-HW defines two criteria for effective witness test cases. The first is similarity: each witness should have a hardware execution trace similar to the failing test \(C_b\). The second is diversity: the witness set should not collapse to one narrow behavioral variant.

Similarity is measured with two components. The first is coverage distance. Verilator reports statement hit counts for the failing test and a candidate witness:
\[
H_b = \{h_b^1, h_b^2, \ldots, h_b^z\}, \qquad
H_w = \{h_w^1, h_w^2, \ldots, h_w^z\}.
\]
Coverage distance is the Euclidean distance
\[
D_{cov}(C_w, C_b) = \sqrt{\sum_{i=1}^{z} (\,h_w^i - h_b^i\,)^2}.
\]

Coverage alone is insufficient for sequential RTL, because it ignores the timing and ordering of state transitions. Wit-HW therefore adds a state distance. For each register, its value transitions under a test case are represented as a sequence
\[
Q = \{(t_1, v_1), (t_2, v_2), \ldots\}.
\]
Given sequences \(Q_w\) and \(Q_b\) for one register, Dynamic Time Warping is computed via
\[
D(i, j) = (Q_w[i] \neq Q_b[j]) + \min
\begin{cases}
D(i-1, j) \\
D(i, j-1) \\
D(i-1, j-1)
\end{cases}
\]
and
\[
DTW(Q_w, Q_b) = D(l_w, l_b),
\]
where \(l_w\) and \(l_b\) are the sequence lengths. For a design with \(k\) registers, the state distance is
\[
D_{state}(C_w, C_b) = \sqrt{\sum_{i=1}^{k} (DTW(Q_w^i, Q_b^i))^2}.
\]

The overall execution distance is then
\[
\text{Distance}(C_w, C_b) = \alpha \cdot D_{cov}(C_w, C_b) + \beta \cdot D_{state}(C_w, C_b),
\]
with normalized weights
\[
\alpha = \frac{1}{z},\qquad \beta = \frac{1}{k}.
\]
The fitness of a candidate test case is
\[
\text{Fitness}(C) = \frac{1}{1 + \text{Distance}(C, C_b)}.
\]
High fitness means that the candidate remains close to the failing run in execution behavior.

Diversity is measured relative to the current seed witness set \(\mathcal{S} = \{C_1,\ldots,C_m\}\). For a new candidate \(C'\),
\[
\text{score}(C') = \frac{1}{m} \sum_{i=1}^{m} \text{Distance}(C', C_i).
\]
A large score means that \(C'\) contributes new behavior rather than duplicating existing witnesses. This diversity score is later used to bias where future mutations are applied.

## 4. Mutation-based witness generation and framework workflow

Wit-HW generates witness tests by mutating the input stimuli of the failing test case. For a test case
\[
C = \big((I_1^1,\ldots,I_n^1), (I_1^2,\ldots,I_n^2), \ldots, (I_1^t,\ldots,I_n^t)\big),
\]
and a selected set of mutation positions \(T_m \subseteq \{1,\ldots,t\}\), each input signal at each selected cycle is mutated with probability \(p=0.5\):
\[
I_i^j =
\begin{cases}
\text{random value}, & \text{if } j \in T_m \text{ with probability } p, \\
I_i^j, & \text{if } j \in T_m \text{ with probability } 1-p.
\end{cases}
\]

Seed test cases are not chosen uniformly. If the current seed set is \(\mathcal{S} = \{C_1,\ldots,C_m\}\), the probability of selecting seed \(C\) is
\[
p_s(C) = \frac{\text{Fitness}(C)}{\sum_{j=1}^{m} \text{Fitness}(C_j)}.
\]
This biases the search toward seeds whose executions remain close to the failing trace. Mutation positions are also prioritized. If the test length is \(t\), the probability of selecting position \(i\) is
\[
p_m(i) = \frac{Priority(i)}{\sum_{j=1}^{t} Priority(j)}.
\]

Position priority is updated by Exponential Weighted Moving Average. If a mutation yielding \(C'\) touched positions \(i \in T_m\), then
\[
Priority(i) = (1-\gamma) \cdot Priority(i) + \gamma \cdot \text{score}(C'),
\]
with \(\gamma=0.1\). Positions that repeatedly generate diverse witnesses become more likely to be mutated again.

Operationally, the framework initializes the seed set with the failing test case \(C_b\), initializes priorities to \(\delta=1.0\), and iterates test generation up to 100 iterations. Each candidate test is simulated on the buggy design and compared against the golden reference. Passing candidates with non-zero distance from existing seeds are added to the witness seed set. Failing candidates are stored separately. If the number of passing seeds is non-zero, the top \(N=10\) witnesses with highest fitness are selected for localization; otherwise the framework falls back to the failing-set path and repeats. The implementation uses Verilator v5.027 to collect statement coverage and VCD waveforms [2508.14414].

The full workflow has two stages. Stage 1 generates witness test cases. Stage 2 applies spectrum-based localization to the failing test \(C_b\) and the witness set \(\mathcal{P}\), computes \(e_p(s)\) for each statement, and ranks statements by the simplified Ochiai score.

## 5. Benchmarks, empirical performance, and ablations

Wit-HW was evaluated on 41 bugs from 10 hardware designs spanning easy, medium, and hard categories. The benchmark suite includes combinational designs such as `decoder_3_to_8` and `alu`, small sequential designs such as `counter`, `led_controller`, `arbiter`, and `fsm_16`, and larger designs such as `sdram_controller`, `sha3`, `i2c`, and `reed_decoder`. The largest design in the evaluation has 4366 LOC.

Across all 41 bugs, Wit-HW localize 49%, 73%, and 88% of bugs within Top-1, Top-5, and Top-10 ranks, with MAR \(=16.90\) [2508.14414]. The same evaluation reports the following baseline results: Tarsel achieves Top-1/Top-5/Top-10 of 22%/39%/54% with MAR \(=35.26\); RTL-Repair achieves 15%/17%/39% with MAR \(=34.23\); and a random-generation variant, Wit-HW\(_{rand}\), achieves 34%/59%/80% with MAR \(=20.80\). The hard-category results are particularly revealing: Wit-HW obtains 3/8/10 bugs at Top-1/Top-5/Top-10, whereas Tarsel reaches 0/2/4 and RTL-Repair 1/1/4.

The ablation study shows that each component contributes. Removing coverage distance (\(\alpha=0\)) reduces the overall result to Top-1/Top-5/Top-10 of 12/24/32 with MAR \(=20.41\). Removing state distance (\(\beta=0\)) gives 17/24/32 with MAR \(=19.48\). Removing diversity guidance and using uniform mutation-position probability gives 15/28/34 with MAR \(=18.02\). The framework therefore benefits from combining statement-hit similarity, sequential-state similarity, and diversity-aware mutation rather than using any one of them alone.

The paper also studies the distance distributions of generated witnesses. Guided Wit-HW produces witness test cases with significantly smaller distances to the failing test than Wit-HW\(_{rand}\), and the localization metrics improve rapidly up to about 100 iterations before plateauing. Runtime is higher than the baselines because of repeated simulation: the average per-bug runtime is 45.7 s for Wit-HW, compared with 1.7 s for Tarsel and 2.1 s for RTL-Repair. A plausible implication is that Wit-HW trades simulation time for a substantially smaller manual debugging burden in the final ranked result.

On 13 real-world bugs collected from open-source hardware projects, Wit-HW localizes 6 bugs at rank 1 or tied rank 1 and 9 bugs within Top-10. The paper describes this as robust performance, especially because RTL-Repair repaired only 2 of those bugs.

## 6. Relation to prior debugging methods, limitations, and future directions

Wit-HW is distinct from repair-oriented methods such as CirFix and RTL-Repair. Those methods use slicing to identify a repair region and then search for a patch, whereas Wit-HW focuses on precise statement ranking. It is also distinct from Tarsel. Tarsel is time-aware and spectrum-based, but still relies on a single failing test. Wit-HW generalizes the evidence available to spectrum-based localization by generating passing witness traces around the failing scenario. Relative to data-driven approaches such as VeriBug, Wit-HW does not require training data and operates directly from the buggy RTL, a validator, and one failing test case.

The framework has clear assumptions. It is simulation-based, assumes the existence of a failing test case, and requires a validator such as a correct RTL design. The implementation is for Verilog, and the paper notes that some SystemVerilog features in external benchmarks were not supported.

Its limitations are also explicit. Constant-execution bugs remain difficult: if the buggy statement is executed in every cycle under both failing and passing tests, witness generation cannot create the differential spectra needed by SBFL. Condition dependency is another hard case: when the execution conditions of buggy and innocent statements are tightly coupled, passing witnesses may not separate them. Complex protocols pose a different problem: random signal-level mutation can violate protocol structure and generate invalid or uninformative tests. Multiple concurrent bugs also degrade witness quality, because candidates either still fail or diverge too far from the original scenario.

The authors suggest several extensions. One is to combine Wit-HW’s suspicious region with mutation-based bug localization so that mutants are restricted to high-suspicion statements. Another is to replace low-level signal mutation with higher-level mutation, concolic testing on RTL, or LLM-aided test generation. They also propose using Quick Error Detection traces to shorten the failing scenario before witness generation, and extending the implementation toward broader HDL support. These directions suggest that Wit-HW is less a replacement for existing hardware debugging infrastructure than a new localization layer centered on witness generation and contrastive execution analysis.

Source: https://www.emergentmind.com/topics/wit-hw