HiFuzz: Hierarchical CPU Fuzzer

• HiFuzz is a hierarchical reinforcement learning-based CPU fuzzer that generates structured, semantically rich test programs for RISC-V verification.
• It employs a two-level generation architecture with a Program Agent and a Basic Block Agent to overcome sparse feedback and maintain semantic validity.
• Empirical results on Rocket, BOOM, and CVA6 demonstrate significant coverage improvements and enhanced bug detection compared to mutation-based and random approaches.

HiFuzz is a hierarchical reinforcement learning–based CPU fuzzer designed to generate structured, semantically rich test programs that drive modern processors into deep architectural and microarchitectural states. Introduced for simulation-based verification of RISC-V cores, it replaces mutation-based input construction with a two-layer generation process consisting of a Program Agent for global program layout and a Basic Block Agent for local instruction composition, while coupling this hierarchy with a semantic-aware basic block encoder and an adaptive coverage reward mechanism to address sparse feedback and reward bias (Wang et al., 7 Jul 2026).

1. Verification setting and problem formulation

HiFuzz is motivated by the verification difficulty posed by modern CPU state spaces, especially in RISC-V cores such as Rocket, BOOM, and CVA6, whose behaviors depend on complex pipelines, speculation, FPU units, caches, CSRs, and privilege modes. In this setting, simulation-based testing remains central, and fuzzing has become an important tool, but the paper identifies three recurrent obstacles: semantic fragility in mutation-based generation, sparse and delayed rewards, and reward masking by easily covered modules (Wang et al., 7 Jul 2026).

The mutation-based fuzzers discussed in the work—RFUZZ, DifuzzRTL, ProcessorFuzz, and TheHuzz—start from seed programs and randomly mutate instructions. The reported consequence is that mutation often destroys semantic validity, including control-flow structure, legal CSR transitions, and dependency relationships. Many generated programs therefore never complete or never reach deep pipeline stages. Coverage feedback arrives only after executing full programs, which yields sparse and delayed rewards for the learning loop. The paper further notes that coverage signals are often dominated by “easy” structures such as frequently updated CSRs like instret and fflags, producing reward bias toward modules that are already easy to revisit.

HiFuzz is also positioned against random constructive generation, represented by Cascade. Constructive generation improves program completion rate because it builds syntactically valid instruction streams from scratch, but it is still largely heuristic and does not learn from prior executions. The paper argues that flat RL over token streams, as in ChatFuzz and GenHuzz, faces a large combinatorial action space, strict ISA-level structural dependencies, sparse feedback, and the masking effect induced by optimizing a single global coverage counter. HiFuzz is therefore designed around three explicit objectives: hierarchical decomposition of generation, dense semantic intrinsic rewards at the basic-block level, and adaptive module-aware extrinsic rewards.

2. Hierarchical generation architecture

HiFuzz uses a two-level hierarchical RL architecture. The high-level Program Agent operates at episode start and selects a global program configuration, including memory footprint, number of basic blocks, basic-block length distribution, instruction-type distribution, control-flow template, and privilege mix. The low-level Basic Block Agent acts once per basic block and chooses a parameterization of the instruction-type mix and the basic-block termination mode. Concrete instructions are not emitted directly by the agents; instead, a constrained generator built on Cascade-like constructive generation instantiates valid instructions that satisfy syntactic and semantic constraints (Wang et al., 7 Jul 2026).

The Program Agent is trained with Rainbow DQN. Its state is a compact summary $S_P \in \mathbb{R}^{11}$ of recent generation statistics, including basic-block completion rates, instruction completion rates, basic-block-size moments, program quality metrics, and training progress indicators. Its action space is discrete, with 1,375 possible combinations formed as the cross-product of 5 memory-footprint levels, 5 basic-block-count levels, 11 per-basic-block length distributions, and 5 instruction-type distributions. The associated Q-function is $Q_\theta(S_P, A_P)$.

The Basic Block Agent is trained with PPO using a multi-head actor–critic design. Its state $S_{BB} \in \mathbb{R}^{12}$ includes the remaining instruction budget in the current program and current basic block, the current privilege level, and a history of recently generated instruction types. Its action $A_{BB}$ is continuous and parameterizes a distribution over instruction categories—such as integer, memory, CSR, FP, and branch—and a distribution over termination modes such as branch, jump, and exception. The policy is written as $\pi_\phi(a_{BB} \mid s_{BB})$.

The hierarchy is formalized through Sutton’s options framework. The Program Agent selects a macro-action that defines a subtask, namely generation under a chosen global configuration, and the Basic Block Agent implements the intra-option policy. A common misconception is that HiFuzz is simply an RL token generator; the paper explicitly states that the low-level agent does not output raw opcodes, and semantic legality is preserved by the constrained generator.

3. Semantic-aware basic block modeling

To mitigate reward sparsity at instruction granularity, HiFuzz introduces a Semantic-Aware Basic Block Encoder that is DUT-agnostic, trained offline, and reused across different cores. Its input is a structured RISC-V tokenization rather than raw assembly text. Each instruction is decomposed into operand roles such as <dsts>, <srcs>, <mem>, <csr>, and <const>, together with execution-unit annotations such as INT_ALU, LOAD, STORE, BRANCH, and FPU, CSR symbolic names such as mstatus and mtvec, and instruction-boundary markers <s> and </s>. This yields a vocabulary of about 343 tokens covering the RISC-V G subset (Wang et al., 7 Jul 2026).

The encoder backbone is a 2-layer Bi-LSTM with attention-based pooling. The embedding dimension is 64, the LSTM hidden dimension is 128 per direction for 256 combined, and the output is a 96-dimensional L2-normalized embedding. The paper states that Bi-LSTM is chosen because basic blocks are short, single-entry single-exit sequences whose semantics are dominated by local data and control dependencies, and because this design is smaller and faster than transformer and RWKV alternatives while converging well in the two-stage training pipeline.

Training proceeds in two stages. Stage 1 is self-supervised masked language modeling on more than 50K unlabeled RISC-V basic blocks from Cascade, using both token-level masking of about 15% of tokens and instruction-level masking. Stage 2 is supervised similarity learning against a hardware-aware similarity metric, BB-Sim, which is based on dynamic-programming sequence alignment and a dependency bonus. The final similarity is

$$S(A,B) = \frac{S_{\text{seq}} + S_{\text{dep}}}{\max(n,m)},$$

and the encoder is trained so that cosine similarity approximates this score:

$$\mathcal{L}_{\text{sim}} = \left(\cos(E(BB_1), E(BB_2)) - S(BB_1, BB_2)\right)^2.$$

During fuzzing, the encoder is frozen and used for intrinsic novelty estimation. If $E(bb)$ is the embedding of a generated basic block and $\{C_k\}$ are online-maintained cluster centers representing explored semantic regions, then the intrinsic reward is

$$R_{\text{int}}(bb) = \min_k \left(1 - \cos\big(E(bb), C_k\big)\right).$$

This reward is immediate and does not require RTL execution. Correlation studies reported in the paper show Pearson $Q_\theta(S_P, A_P)$0 between embedding cosine similarity and BB-Sim, which the authors interpret as evidence that the embedding space reflects hardware-relevant similarity.

4. Reward shaping and adaptive coverage control

HiFuzz separates local semantic exploration from global coverage optimization through two reward channels. At the basic-block level, the intrinsic reward comes from encoder-based semantic novelty. Extrinsic coverage reward for the Basic Block Agent is normalized against a dynamic baseline:

$Q_\theta(S_P, A_P)$1

Here, $Q_\theta(S_P, A_P)$2 is the coverage gain attributable to the current program or basic block, $Q_\theta(S_P, A_P)$3 is the mean reward over a recent buffer of episodes, and $Q_\theta(S_P, A_P)$4 is a diminishing factor. The paper presents this as a way to discourage plateau behavior by rewarding relative improvement rather than absolute gain (Wang et al., 7 Jul 2026).

The Basic Block Agent uses a dual-advantage PPO objective with separate intrinsic and extrinsic critics:

$Q_\theta(S_P, A_P)$5

with $Q_\theta(S_P, A_P)$6 and $Q_\theta(S_P, A_P)$7. The total loss is

$Q_\theta(S_P, A_P)$8

The Program Agent receives extrinsic reward from the Adaptive Coverage Reward Mechanism, or ACRM. ACRM decomposes coverage by module and uses module-level line, toggle, MUX, and control-register coverage gathered through Verilator and FIRRTL-style passes. Treating each module as an arm in a multi-armed bandit, it defines a historical mean coverage gain $Q_\theta(S_P, A_P)$9 and a saturation term

$S_{BB} \in \mathbb{R}^{12}$0

An intermediate score is then computed as

$S_{BB} \in \mathbb{R}^{12}$1

normalized with a softmax temperature $S_{BB} \in \mathbb{R}^{12}$2:

$S_{BB} \in \mathbb{R}^{12}$3

The Program Agent’s extrinsic reward is

$S_{BB} \in \mathbb{R}^{12}$4

The stated purpose is to shift reward mass away from already saturated modules and toward under-covered ones such as the FPU and complex cache structures. The Program Agent is trained with Rainbow DQN using the standard DQN loss, augmented by Rainbow components such as prioritized replay and n-step returns.

5. End-to-end workflow and empirical results

The end-to-end workflow begins with Program-Agent selection of a global configuration. The generator then aligns code and data segments, assigns basic-block start and end addresses, and ensures non-overlapping in-bounds regions. For each basic block, the Basic Block Agent proposes an instruction-type mix and termination distribution, and the constrained generator instantiates concrete instructions consistent with operand dependencies, legal CSR transitions, valid jump and branch targets, and legal memory addresses. A basic-block embedding is computed immediately for intrinsic reward, while the fully generated program is pre-simulated at ISA level with Spike if needed and then executed on RTL through Verilator and a cocotb testbench. Coverage metrics and architectural deviations are collected, ACRM produces extrinsic reward, and Rainbow DQN and PPO update the two agents over 24-hour campaigns (Wang et al., 7 Jul 2026).

Three open-source RISC-V cores are used as DUTs: Rocket, BOOM, and CVA6. Spike serves as the golden model for all three. Baselines include DifuzzRTL, ProcessorFuzz, Cascade, a DQN+PPO hierarchical baseline without the BB Encoder and ACRM, and variants disabling each mechanism separately.

DUT	Characterization	Reported outcome
Rocket	In-order, Chisel-based	Control-register coverage 1,102,343 in 24h
BOOM	Out-of-order, Chisel-based	Significantly higher MUX coverage than baselines
CVA6	64-bit, 6-stage SystemVerilog core with fpnew FPU	Total coverage 55,241, a 7.2% gain over Cascade

The most prominent quantitative result is Rocket control-register coverage after 24 hours: DifuzzRTL reaches 133,971, ProcessorFuzz 103,941, Cascade 740,329, the DQN+PPO baseline 777,916, and HiFuzz 1,102,343. Relative to Cascade, HiFuzz reports a 48.9% improvement in control-register coverage, while coverage per test increases from 34.53 to 116.60, approximately $S_{BB} \in \mathbb{R}^{12}$5. On CVA6, total coverage improves from 51,541 for Cascade to 55,241 for HiFuzz, a 7.2% increase under the same 24-hour budget. On Rocket line and toggle coverage, HiFuzz reaches 75.98% and 67.58%, respectively; the paper characterizes these gains as modest but consistent because these metrics saturate quickly.

Program quality metrics also shift. Prevalence, defined as the fraction of meaningful instructions, rises to 96.7% for HiFuzz, compared with 92.0% for Cascade, 66.5% for ProcessorFuzz, and 6.5% for DifuzzRTL. Average dependency-chain length is 18.5 for HiFuzz, with median 6.0, compared with 9.0 for Cascade and 2.1 for DifuzzRTL and ProcessorFuzz. ISA coverage over RISC-V G reaches 96% for HiFuzz versus 93% for Cascade and 63% for the mutation-based baselines.

The ablation study on Rocket control-register coverage attributes a 34.45% gain over the DQN+PPO baseline to the BB Encoder alone, a 44.15% gain to ACRM alone, and a 51.87% gain to the full system. This indicates that the encoder and ACRM address different bottlenecks: local semantic novelty and module-level reward balancing.

For bug detection, the Encarsia benchmark injects 30 bugs per DUT, split into 15 mix-ups and 15 broken conditionals. HiFuzz detects 12 of 30 on Rocket and 18 of 30 on BOOM, for a total of 30 of 60 across the two cores, compared with 26 of 60 for Cascade and 25 of 60 for DifuzzRTL and ProcessorFuzz. The paper states that HiFuzz particularly outperforms on BOOM, where more complex microarchitectural structures demand richer test programs. On CVA6, HiFuzz also triggers known real-world bugs previously reported by Cascade and GenHuzz.

6. Scope, limitations, and relation to adjacent literature

The evaluation scope is explicitly limited to open-source RISC-V cores at roughly the 1M-gate scale. Generation is single-hart, excludes the A extension, and does not address multi-hart synchronization; concurrency-heavy bugs such as memory-ordering violations are therefore out of scope. Porting the approach to ARM or x86-64 would require redesign of the structured tokenizer and constrained generator to accommodate new opcode categories, register classes, addressing modes, implicit operands, and privilege and exception semantics. ACRM also assumes access to module-level coverage metrics; with only global coverage, it degrades to a global reward and loses its rebalancing function. The paper further notes offline encoder training costs, RL overhead relative to pure constructive methods, and the use of single-seed experiments due to resource cost (Wang et al., 7 Jul 2026).

Within the broader hardware-fuzzing landscape, HiFuzz belongs to a line of work that treats coverage-guided test generation as a practical alternative to undirected constrained-random verification. Earlier work such as “Fuzzing Hardware Like Software” demonstrated that Verilator-translated RTL models can be fuzzed with software-style greybox techniques and reported a two orders-of-magnitude reduction in run time to achieve FSM coverage on selected OpenTitan IPs, while exceeding 88% HDL line coverage in three out of four designs even without initial seeds (Trippel et al., 2021). HiFuzz differs in targeting CPU verification specifically and in moving from mutation or bus-transaction fuzzing to semantically constrained program synthesis guided by hierarchical RL.

The name “HiFuzz” is also potentially ambiguous in bibliographic search. A distinct framework, “HIFuzz: Human Interaction Fuzzing for small Unmanned Aerial Vehicles,” uses the same name for a multi-level human-in-the-loop CPS testing methodology rather than CPU fuzzing (Chambers et al., 2023). This naming overlap does not indicate methodological continuity; the 2026 HiFuzz system is specifically a hierarchical reinforcement learning framework for semantic-aware and adaptive CPU fuzzing.