NecoFuzz: Fuzzing Framework for Nested Virtualization

• NecoFuzz is a fuzzing framework that targets nested virtualization logic in hypervisors by generating VM instances with states near the validity boundary.
• It employs specification-guided, boundary-oriented VMCS mutations and extends AFL++ with hardware oracle feedback to achieve high code coverage on Intel VT-x and AMD-V platforms.
• NecoFuzz uncovered six previously unknown vulnerabilities across major hypervisors, demonstrating its practical impact on enhancing cloud security.

NecoFuzz is a fuzzing framework for nested virtualization-specific logic in hypervisors. It addresses a setting in which nested virtualization is widely supported by major cloud vendors, allowing users to leverage virtualization-based technologies in the cloud, while simultaneously increasing host hypervisor complexity and introducing a new attack surface in cloud platforms. NecoFuzz is presented as the first fuzzing framework that systematically targets nested virtualization-specific logic in hypervisors. Its central mechanism is the synthesis of executable fuzz-harness VMs whose internal states are near the boundary between valid and invalid, guided by an approximate model of hardware-assisted virtualization specifications. Implemented on Intel VT-x and AMD-V by extending AFL++ to support fuzz-harness VMs, it achieved 84.7% and 74.2% code coverage for nested virtualization-specific code on Intel VT-x and AMD-V, respectively, and uncovered six previously unknown vulnerabilities across three hypervisors, including two assigned CVEs (Ishii et al., 9 Dec 2025).

1. Problem setting and research target

Nested virtualization concerns the execution of a virtual machine inside another virtual machine. In NecoFuzz’s terminology, the host layer is “L0”, the guest that generates and executes nested inputs is “L1”, and the nested guest is “L2”. The paper situates the problem in cloud platforms, where support for nested virtualization expands the exposed hypervisor attack surface and complicates the handling of virtualization control structures, CPU feature configurations, and nested entry and exit paths (Ishii et al., 9 Dec 2025).

The stated research gap is that, while many prior studies have explored hypervisor fuzzing, none has explicitly addressed nested virtualization due to the challenge of generating effective virtual machine instances with a vast state space as fuzzing inputs. NecoFuzz targets this gap by treating VM instances themselves as structured fuzzing inputs and by focusing on states that are neither trivially valid nor obviously rejected. This emphasis reflects the paper’s premise that vulnerabilities in nested virtualization often stem from incorrect handling of unexpected VM states.

A common simplification in hypervisor fuzzing is to concentrate on syscall-driven or broadly random input generation. NecoFuzz explicitly rejects that emphasis for nested virtualization-specific code. Its design instead centers on specification-guided, boundary-oriented generation, with the stated goal of improving coverage of security-critical code across different hypervisors.

2. System architecture and execution model

NecoFuzz is built as a three-layered stack comprising a host layer, a VM generator layer, and a nested guest layer (Ishii et al., 9 Dec 2025).

At the host layer, or “L0”, the hypervisor under test is a bare-metal hypervisor such as KVM, Xen, or VirtualBox. An agent program in the host OS acts as an AFL++ fork-server: it receives 2 KiB fuzz payloads, launches the fuzz-harness VM, collects coverage via kcov/gcov, and reports back to AFL++. The same agent monitors kernel logs, including KASAN, UBSAN, and Xen diagnostics, to detect anomalies. This arrangement relocates the ordinary fuzzer–target interaction from process execution to VM lifecycle control.

The VM generator layer resides in guest “L1” and contains three principal components. The first is the VM Execution Harness, a UEFI-statically-linked executor of approximately 3.5 KLOC. It interprets a small DSL template of VMX/SVM instructions, including vmxon, vmclear, vmlaunch, vmresume, cpuid, and mov cr0, and it mutates instruction order and operands to fuzz nested-entry and exit paths. The second is the VM State Validator, extracted and adapted from approximately 2.5 KLOC of Bochs VMCS/VMCB validation logic. This validator rounds raw fuzz bits to a “mostly valid” VM state and then selectively flips a small number of bits near the validity boundary. It also uses the real CPU as an oracle: it writes the candidate VMCS, attempts vmlaunch, and reads back VMCS to refine the validator at runtime. The third is the vCPU Configurator, which generates bit-masks of CPU features such as EPT, unrestricted guest, and AVIC from fuzz input, and, via a hypervisor-specific adapter, reloads kvm-intel.ko or kvm-amd.ko with new module parameters or relaunches QEMU with updated -cpu flags.

The fuzzing core is an extended AFL++. The AFL++ fork-server is modified to handle VM launch and teardown instead of execve. Its shared-memory bitmap is adapted for kcov/gcov traces, and a new snapshot/restore mechanism is introduced to roll back the host hypervisor on crash. The nested guest layer, “L2”, contains a minimal guest OS and fuzzed instructions, yielding an execution model in which structured virtualization state, instruction streams, and CPU feature configurations are jointly mutated.

3. Approximate model of virtualization specifications

NecoFuzz models the space of all possible VMCS or VMCB bit-vectors as a finite state space (Ishii et al., 9 Dec 2025). Let $N$ be the total width in bits of the VM control structure, with the example $N \approx 8\,000$, and define

$S = \{\, s \in \{0,1\}^N \}\,.$

The CPU defines a validity predicate $$\mathit{valid\_hw}(s)\in\{\text{true},\text{false}\}$$ via approximately 50 pages of Intel/AMD SDM prose plus silent “round-to-nearest” corrections. NecoFuzz then defines a boundary region between valid and invalid states:

$$B = \{\, s \in S \mid \exists\, b\;\text{(small Hamming-ball)}:\; b(s)=s',\;\mathit{valid\_hw}(s')\neq \mathit{valid\_hw}(s)\}\,.$$

Intuitively, $B$ contains states for which flipping a few bits can switch validity. The paper further defines a distance scoring function

$$f: S \longrightarrow \mathbb{R}_{\ge0},\quad f(s)=\mathrm{Ham}\bigl(s,\mathrm{RoundToValid}(s)\bigr)$$

where $\mathrm{RoundToValid}$ applies Bochs-based corrections so that $$\mathit{valid\_hw}(\mathrm{RoundToValid}(s))=\text{true}$$. A good fuzz input lands in the “fuzzy shell” around the boundary:

$f_{\min} \le f(s) \le f_{\max} \quad\mbox{with}\quad f_{\min},f_{\max}\in [1,\,20]\,.$

This formulation operationalizes the paper’s stated strategy: the VMCS should be neither trivially valid nor obviously rejected, so that nested-validation code is exercised more deeply. The use of a real hardware oracle is significant because the validity predicate is not represented as a closed-form specification; it is mediated by SDM prose and silent hardware behavior. This suggests that NecoFuzz treats specification conformance as an approximate, runtime-refined target rather than as a purely static constraint system.

4. Fuzz-harness VM synthesis and AFL++ extension

NecoFuzz’s VM synthesis procedure is summarized as “Boundary-Oriented VMCS Mutation” (Ishii et al., 9 Dec 2025). Starting from a seed state $N \approx 8\,000$0 and raw fuzz bits $N \approx 8\,000$1, the procedure first computes RoundToValid(s₀) using a Bochs-derived validator. It then selects fields according to SelectFields(b[1]), choosing $N \approx 8\,000$2 fields with $N \approx 8\,000$3. For each selected field, it computes an offset from subsequent fuzz bits, flips the selected bit, and updates the candidate state. If the result remains hardware-valid, the candidate is kept to push deeper toward the invalid boundary; otherwise it is pushed back toward the valid boundary by another RoundToValid. The main fuzzing loop maintains a priority queue of seeds ordered by coverage metric $N \approx 8\,000$4, obtains new AFL++ inputs, mutates a seed, launches the VM, collects coverage, updates the queue when new basic blocks are hit, and saves crashing states when the agent detects an anomaly.

Three implementation details are emphasized. First, RoundToValid uses Bochs’ VMenterLoadCheck... routines to correct control, host-state, and guest-state fields in sequence. Second, mutation size is kept small, specifically 1–8 bits across up to 3 fields, to stay near the boundary region $N \approx 8\,000$5. Third, AFL++’s coverage map guides seed selection, but NecoFuzz also explores breadth-first around the boundary.

The AFL++ integration is motivated by three explicit challenges: AFL++ assumes fork()/exec() of native binaries, but NecoFuzz must launch QEMU/KVM or Xen hypervisor with a UEFI image; coverage must be harvested from the host hypervisor via kcov/gcov rather than from a normal process; and crashes of the host kernel require automatic rollback through watchdog and host reboot mechanisms. The implementation therefore replaces the ordinary child-process execution path with VM launch via qemu-system-x86_64, uses a shared-memory bitmap adapted for kcov/gcov traces, and adds snapshot/restore support. An example configuration exposes CPU virtualization features through AFL_VCPUs="--cpu host,vmx=on,kvm-pv=off,vmware=off", and the host agent initializes coverage collection through /sys/kernel/debug/kcov.

The paper characterizes this synthesis method as “specification-guided, boundary-oriented generation.” In context, that phrase denotes the joint use of an approximate VMCS/VMCB specification, minimal structured mutations, and runtime hardware feedback to construct executable fuzz-harness VMs rather than opaque byte streams.

5. Coverage results and ablation analysis

The evaluation focuses on nested-specific code under linux/arch/x86/kvm/{vmx,nested.c} or xen/arch/x86/hvm/{vmx,nestedsvm.c} and defines

$N \approx 8\,000$6

The primary coverage results are reported as a 48 h median of five runs (Ishii et al., 9 Dec 2025).

Fuzzer	Intel VT-x	AMD-V
NecoFuzz	84.7 %	74.2 %
Syzkaller	61.4 %	7.0 %
IRIS	52.3 %	–
Selftests	57.8 %	73.4 %
KVM-unit-tests	72.0 %	69.8 %

The corresponding code totals are 1,681 lines for Intel VT-x nested-specific code and 387 lines for AMD-V nested-specific code. Statistical significance is reported with Mann–Whitney U: Intel $N \approx 8\,000$7, Cohen’s $N \approx 8\,000$8; AMD $N \approx 8\,000$9, $S = \{\, s \in \{0,1\}^N \}\,.$0. Over the 48 h coverage curve, NecoFuzz on Intel quickly rose from approximately 65 % to 84.7 %, while Syzkaller plateaued at approximately 61 %. On AMD, NecoFuzz reached 74.2 %, whereas Syzkaller remained at approximately 7 %.

A 24 h ablation study isolates the contribution of the major components. The full configuration achieved 84.7 % on Intel and 74.2 % on AMD. The “harness only” variant achieved 78.6 % and 54.0 %; the “validator only” variant 67.8 % and 58.4 %; the “vCPU configurator only” variant 73.7 % and 68.2 %; and the “none (static template/seed)” baseline 56.5 % and 51.7 %.

These results are presented by the paper as evidence that all three components contribute materially. The strongest interpretive claim made in the paper is that, while kcov/gcov feedback helps prune seeds, most coverage gains arose from exploring new VMCS boundary states rather than deeper CFG paths. A plausible implication is that, in nested virtualization fuzzing, state-space construction is the primary difficulty, whereas ordinary coverage guidance is supportive but not sufficient.

6. Vulnerabilities, security insights, and operational recommendations

NecoFuzz found six new bugs across three hypervisors (Ishii et al., 9 Dec 2025). In KVM on Intel, it identified a missing VMCS12 CR4.PAE check, detected as a UBSAN crash and assigned CVE-2023-30456, later patched. The paper’s highlight for this case states that L1 sets IA-32e-mode=1 but leaves CR4.PAE=0; the CPU silently accepts, KVM misinterprets page-walk code, and the consequence is an out-of-bounds write. In VirtualBox on Intel, it found that nested entry MSR load (KernelGSBase) accepted a non-canonical address, causing a host general protection fault; this issue was assigned CVE-2024-21106 and patched. In KVM on Intel/AMD, it identified EPTP page-table root misuse, detected as an assertion failure and fixed in 0e3223d8d.

Xen-specific findings span both Intel and AMD paths. On Intel, an unsanitized activity-state=WAIT condition, summarized in the paper’s highlights as activity-state=3 (WAIT-FOR-SIPI) not filtered, caused a host reset or hang and was fixed via a Xen patch. On AMD, a broken VMCB CR0/EFER interlock caused an assertion failure and was confirmed as issue #215; the highlighted boundary state is LME=1 ∧ PG=0. A second Xen AMD issue involved a VGIF inject logic assumption, also detected as an assertion failure and confirmed as issue #216; the highlighted boundary state is VGIF=1∧VGIF_state=0.

The paper states that all bugs were triggered by NecoFuzz’s near-boundary VMCS mutations combined with atypical vCPU configurations. This connects directly to the framework’s design rationale: the most security-relevant nested states are often those lying near the interface between hardware acceptance and hypervisor-side semantic assumptions.

The security insights are stated explicitly. “Boundary-Aware Mutation → Highest ROI” asserts that mutating the entire VMCS while keeping inputs near “just valid” and “just invalid” exercises deep validation logic normally untested by random or syscall-driven fuzzers. “Real Hardware Oracle for Validator” states that using the physical CPU to refine the validator closes semantic gaps caused by silent rounding and undocumented constraints. “Hypervisor-Agnostic VM Generator” states that a single Bochs-based validator and instruction harness ports easily across KVM, Xen, and VirtualBox, enabling consistent coverage. “Coverage Guidance Secondary to State Diversity” states that the principal gains come from exploring new VMCS boundary states. “Minimalism & Repeatability” states that a UEFI-only executor with no guest OS and a self-contained fat UEFI binary simplifies reset and retry, improving fuzz throughput.

The accompanying operational recommendations are correspondingly concrete: integrate a specification-based VMCS/VMCB model into the fuzzer; automate host-side hypervisor recovery through watchdog or snapshot rollback; combine small, structured instruction-stream templates with random operand mutation; expose and mutate all vCPU feature flags, not just VMCS fields; and regularly test on bare metal with sanitizers enabled, including KASAN, UBSAN, and Xen assertions. In the paper’s own conclusion, systematic, boundary-oriented VM state generation effectively uncovers deep bugs in nested virtualization implementations, suggesting a new direction for hypervisor fuzzing in production cloud environments.