SmartFuzz: Semantically Guided Fuzzers

• SmartFuzz is a semantically guided fuzzing framework that combines domain-specific grammar and LLM-driven parameter interpretation to generate valid, high-value test inputs.
• In the 5G context, it employs AFL++ with a custom JSON grammar for targeted mutations, boosting unique code coverage by 10–20% and identifying critical crashes within hours.
• For Ethereum smart contracts, its multi-agent reflection process orchestrates expert-driven adjustments that significantly enhance vulnerability detection and reduce false negatives.

SmartFuzz refers to two distinct but conceptually related fuzzing frameworks: (1) a methodology targeting the 5G OpenAirInterface5G (OAI5G) software stack, combining grammar-aware mutation and language-model-driven parameter interpretation (Wu et al., 2023), and (2) a multi-agent, reflection-driven framework for vulnerability discovery in Ethereum smart contracts (Chen et al., 15 Nov 2025). Both instantiations depart from conventional coverage-guided fuzzers by integrating semantic guidance, machine learning, or collaborative agent-based orchestration to maximize bug and vulnerability discovery along semantically valid execution paths.

1. Conceptual Foundations

SmartFuzz in both contexts aims to overcome the core limitation of classical fuzzing: the tendency to produce syntactically or semantically invalid inputs that do not exercise meaningful or vulnerable program logic. By embedding domain-specific knowledge—be it via explicit configuration grammars, automated parameter documentation, or LLM agent collaboration—SmartFuzz systematically drives test exploration toward deeper, semantically correct states associated with higher defect density or vulnerability likelihood.

In OAI5G, SmartFuzz leverages AFL++ with a hand-crafted grammar, acting only on valid configuration file structures (Wu et al., 2023). In smart contract fuzzing, SmartFuzz implements a multi-agent system whose members maintain semantic contract awareness and continuously reflect on runtime feedback to steer future input generation (Chen et al., 15 Nov 2025).

2. Architecture and Workflow in 5G Wireless Protocol Fuzzing

The SmartFuzz methodology for OAI5G consists of two tightly integrated components:

• AFL++ Grammar-Aware Fuzzing:

The OAI5G codebase is rebuilt using afl-clang-fast instrumentation to enable precise edge-coverage feedback. A custom JSON grammar encodes the permissible configuration structure, enabling the AFL++ grammar mutator (drawing on F1 and Nautilus methods) to alter parse trees in a type- and context-sensitive fashion without violating schema. Mutations include token flipping, value delta perturbations, and subtree swaps across sections such as <radioParameters> and <carrierConfig>. Coverage-increasing inputs are retained, while crash- or hang-triggering seeds are cataloged for manual triage. The main fuzz loop, formalized as:

algorithm SmartFuzzFuzz(Seeds, Grammar, Timeout): Queue ← Seeds Corpus ← ∅ while not Timeout.expired(): Input ← AFL_Select(Queue) ParseTree ← parse(Input, Grammar) Mutant ← GrammarMutate(ParseTree) Outcome ← run_gNB(Mutant) Coverage ← Outcome.coverageBits if Coverage ⊃ Corpus: Corpus ∪= Coverage Queue.enqueue(Mutant) if Outcome.isCrash(): recordCrash(Mutant, Outcome.stack)

(Wu et al., 2023)

• LLM-Driven Parameter Interpretation:

For each exercised configuration parameter, a Google Bard API pipeline queries the meaning of the parameter within OAI5G context. Collected documentation is output as human-readable files, streamlining parameter auditing and rapid analysis. Automated validation (keyword matching) and expert review ensure terminological relevance. A plausible implication is that this approach both reduces manual documentation effort and enhances the quality of subsequent debugging and patch development.

3. Multi-Agent Reflection Architecture in Smart Contract Fuzzing

The SmartFuzz framework for Ethereum contracts formalizes fuzzing as a Continuous Reflection Process (CRP):

$$\mathrm{CRP} = (\mathcal{S}, \mathcal{A}, \mathcal{T}, \mathcal{E}, \mathcal{F}, \pi)$$

• $\mathcal{S}$: set of fuzzing states (transaction sequences)
• $\mathcal{A}$: revision actions (transaction order mutation, argument tweaking, sender/amount swapping)
• $$\mathcal{T}:\mathcal{S}\times\mathcal{A}\to\mathcal{S}$$: deterministic state transitions
• $\mathcal{E}$: EVM, ABI/seedpool, and test oracles
• $\mathcal{F}$: real-time feedback (execution, vulnerability hit)
• $\pi$: collaborative multi-agent policy

Each reflection round $i$ involves:

$$s_{i+1} = \mathcal{T}(s_i,\,\pi(s_i,\mathcal{F}(s_i)))$$

until a vulnerability triggers or resource bounds are exhausted.

Reactive Collaborative Chain (RCC)

The RCC decomposes CRP into subtasks and assigns them to six domain-specific LLM-driven "expert" agents:

• TxSeqDrafter: initial sequence creation
• TxSeqRefiner: global sequence-level edits
• FunChecker: ensures ABI function call validity
• ArgChecker: type/range correction for arguments
• SNDChecker: sender selection from seed pool
• AMTChecker: assignment of payment amounts

The RCC executes in ordered phases: drafting, global reflection, local reflection (per field), and runtime testing, with each agent restricted by permissions tuned to contract and blockchain semantics. Prompt templates for each agent encode task context, execution feedback, and desired output format (Chen et al., 15 Nov 2025).

4. Metrics and Experimental Evaluation

SmartFuzz for OAI5G

Metrics are canonically defined:

• Code Coverage:

$C = \frac{B_{exec}}{B_{total}} \times 100\%$ where $B_{exec}$ is the number of instrumented branches exercised.

• Mutation Rate:

$$\mu = \frac{\text{Number of mutated tokens}}{\text{Total tokens per input}}$$

• Vulnerability Discovery Rate:

$$V = \frac{\text{Number of unique crashes or CVEs found}}{\text{Fuzzing time (hours)}}$$

Empirical observations:

• AFL++ grammar mutator yields 10–20% more unique branch coverage than bit-flipping alone
• First crash observed within 2 hours in 24-hour campaigns
• Five distinct crash-inducing configurations identified in the physical-layer PBCH autotest corpus (Wu et al., 2023)

SmartFuzz for Smart Contracts

Evaluation on datasets D1 (85 contracts: 154 bugs) and D2 (108 contracts, covering six vulnerability classes) demonstrates:

• True positives (TP): 150/154 on D1 (97.4%)
• Improvement over baselines: 5.8–74.7% more vulnerabilities found compared to RLF, ILF, Smartian, sFuzz, SmarTest, and Mythril
• Impact of Reflection: Disabling reflection reduces discovery from 150 to 11 TP; 88% of TP found within first five reflection rounds
• Real-world DApp evaluation: 97.2% true positive rate on D2, with up to 80% reduction in false negatives versus MuFuzz
• Average rounds per DApp: 4.5 rounds, 1–5 minutes per file (Chen et al., 15 Nov 2025)

Tool	TP (D1)	FN (D1)
Mythril	35	119
SmarTest	103	51
Smartian	43	111
ILF	129	25
RLF	141	13
SmartFuzz	150	4

5. Technical Limitations and Challenges

• Both variants depend on the semantic fidelity of their grammar/agent models and, in the case of LLM integration, on the quality and domain adaptation of the underlying LLM.
• OAI5G SmartFuzz addresses parameter complexity (deeply nested, interdependent fields) and long execution times (physical-layer compute intensity) through grammar constraints, type-aware mutations, extended timeouts, and multi-core parallelization (AFL Linux KCM module) (Wu et al., 2023).
• Smart Contract SmartFuzz notes LLM dependency (potential hallucinations after many rounds), high compute cost, and the assumption of well-formed contract ABIs. Dataset biases and potential oracle misclassifications are identified as validity threats (Chen et al., 15 Nov 2025).
• In both contexts, invalid seeds or poorly crafted grammar/seed pools reduce exploration efficacy.

6. Future Directions

Future enhancements as articulated include:

• OAI5G SmartFuzz: extension of the two-stage approach to higher layers (RRC, NAS), comparative studies with alternative LLMs (e.g., ChatGPT), and reinforcement learning-guided mutation strategies (Wu et al., 2023).
• Smart Contract SmartFuzz: integration of static-analysis outputs for prompt conditioning, RCC extension to multi-contract/Cross-Chain workflows, LLM agent fine-tuning via historical failure cases, and adaptive reflection round budgeting (Chen et al., 15 Nov 2025).
• A plausible implication is that broader adoption of agent-based or LLM-guided fuzzing in both software infrastructure and blockchain domains will yield frameworks resilient to both syntactic and semantic invalidity, and may generalize to other high-assurance software targets.

7. Relation to Broader Fuzzing Research

SmartFuzz exemplifies the trend toward semantically informed fuzzing, unifying grammar-driven approaches (as in AFL++, Nautilus, F1) with automated ML- or LLM-mediated analysis. The adoption of multi-agent reflection policies and collaborative orchestration (Reactive Collaborative Chain) surpasses single-agent or pure random-mutation systems, enabling targeted discovery of deep, stateful defects.

Tables and quantitative comparisons in (Chen et al., 15 Nov 2025, Wu et al., 2023) evidence significant gains over purely coverage-based or symbolic execution approaches, highlighting the value of combining domain grammar, semantic validation, and AI-assisted input generation in complex software and decentralized application contexts.

---

References:

• "Smart Fuzzing of 5G Wireless Software Implementation" (Wu et al., 2023)
• "Multi-Agent Collaborative Fuzzing with Continuous Reflection for Smart Contracts Vulnerability Detection" (Chen et al., 15 Nov 2025)