SV-LLM: Automated SoC Security Verification

• SV-LLM is a multi-agent system that integrates LLM capabilities to automate and streamline register-transfer level security verification in SoC hardware.
• The architecture leverages specialized agents for asset identification, threat modeling, vulnerability detection, and simulation-based testbench generation.
• Empirical evaluations show significant improvements in automation and accuracy, reducing manual intervention and enhancing overall hardware security verification.

SV-LLM refers to a multi-agent, LLM-based system specifically designed to automate and enhance the process of security verification in system-on-chip (SoC) hardware, particularly at the register-transfer level (RTL). SV-LLM represents a significant integration of advanced LLM capabilities—including natural language understanding, code generation, domain-specific fine-tuning, and retrieval-augmented reasoning—across the entire security verification lifecycle for hardware systems. The architecture and operational paradigm of SV-LLM transform hardware security verification from a manual, fragmented, expert-driven workflow into a coordinated, automated, and data-driven process, demonstrated via practical case studies and quantitative evaluation.

1. Multi-Agent System Architecture

SV-LLM implements a layered and modular architecture composed of distinct yet interconnected agents, each responsible for a specific facet of SoC security verification. This structure enables parallelization, specialization, and iterative refinement across the security flow.

• Layered Design: The system is organized into six layers:
  1. Application (user interface and input/output),
  2. Supervisor (context interpretation, input completion, task planning),
  3. Orchestrator (execution management, agent coordination and sequencing),
  4. Agent (specialized LLM-driven modules),
  5. Data (knowledge bases, vector stores, design repositories),
  6. Infrastructure (LLM APIs, GPU clusters, model endpoints).

• Agent Composition: There are six core LLM-based agents:

  • Security Verification Chat Agent
  • Security Asset Identification Agent
  • Threat Modeling & Test Plan Generation Agent
  • Security Vulnerability Detection Agent
  • Simulation-Based Security Bug Validation Agent
  • Security Property and Assertion Generation Agent

Each agent may invoke "sub-agents" for subtasks (e.g., module summarization, scenario generation), and overall coordination is handled by the Supervisor and Orchestrator layers to support multi-turn, iterative, and context-aware workflows.

2. Specialized Agent Functions

Each SV-LLM agent is tailored to a distinct security verification activity:

1. Security Verification Chat Agent: Provides up-to-date, retrieval-augmented responses to hardware security queries using curated vector databases and, if needed, external web sources. It features structured, multi-turn dialogue with hallucination mitigation.
2. Security Asset Identification Agent: Extracts and documents security-critical assets—such as registers, logic blocks, and signals—through prompt engineering and retrieval-augmented specification summarization. It operates even without RTL access, using modular summaries and multi-agent critique to prune false positives.
3. Threat Modeling & Test Plan Generation Agent: Maps assets to potential threats (hardware, supply-chain, or software), derives security policies, explores attack vectors, and generates verification test plans. RAG pipelines synthesize evidence from domain literature and internal design context.
4. Security Vulnerability Detection Agent: Performs domain-adapted, prompt-tuned RTL analysis to detect security bugs (e.g., privilege escalation risks, insecure state transitions). The agent relies on a fine-tuned variant of Mistral-7B-Instruct, trained on hardware vulnerability datasets for enhanced detection accuracy.
5. Simulation-Based Security Bug Validation Agent: Automatically synthesizes and validates testbenches that trigger security bugs in RTL modules, converting high-level scenarios into correct SystemVerilog testbenches, running simulation, and checking expected regions of interest.
6. Security Property and Assertion Generation Agent: Automatically generates formal security properties and assertions, such as SystemVerilog Assertions (SVA), grounded in mapped design structures, threat models, and relevant Common Weakness Enumeration (CWE) classes, including self-refinement for syntactic and context correctness.

3. Learning Paradigms and Optimization Strategies

SV-LLM employs a diverse mixture of LLM adaptation and grounding techniques, optimizing for each verification task:

• In-Context Learning: Used for extraction, reasoning, and generation tasks amenable to stepwise, demonstration-driven prompting (e.g., asset discovery, property synthesis).
• Fine-Tuning: Applied to the Security Vulnerability Detection Agent, which utilizes a parameter-efficient fine-tuned Mistral-7B-Instruct model trained on hardware-specific vuln datasets to internalize concepts such as state machines and privilege transitions, sharply improving bug discovery rates.
• Retrieval-Augmented Generation (RAG): Supports both Chat and Threat Modeling agents, providing grounded, hallucination-resistant evidence synthesis, and context-rich responses by querying design-specific and academic/industry vector stores.

Summary Table: Learning Paradigms by Agent

Agent	In-context	Fine-tuning	RAG
Security Verification Chat	✓	–	✓
Security Asset Identification	✓	–	✓
Threat Modeling & Test Plan Generation	✓	–	✓
Vulnerability Detection	–	✓	–
Simulation-Based Bug Validation	✓	–	–
Property & Assertion Generation	✓	–	–

4. Automation, Efficiency, and Quantitative Results

SV-LLM demonstrates substantial gains in automation and accuracy compared to traditional and ML-baseline security verification methods.

• Manual Intervention Reduction: SV-LLM automates translation of requirements to formal specifications, asset identification, threat modeling, property synthesis, and testbench generation, historically manual and error-prone tasks.
• Empirical Performance:
  • Vulnerability Detection Accuracy: Fine-tuned SV-LLM agent achieves 84.8% detection accuracy (vs. 42.5% for non-fine-tuned and 91.3% for GPT-4o).
  • Bug-Validated Testbench Rate: SV-LLM achieves up to 89% correct generation (vs. 18%-43% for prompt-only baselines).
  • Chat Agent: Outperforms ChatGPT-4o in accurate domain queries, eliminating hallucination through RAG.
• Efficiency Gains: The system eliminates per-design bottlenecks, enables parallel processing, and automates error correction via multi-turn refinement.

Summary Table: SV-LLM Verification Capabilities

Verification Task	SV-LLM Automation	Previous Approaches
Asset identification	✓	Partial, manual
Threat modeling & policy	✓	Manual, expert-only
Vulnerability/bug detection	✓	Static/dynamic, partial
Testbench generation	✓	ML/GA-based, functional
Property/assertion generation	✓	Manual, semi-auto

5. Transformation of SoC Security Verification Practices

The deployment of SV-LLM marks a paradigm shift in SoC hardware security, enabling integrated, scalable, and robust verification previously reliant on scarce expert resources:

• Pre-SV-LLM: Fragmented, highly manual workflows; slow iteration; significant error potential; bottlenecked by expert bandwidth and limited design coverage.
• With SV-LLM: All critical verification phases are automated, coordinated, and scalable. Modular extraction enables asset and policy identification without RTL, and assertion/testbench generation is systematized and validated. Iterative, agent-driven dialogue reduces context errors and supports proactive risk mitigation early in the design cycle.

Case studies demonstrate accurate identification of security assets, automatic assertion synthesis aligned to CWE classes, and full workflow bug validation for authentic open-source hardware modules—all without manual intervention or reliance on proprietary toolchains.

6. Technical Details, Data Formats, and Comparative Evaluation

SV-LLM outputs are delivered in tool-friendly formats, including structured JSON lists for asset maps and formal SystemVerilog Assertions. RAG workflows use embedding-based retrieval to maintain factual grounding. Modular RAG-based summarization enables full coverage of large specifications without exceeding LLM context limitations. Comparative analysis shows that SV-LLM outperforms both static (formal/concolic), dynamic (fuzz, pen-testing), and ML-based approaches in breadth, accuracy, automation, and tool compatibility.

• Example Accuracy Formula:

$\Delta A = A_{FT} - A_{NF}$

For vulnerability detection,

$$A_{FT} = 84.8\%; \; A_{NF} = 42.5\%; \; \Delta A = 42.3\%$$

• Testbench validation rate:

$$\text{Validation Rate} = \frac{\text{Number of successfully bug-triggering testbenches}}{\text{Total testbenches generated}}$$

• Example Security Property (SystemVerilog):

assert property (@(posedge clk) disable iff (!rst_n) (dbg_sel && dbg_en) |-> (csr_q.enable_dma == 1'b0 && csr_q.dma_prio == 3'h0) );

Conclusion

SV-LLM constitutes a comprehensive, agent-driven system for automated, scalable, and robust system-on-chip security verification, integrating the latest LLM advances with task-specific adaptation strategies. Its modular multi-agent design, empirical superiority in accuracy and automation, and demonstrated applicability to realistic SoC hardware establish it as a foundational technology for modern hardware security verification practices.