Smart-LLaMA-DPO

• Smart-LLaMA-DPO is a state-of-the-art framework designed for highly accurate and explainable smart contract vulnerability detection, utilizing advanced preference-based alignment techniques.
• It employs a three-stage training process (CPT, SFT, DPO) and integrates vulnerability detection and explanation generation into a single unified model built on LLaMA-3.1.
• The framework achieves superior performance over baselines in detecting both standard and complex machine-unauditable vulnerabilities, while producing expert-quality, detailed explanations.

Smart-LLaMA-DPO is a state-of-the-art framework for explainable smart contract vulnerability detection, leveraging advanced preference-based alignment techniques to produce both high-accuracy vulnerability predictions and expert-level explanations. Built on LLaMA-3.1-8B, Smart-LLaMA-DPO introduces methodological innovations in dataset construction, domain-specific pre-training, joint detection-explanation learning, and Direct Preference Optimization (DPO), yielding superior results for both standard and complex vulnerability classes.

1. Model Architecture and Training Strategy

Smart-LLaMA-DPO follows a three-stage post-training process—Continual Pre-Training (CPT), Supervised Fine-Tuning (SFT), and Direct Preference Optimization (DPO)—to achieve robust detection and high-quality explanation generation in a unified, end-to-end LLM.

• CPT Stage: The model is adapted to the smart contract domain using large-scale Solidity data from over 186,000 unique Ethereum contracts (501M tokens), further augmented by 100,000 samples from broader code and technical domains. Deduplication (Jaccard Index) ensures token diversity and eliminates redundant logic. The CPT loss function is a masked LLMing objective:

$$L_{CPT} = -\mathbb{E}_{x \sim D}\left[\sum_{i=1}^{n} \log P(x_i | x_{<i}, c_i)\right]$$

• SFT Stage: The model is then supervised on a comprehensive dataset containing detection labels, detailed textual explanations, and precise vulnerability locations. SFT loss is formulated as an equal weighted sum:

$L_{SFT} = \frac{1}{2}(L_{detect} + L_{explain})$

with

$$L_{detect} = -\sum_{(x,y) \in D_{vul}} \log P(y|x;\theta)$$

$$L_{explain} = -\sum_{(x,e) \in D_{exp}} \sum_{i=1}^{|e|} \log P(e_i|x, e_{<i};\theta)$$

• Unified Detection and Explanation: Unlike two-stage or cascaded models, Smart-LLaMA-DPO performs vulnerability detection and explanation in one forward pass, reducing mode inconsistency and exploiting shared representations for both capabilities.
• DPO Stage: Direct Preference Optimization is used to align the model not only with correct detection but also with human-preferred explanation quality. The DPO objective compares pairs of outputs $(y_w, y_l)$ per input $x$, where $y_w$ is an expert-verified explanation and $y_l$ is an inferior counterpart, using:

$$\mathcal{L}_\text{DPO}(\pi_\theta; \pi_\text{ref}) = -\mathbb{E}_{(x,y_w,y_l)\sim \mathcal{D}}\left[\log \sigma\left(\beta \log\frac{\pi_\theta(y_w|x)}{\pi_\text{ref}(y_w|x)} - \beta \log\frac{\pi_\theta(y_l|x)}{\pi_\text{ref}(y_l|x)}\right)\right]$$

where $\beta$ is a scaling parameter and $\pi_\text{ref}$ is the SFT reference.

2. Dataset Construction and Preference Pair Design

Coverage, specificity, and annotation rigor are emphasized in Smart-LLaMA-DPO's datasets:

• Vulnerability Types: Four major Solidity/EVM vulnerabilities—Reentrancy (RE), Timestamp Dependence (TD), Integer Overflow/Underflow (IO), and Delegatecall (DE)—are annotated alongside seven "machine-unauditable" (MU) categories (e.g., price oracle abuse, privilege escalation, inconsistent state updates).
• Data Sources: Merged from peer-reviewed sources and expert-labeled datasets, including Etherscan contracts, GitHub, professional audits, blog sources, and prior academic datasets ([liu2023rethinking], [yu2023pscvfinder], [zhang2023demystifying]).
• Annotation Pipeline: Multi-tiered, with predefined vulnerability-type criteria, iterative LLM- and human-in-the-loop review, and explicit instruction to provide precise code locations and detailed explanatory rationales.
• Preference Pairs for DPO: For each relevant contract, human experts generate and validate a pair of explanations: a high-quality, thorough rationale (preferred) and a lower-quality, less complete rationale (non-preferred). This enables DPO to learn nuanced distinctions of expert explanatory style and substance, rather than just correctness.

3. Direct Preference Optimization for Explanation Quality

Direct Preference Optimization (DPO) in Smart-LLaMA-DPO enables fine-grained alignment beyond classical RLHF approaches:

• Pairwise Optimization: By comparing expert-preferred vs. suboptimal explanations for each vulnerability case, the model is directly optimized to prefer explanations that are correct, thorough, and clear.
• No Explicit Numerical Reward: Human feedback appears as ordinal ranking (preference pairs), allowing the policy to adapt to expert notions of quality and relevance, rather than optimizing a possibly misaligned scalar value.
• Advantages over RLHF/RLAIF: DPO is more stable, efficient, and directly reflects paired expert assessments, avoiding complications of training reward models or performing noisy on-policy rollouts.

4. Evaluation Methodology and Empirical Results

Smart-LLaMA-DPO's efficacy is established across multiple axes:

• Detection Accuracy and F1: Outperforms all baselines (including iAudit and FTSmartAudit) across all major categories:
  • Reentrancy F1: 88.50% (vs. previous best 82.32%)
  • Timestamp Dependency F1: 96.43%
  • Integer Overflow/Underflow F1: 88.29%
  • Delegatecall F1: 84.85%
  • Machine-Unauditable F1: 83.41% (25.98% above best prior)
• Average Gains: +10.43% F1 and +7.87% accuracy compared to prior state-of-the-art.
• Explanation Evaluation: Both LLM-based and human expert scoring (4-point Likert scale) confirm substantial improvements in correctness (81.15% +3/4), thoroughness (83.88%), and clarity (94.63%). The model reliably produces expert-level, logically founded, actionable rationales, with improved location fidelity and security specificity.
• Qualitative Analysis: Case studies reveal that Smart-LLaMA-DPO explanations are more nuanced, contextually aware, and free of over-simplification or missed vulnerabilities, particularly excelling on machine-unauditable cases that defeat classical detectors.

5. Impact on Blockchain Security and Theoretical Implications

• Explainable Auditing: Smart-LLaMA-DPO's unified detection-explanation approach is aligned with the requirements of security professionals, facilitating both automated triage and actionable human review.
• Coverage of Machine-Unauditable Vulnerabilities: The architectural and data innovations allow detection and rationalization of bug types that static or traditional ML tools fail to address, closing key gaps in real-world audit pipelines.
• Preference Alignment and Robustness: Human-in-the-loop DPO elevates not only outcome accuracy but also the communicability and trustworthiness of explanations, increasing the model's value in regulated or high-assurance environments.
• Generalization and Extensibility: The approach is applicable to additional programming languages and bug types as new data and annotation standards are developed.

6. Limitations and Future Work

• Fine Annotation Cost: High-quality preference pair generation and expert review are resource intensive.
• Extending Coverage: Incorporating further vulnerability types or languages will require continued curation and adaptation of the data pipeline.
• Model Post-Processing: Ongoing research may address rare output redundancy or subtle explanation artifacts.
• Automation in Annotation: Improving the scalability of expert-level annotation via semi-automated techniques or hierarchical LLM expert pipelines remains an open research area.

7. Resources and Reproducibility

All code, data, and model weights for Smart-LLaMA-DPO are publicly available:

• Code & Data Repository
• Model Weights (Zenodo)

---

Smart-LLaMA-DPO establishes a new benchmark for explainable, high-accuracy smart contract auditing with LLMs, advancing both the technical rigor of preference-based alignment and the practical utility of automated blockchain security tools.