Detecting Buggy Contracts via Smart Testing (2409.04597v1)

Abstract: Smart contracts are susceptible to critical vulnerabilities. Hybrid dynamic analyses, such as concolic execution assisted fuzzing and foundation model assisted fuzzing, have emerged as highly effective testing techniques for smart contract bug detection recently. This hybrid approach has shown initial promise in real-world benchmarks, but it still suffers from low scalability to find deep bugs buried in complex code patterns. We observe that performance bottlenecks of existing dynamic analyses and model hallucination are two main factors limiting the scalability of this hybrid approach in finding deep bugs. To overcome the challenges, we design an interactive, self-deciding foundation model based system, called SmartSys, to support hybrid smart contract dynamic analyses. The key idea is to teach foundation models about performance bottlenecks of different dynamic analysis techniques, making it possible to forecast the right technique and generates effective fuzz targets that can reach deep, hidden bugs. To prune hallucinated, incorrect fuzz targets, SmartSys feeds foundation models with feedback from dynamic analysis during compilation and at runtime. The interesting results of SmartSys include: i) discovering a smart contract protocol vulnerability that has escaped eleven tools and survived multiple audits for over a year; ii) improving coverage by up to 14.3\% on real-world benchmarks compared to the baselines.

• The paper introduces SmartSys, a hybrid analysis system integrating concolic execution and fuzzing to detect previously undetected smart contract vulnerabilities.
• It employs a forecasting model to dynamically choose the optimal testing method, boosting benchmark coverage by up to 14.3%.
• SmartSys reduces computational cost and enhances detection capabilities, addressing long-standing security issues in blockchain technology.

Detecting Buggy Contracts via Smart Testing

Abstract

The paper investigates the vulnerabilities inherent in smart contracts and presents a hybrid dynamic analysis system, SmartSys, designed to detect these vulnerabilities with enhanced efficiency. The system integrates the advantages of concolic execution and foundation model-assisted fuzzing to overcome the existing limitations in scalability and deep bug detection. SmartSys dynamically decides the optimal analysis technique based on performance forecasts. This approach was shown to discover previously undetected vulnerabilities and improve coverage on standard benchmarks.

Introduction

Smart contracts, pivotal for blockchain-based transactions, are plagued by vulnerabilities leading to considerable financial losses. Dynamic analysis techniques such as fuzzing and concolic execution have been developed to detect bugs in these contracts. However, these techniques struggle with complex code patterns, limiting their scalability and ability to uncover deeply embedded bugs. Concolic execution, while promising, requires significant computational resources and cannot universally solve path constraints.

SmartSys revolutionizes this process by employing a self-deciding system based on foundation models, focusing on when to utilize concolic execution versus fuzzing. The system augments foundation models' capabilities with feedback from dynamic analysis to quickly identify and correct potential vulnerabilities. SmartSys's design allows it to surpass existing tools by detecting a smart contract protocol vulnerability persisting undetected by previous tools and efforts for over a year. It also improved benchmark coverage by up to 14.3%.

Figure 1: SmartSys's workflow illustrating the interaction between different analysis models to optimize vulnerability detection.

Methodology

SmartSys employs an interactive model focusing on hybrid fuzzing strategies to detect deep-seated bugs in smart contracts. It incorporates an intelligent forecasting model that evaluates the efficacy of concolic execution against fuzzing methods. Depending on the forecast, the generator model enhances fuzz targets to penetrate code constraints previously inaccessible.

The methodology involves:

• Forecasting: A fine-tuned model predicts whether to leverage concolic execution or fuzzing to surmount detected coverage plateaus.
• Generator Model: It produces context-aware, effective fuzz targets with minimized hallucinations, utilizing feedback loops from compilers and dynamic analyses.
• Dynamic Analysis Optimization: Tailored enhancements such as Directed Automated Random Testing (DART) style concolic execution for smart contracts ensure unique blockchain-specific optimizations.

Evaluation

The effectiveness of SmartSys was evaluated on multiple real-world benchmarks. SmartSys demonstrated superior performance in detecting vulnerabilities compared to other tools by achieving higher coverage and detecting critical bugs that had previously avoided discovery.

Code Coverage

SmartSys exhibited notably higher code coverage across tested benchmarks, outperforming existing tools such as Optik, Manticore, and Echidna. The integration of interactive models further delayed the performance plateau, optimizing both speed and coverage.

Figure 2: Code coverage comparisons of SmartSys with similar tools on the three benchmarks of D1.

Component Effectiveness

The effect of individual SmartSys components was investigated. The hallucination suppression mechanism and interactive model system, combined with selective invocations of fuzzing techniques, significantly improved coverage and reduced execution times.

Figure 3: The effect of the hallucination suppression algorithm and interactive forecast and generator models on the three benchmarks of D1: The top row shows the effect of the hallucination suppression algorithm; the bottom row shows the effect of interactive models.

Bug Detection

SmartSys proficiently detected bugs in new smart contracts, showcasing an ability to identify vulnerabilities in recent audit reports effectively and address common deep bug challenges that traditional tools did not resolve.

Figure 4: Bug detection results on Immunefi's top five zero-day vulnerabilities.

Conclusion

The research highlights SmartSys as a robust approach to enhancing the detection of vulnerabilities in smart contracts. Combining a finely-tuned forecasting model with a conscientious application of concolic execution and fuzzing increases SmartSys's scope and effectiveness, allowing it to detect vulnerabilities previously overlooked by other tools. The implications of this research suggest that similar interactive model systems could significantly advance the efficiency of security measures in other complex software environments beyond blockchain technology.