--- title: 'SolAgent: Multi-Agent Smart Contract Generator' url: https://www.emergentmind.com/topics/solagent type: topic --- # SolAgent: Multi-Agent Smart Contract Generator SolAgent is a specialized, tool-augmented multi-agent framework designed to generate secure and correct Solidity smart contracts, emulating the iterative code development workflow of human experts. By combining large language models (LLMs), program analysis tools, and file-system operations within a dual-loop architecture, SolAgent directly targets two persistent challenges in smart contract generation: functional correctness (passing all specified tests) and security (absence of vulnerabilities). Experiments on the SolEval+ benchmark demonstrate that SolAgent achieves superior performance in Pass@1 and vulnerability reduction compared to baseline LLMs, code assistants, and generic agent frameworks [2601.23009]. ## 1. Multi-Agent Architecture and Core Components SolAgent employs a division of labor across specialized agents: - **Coding Agent**: Receives natural language requirements $R$ and the project context, producing an initial Solidity source file $C_0$. - **Refining Agent**: Takes the latest code artifact $C_{t-1}$ and aggregated feedback $F_t$ (encompassing test and security results), then outputs a refined version $C_t$, correcting errors or mitigating detected vulnerabilities. A dual-loop refinement mechanism orchestrates the agent interactions: - **Inner Correctness Loop (via Forge)**: Utilizes the Foundry/Forge compiler and associated test harness to run comprehensive test suites, providing feedback such as test pass rates and specific assertion or stack trace failures ($F_\text{forge}$). - **Outer Security Loop (via Slither)**: Integrates the Slither static analyzer, which reports potential vulnerabilities with per-alert severity (Low/Medium/High), guiding security-related refactoring (e.g., enforcing checks-effects-interactions or adding access controls). File-system tools (`list_dir`, `read_file`) empower the framework to reason across repository structures, load dependencies, and manage context, enabling multi-file smart contract synthesis beyond isolated code snippets. ## 2. Algorithmic Workflow and Stopping Strategies The iterative workflow dynamically alternates between correctness and security refinement, employing both success and early-stopping mechanisms: - **Dynamic Stopping Conditions**: 1. **Success**: All tests pass ($\text{pass rate} = 1.0$), and no unresolved high or medium severity vulnerabilities remain. 2. **Stagnation**: No progress in either pass rate or vulnerability count for $N$ consecutive rounds (default $N=2$). 3. **Oscillation**: Feedback similarity between consecutive rounds exceeds threshold $\tau$ (e.g., $\tau=0.9$ using sequence matching ratio), indicating the agent is trapped in a repetitive cycle. - **Feedback Aggregation**: At each round, outputs from Forge (functionality correctness) and Slither (security inspection) are serialized and aggregated for downstream decision-making. - **Best Solution Selection**: Track the best-performing code artifact $C_{\text{best}}$ and associated score (weighted by pass rate and negative vulnerability count) across all refinement rounds. - **Outline Pseudocode**: ```python # Pseudocode summary Input: R (requirements), T (tests), MaxRounds, N (stagnation), τ (oscillation) Output: C_best C0 = CodingAgent.generate(R) C_best = C0; Score_best = -inf for t in 1..MaxRounds: pass_rate, failures = RunForge(C_{t-1}, T) vulnerabilities = RunSlither(C_{t-1}) Ft = aggregate(pass_rate, failures, vulnerabilities) if (pass_rate == 1.0 and no high/med vulns) or stagnation or oscillation: break C_t = RefiningAgent.refine(C_{t-1}, Ft) score_t = weight(pass_rate, -|vulns|) if score_t > Score_best: C_best = C_t; Score_best = score_t return C_best ``` ## 3. Metrics and Experimental Results SolAgent is evaluated using the SolEval+ benchmark, comprising 81 file-level Solidity tasks and 1,188 hand-written Forge tests assessing key correctness and security properties. - **Evaluation Metrics**: - **Pass@k**: $$ \text{Pass@}k = \mathbb{E} \left[1 - \frac{C(n-c,\,k)}{C(n,\,k)} \right] $$ where $n$ is the number of samples, $c$ the passing samples, and $C(\cdot,\cdot)$ denotes the binomial coefficient. - **Compile Rate**: $$ \text{Rate}_\text{compile} = \frac{1}{N} \sum_{i=1}^N \mathbb{1}[\text{compile}(s_i)] $$ - **Vulnerability Reduction**: $$ \Delta V\% = \frac{V_\text{base} - V_\text{SolAgent}}{V_\text{base}} \times 100 $$ Key experimental outcomes (Pass@1, Compile Rate): | Source / Model | CompileRate | Pass@1 | |---------------------------|-------------|----------| | Human Baseline Repo | 100.00% | 100.00% | | Vanilla LLM (Claude) | 39.51% | 25.59% | | GitHub Copilot | 32.10% | 10.02% | | DeepCode | 37.04% | 13.55% | | MetaGPT | 35.80% | 11.78% | | Qwen-Agent | 45.68% | 28.37% | | **SolAgent (Claude)** | **95.06%** | **64.39%** | SolAgent achieves a 127.1% relative Pass@1 improvement over the best vanilla LLM (Qwen-Agent: 28.37%). In total vulnerability reduction, SolAgent achieves 15.7% fewer static alerts compared to the human baseline (on 77 shared files: 293 vs. 247 alerts). Using a GPT-5-Mini base, a maximum reduction of 39.77% is observed (259 to 156 alerts). Statistical variance for Pass@1 per correctly compiled file is reported (e.g. SolAgent(Claude): $0.7795 \pm 0.2941$). ## 4. Knowledge Distillation Procedure SolAgent's high-quality trajectories are used as distillation data to train smaller open-source models employing instruction-following and demonstration learning paradigms: - **Trajectory Collection**: - *Full-Context*: Original requirements plus detailed comments, agent dialogue, and final artifact. - *Compressed-Context*: Summarized requirements and new agent-generated dialogues. - **Supervised Training Objective**: $$ L(\theta) = -\sum_i \log P_\theta(C^*_i~|~R,~C^*_{