SIADAFIX: Adaptive Program Repair
- SIADAFIX is an adaptive, LLM-based framework for repository-level bug repair that treats issue descriptions as orchestration signals.
- It combines fast thinking for rapid issue structuring and classification with a slow, tool-using bug-fix agent for multi-step repair and validation.
- The framework achieves state-of-the-art pass@1 rates on SWE-bench Lite by dynamically allocating repair effort across easy, middle, and hard tasks.
Searching arXiv for the SIADAFIX paper and closely related benchmark context. SIADAFIX is an adaptive, LLM-based automated program repair framework centered on “issue description response” for repository-level bug fixing. It combines “fast thinking” components that optimize, classify, and review natural-language issue descriptions with a “slow thinking” bug-fix agent that performs multi-step, tool-using repair. The framework is designed for real-world, repository-level bugs in which an agent must interpret a noisy issue report, navigate a codebase, generate a patch, and validate it against tests. Its central claim is that repair quality and efficiency can both improve when the issue description is treated as a first-class orchestration signal and when repair effort is allocated adaptively across easy, middle, and hard tasks rather than through a single fixed workflow (Cao et al., 17 Oct 2025).
1. Problem setting and design rationale
SIADAFIX addresses repository-level program repair, a setting in which the system must understand a target repository, infer intent from an often underspecified issue description, localize the defect, modify code, and confirm correctness through test execution. The motivating observation is that existing LLM-based agents for this setting can fail on a substantial fraction of real-world issues, frequently employ non-adaptive workflows, and do not systematically use the issue description as a structured control signal (Cao et al., 17 Oct 2025).
The framework is motivated by several limitations identified in the underlying study. One is “brittle deep thinking,” where slow, autonomous, multi-step agents can exhibit “inertial thinking”: once an early hypothesis is wrong, later steps inherit and reinforce that error. Another is the inefficiency of non-adaptive pipelines that apply either lightweight prompting or expensive multi-step search to every issue regardless of complexity. A third is poor use of issue descriptions. In the reported setting, issue reports in SWE-bench Lite are noisy and inconsistent, while benchmarks such as SWE-bench Verified exhibit higher performance partly because issue statements are manually cleaned or annotated. SIADAFIX is therefore framed as a mechanism to balance repair cost and repair accuracy by structuring issue descriptions, estimating difficulty, and routing each problem to a workflow commensurate with its apparent complexity (Cao et al., 17 Oct 2025).
The paper explicitly grounds this design in a dual-process distinction. “Fast thinking” corresponds to quick, low-cost decisions such as issue optimization, difficulty classification, patch checking, and patch selection. “Slow thinking” corresponds to deliberative, tool-using, multi-step reasoning in the bug-fix agent. In operational terms, fast components decide how much effort to invest and act as independent reviewers, while the slow component performs the heavy repair work.
2. Architectural organization
At a high level, SIADAFIX receives an issue description and a target codebase, then separates processing into a Workflow Decision Phase and a Workflow Execution Phase. The decision phase applies an Issue Description Optimizer and an Issue Description Difficulty Classifier to produce the “issue description response,” which consists of an optimized issue report plus a predicted difficulty label in . The execution phase then invokes a Bug Fix Agent, optionally followed by a Checker or Enhanced Checker, and in hard cases a Selector that chooses among multiple candidate patches (Cao et al., 17 Oct 2025).
The Issue Description Optimizer is an LLM prompt that rewrites the raw issue into a structured report containing Issue Overview, Detailed Problem Description, Reproduction Steps, Expected Behavior, and Acceptance Criteria. The optimizer is instructed to extract potential issues from the original wording, clarify test scenarios, provide complete reproduction steps, and define success criteria, while not omitting or altering information from the original description. This structured output serves as a common specification for subsequent components.
The Issue Description Difficulty Classifier is a RandomForestClassifier trained offline with scikit-learn. It uses textual features including char_count, word_count, sentence_count, code_blocks, error_mentions, tech_mentions, sentiment_score, line_count, avg_word_length, unique_word_ratio, code_pattern_count, question_count, urls, version_mentions, number_count, uppercase_ratio, punctuation_ratio, chars_per_word, and sentences_per_line. The paper further states that the classifier considers semantic complexity, technical difficulty, code impact scope, and historical repair data, although an explicit formula is not provided.
The Bug Fix Agent is a Claude-4 Sonnet–based, tool-using agent. Its tools are edit, regex_search_files, run_cmd, list_code_definition_names, and fix_attempt_completion. Within a repair trajectory, the agent plans and analyzes the issue, explores the repository under /testbed, searches for relevant code with regex_search_files, reads and interprets functions and call stacks, locates or creates tests to reproduce the bug, applies edits via edit, runs tests with run_cmd, iterates when failures remain, and finally reports completion with fix_attempt_completion.
The Checker is an independent LLM agent prompted with “ZERO TOLERANCE CODE REVIEW.” It consumes the issue description and a code patch; the enhanced variant also receives the bug-fix trajectory. Its output is JSON of the form:
1 2 3 4 5 6 7 |
{
"analysis": "...",
"result": {
"is_fixed": true,
"check_summary": "..."
}
} |
The checker performs conceptual review against root cause, symptom coverage, edge cases, regressions, design issues, performance considerations, and architecture alignment. The Selector is used in hard mode when multiple patch diffs are available. It evaluates candidates on relevance, completeness, safety, and code quality, and returns a selected_patch_index and selection reasoning.
3. Issue description response and adaptive routing
The term “issue description response” denotes the processed representation derived from the raw issue description. It consists of the optimizer’s structured bug report and the classifier’s difficulty label. In the SIADAFIX design, this response is not merely auxiliary context; it determines the subsequent workflow and thus functions as the orchestration mechanism for the system as a whole (Cao et al., 17 Oct 2025).
The optimizer’s role is to transform a noisy natural-language issue into a more specification-like object. The resulting report makes reproduction steps explicit, articulates expected behavior, identifies acceptance criteria, and may include root-cause hypotheses. This structured context is then consumed by the bug-fix agent to design tests and reason about patch validity more systematically. The same issue representation also informs the checker’s judgment about whether a patch aligns with the actual problem rather than merely suppressing observed failures.
Difficulty classification provides the routing signal for adaptive effort allocation. Easy issues are described as simple logical or API-usage errors with limited scope, clear descriptions, and lower complexity indicators. Middle issues may span multiple functions or modules and exhibit moderate ambiguity. Hard issues are characterized as complex systemic problems, vague descriptions, or heavy framework use, often with large impact scope, longer descriptions, numerous code blocks, and substantial technical terminology. The classifier therefore determines whether SIADAFIX should favor minimal test-time expenditure, iterative repair-and-check depth scaling, or multi-candidate width scaling.
A plausible implication is that the issue description response plays a role analogous to a task-conditioned controller: it converts an unstructured bug report into a compact decision object that specifies both the content of the task and the computational budget appropriate for that task. The paper’s framing supports this interpretation, although it does not name the mechanism in those terms.
4. Repair modes and test-time scaling
SIADAFIX defines three operational modes—Easy, Middle, and Hard—selected adaptively from the issue description response. These modes differ in how much external verification and sampling are performed around the bug-fix agent (Cao et al., 17 Oct 2025).
Easy mode is the shortest pipeline. The classifier labels the issue as Easy, the optimizer produces a structured report, and the Bug Fix Agent performs a single repair run. The paper characterizes this regime as “fast generalization for simple problems.” There is no checker or selector by default, and the main test-time expenditure lies inside the bug-fix trajectory itself.
Middle mode introduces depth scaling. The workflow is optimizer bug-fix attempt checker, followed by another bug-fix iteration if the checker returns is_fixed = false. This yields a repair–check loop that continues until success or budget exhaustion. The checker therefore acts as an independent corrective mechanism that can redirect the agent when an initial attempt matches tests imperfectly or fails to solve the root cause.
Hard mode introduces width scaling. After optimization, SIADAFIX launches multiple bug-fix runs, with environment resets between iterations to avoid path dependency and cumulative corruption from earlier edits. The Selector then compares multiple candidate diffs and chooses the best patch. The criteria used are relevance, completeness, safety, and code quality. The paper describes this as the strongest mode and explicitly connects it to test-time scaling at the level of full agent runs.
The reported discussion notes that the paper does not provide explicit pass@k derivations, but it does present the underlying intuition in terms familiar from test-time scaling. If each independently sampled candidate succeeds with probability , then the probability that at least one of candidates succeeds is:
Within SIADAFIX, hard mode approximates this multi-sample improvement through multiple full repair attempts, while the selector and test-based evaluation attempt to identify the correct candidate. This suggests that the framework interprets test-time scaling not merely as additional token generation but as controlled expansion of agent trajectories combined with verification.
5. Experimental evaluation
The principal experimental setting is SWE-bench Lite, described in the paper as a benchmark of 300 real issues from repositories such as Django, SymPy, scikit-learn, matplotlib, pytest, and sphinx. Each issue includes a problem description, reproduction steps, an expected result in the form of a ground-truth patch and tests, and a Docker-based harness for automatic validation. The paper treats SWE-bench Lite as especially suitable for SIADAFIX because its issue descriptions are not manually cleaned or annotated, making issue-description optimization materially relevant (Cao et al., 17 Oct 2025).
All reported open-source baselines use Claude-4 Sonnet. The baselines listed are SWE-agent, KGCompass, ExpeRepair-v1.0, and Refact.ai Agent. The primary metric is pass@1, defined here as the fraction of tasks for which the first returned patch passes benchmark tests.
| Method | Pass@1 |
|---|---|
| SIADAFIX | 182/300 = 60.7% |
| ExpeRepair-v1 | 181/300 = 60.3% |
| Refact.ai Agent | 180/300 = 60.0% |
| KGCompass | 175/300 = 58.3% |
| SWE-agent | 170/300 = 56.7% |
On this benchmark, SIADAFIX achieves 182/300 resolved problems, reported as 60.7% pass@1, which the paper describes as state-of-the-art among open-source methods on SWE-bench Lite when using Claude-4 Sonnet. The stated margins are +4.0 points over SWE-agent, +2.4 over KGCompass, +0.4 over ExpeRepair-v1, and +0.7 over Refact.ai Agent. Per-project highlights reported in the paper include sympy/sympy at 44/77 (57.1%), matplotlib/matplotlib at 14/23 (60.9%), and sphinx-doc/sphinx at 9/16 (56.3%), where SIADAFIX is described as best among the compared methods, while remaining comparable or slightly behind on some small projects such as requests and flask.
The protocol further states that the classifier was trained on internal LI AUTO data comprising 500k+ code generation requests and then applied to SWE-bench Lite issues based on similarity in description style. The implementation environment includes a /testbed directory and SIADA-CLI tools such as ripgrep-based search and tree-sitter-based analysis.
6. Component contributions, limitations, and practical significance
The paper reports an ablation sequence that isolates the contributions of the Bug Fix Agent (BFA), Checker (CH), Optimizer (OP), and Classifier (CL). These results are central to the framework’s claim that issue-description processing and adaptive routing each add measurable value (Cao et al., 17 Oct 2025).
| Configuration | Pass@1 |
|---|---|
| BFA | 142/300 = 47.3% |
| BFA + OP | 157/300 = 52.3% |
| BFA + CH | 159/300 = 53.0% |
| BFA + CH + OP | 173/300 = 57.7% |
| BFA + CH + OP + CL | 182/300 = 60.7% |
The interpretation given in the paper is additive and then synergistic. Relative to BFA alone, the Checker yields a gain of 5.7 points, indicating that independent verification reduces false-positive fixes. The Optimizer yields a gain of 5.0 points, supporting the view that structured issue descriptions improve downstream repair. Combining Optimizer and Checker produces a 10.4-point gain over baseline, and adding the Classifier contributes a further 3.0 points to reach the final 60.7%. Per-project ablations are reported to show similar incremental improvements across most projects, including Django, matplotlib, pytest, and scikit-learn.
The limitations discussed are equally specific. SIADAFIX remains dependent on issue quality: the optimizer can clarify and structure, but severely underspecified issues remain difficult. The difficulty classifier can misclassify issues because it is trained on internal LI AUTO data rather than directly on SWE-bench issues; this can lead to overkill on easy bugs or underpowered workflows on hard ones. The reported results depend on Claude-4 Sonnet, and performance under weaker models or tighter budgets is left unresolved. Evaluation is confined to Python-based open-source projects in SWE-bench Lite, and the agent assumes tools and environment conventions such as ripgrep, tree-sitter, and /testbed, so portability to other settings may require additional engineering.
The paper identifies three future directions: specialized architecture analysis tools for better handling of inter-module interactions, multi-modal information fusion across code, documentation, tests, and possibly runtime logs, and domain knowledge integration for specialized areas such as machine learning, graphics, and databases. Taken together, these directions indicate that SIADAFIX is presented not as a closed solution to automated program repair but as a concrete framework in which adaptive orchestration, issue-structured reasoning, and test-time scaling can be combined within an open-source implementation, SIADA-CLI.