---
title: 'Sorald: Rule-based Automated Repair for Java'
url: https://www.emergentmind.com/topics/sorald
type: topic
---

# Sorald: Rule-based Automated Repair for Java

Sorald is a rule-based automated program repair system for Java that targets selected SonarQube/SonarJava static-analysis violations and generates patches intended to make those violations disappear by rewriting program structure rather than merely reporting warnings. Its central design premise is that, for a subset of static-analysis rules, the path from a reported violation to an acceptable repair can be encoded as deterministic AST-level transformations, shifting developer effort from manual interpretation and editing to inspection and approval of generated fixes [2103.12033]. A later case study evaluates Sorald under a broader, multi-dimensional framework and treats it as a proof of concept for assessing not only violation removal but also side effects on semantics, newly introduced violations, and structural quality [2508.15135].

## 1. Definition, motivation, and repair model

Sorald was introduced to address two recurrent limitations of static analysis. First, developers are flooded with warnings, including false positives, contextually low-impact findings, and issues not worth the remediation effort. Second, warnings are often not actionable: messages may be difficult to interpret, documentation may be vague or underspecified, and the path from “there is a problem” to “this is the concrete patch” is nontrivial [2103.12033]. In the original formulation, Sorald therefore targets SonarJava rules labeled as potential bugs and aims to reduce the burden on developers from “interpret and fix” to “inspect and approve.”

The repair objective is explicitly constrained. A fix for a violation \(v\) is defined as a patch such that SonarJava no longer reports \(v\), and the patched code still implements the behavior expected by the developer [2103.12033]. This definition is narrower than general-purpose APR: Sorald does not attempt to synthesize arbitrary behavioral repairs, and it does not claim that disappearance of a static-analysis warning is sufficient evidence of correctness.

The tool is intentionally selective. In the 2021 study, Sorald focuses on rules where the violating pattern and intended fix are clear enough that a deterministic AST transformation makes sense, the rule is labeled as a bug rather than merely a style issue or code smell, and occurrences are frequent in real repositories [2103.12033]. This suggests a repair philosophy centered on operational regularity and predictable patch generation rather than search-based or probabilistic synthesis.

## 2. Architecture and transformation strategy

Sorald’s architecture has three main steps: violation mining, violation fixing, and high-fidelity pretty-printing [2103.12033]. In the first step, Sorald invokes SonarJava, or another rule-based analyzer, to detect violations and obtain their exact source positions. In the second step, it parses Java sources with Spoon, constructs an AST composed of typed `Ct*` nodes such as `CtInvocation`, `CtIf`, and `CtMethod`, matches reported violations to AST elements, and applies rule-specific repair processors to the violations it knows how to handle. In the third step, the modified AST is converted back to Java source code while preserving whitespace, comments, parenthesization, and formatting as much as possible.

A key design choice is that Sorald treats SonarJava as a violation oracle. Detection is delegated to the analyzer’s official implementation, and Sorald consumes the reported rule identifiers and source positions to locate corresponding AST nodes [2103.12033]. This avoids reimplementing rule detection logic and allows the repair engine to inherit the correctness and maturity of a widely used industrial analyzer. The same architectural pattern could, in principle, be reused with analyzers such as SpotBugs or PMD if they provide accurate source locations.

Repair logic is encoded as metaprogramming templates over the AST. Each supported rule has a processor that verifies the relevant AST context, constructs the replacement code, and inserts, replaces, or deletes nodes accordingly [2103.12033]. The process is deterministic: for a given violation, Sorald generates exactly one patch, and the same violation yields the same patch. This determinism supports predictability, repeatability, and systematic evaluation.

Not every detected violation is considered repairable. Sorald introduces an Assumption Checker that encodes rule-specific repair preconditions and filters raw violations accordingly [2103.12033]. If a violation does not satisfy the encoded assumptions, it is excluded from repair and counted as a non-target violation. The distinction is formalized through the target set \(T_r \subseteq V_r\), where \(V_r\) is the set of violations detected for rule \(r\), and by the ratio

\[
\mathit{TDR}_r = \frac{|T_r|}{|V_r|}.
\]

Repair effectiveness is then measured through the Fixed Target Ratio and Fixed Detected Ratio:

\[
\mathit{FTR}_r = \frac{|V_r^\text{before}| - |V_r^\text{after}|}{|T_r|}
\]

\[
\mathit{FDR}_r = \frac{|V_r^\text{before}| - |V_r^\text{after}|}{|V_r^\text{before}|}.
\]

These metrics separate “how much of the rule is in scope” from “how much of the in-scope subset is actually fixed” [2103.12033].

High-fidelity pretty-printing is a nontrivial component of the system. Sorald adopts the Source Fragment Tree idea of storing documentary structure separately from AST nodes, reusing original fragments for unchanged nodes and pretty-printing only transformed regions [2103.12033]. The intended consequence is smaller diffs and easier code review, which is particularly important when repairs are proposed through pull requests.

## 3. Supported rules and repair templates

The original paper reports support for 10 SonarJava rules, all labeled as “BUG” [2103.12033]. The authors also manually analyzed all 631 SonarJava rules and classified the 153 bug rules into 77 fully fixable, 20 partially fixable, and 56 unfixable; using three implementation criteria—rule type “bug,” fixability under templates, and frequency in a large dataset—they obtained a candidate set of 36 rules, of which 10 were implemented first. These 10 rules cover 35% of all violations of the 97 fixable or partially-fixable bug rules in the 161-project dataset [2103.12033].

| Rule | Violation pattern | Patch shape |
|---|---|---|
| S1217 | `Thread.run()` called directly | Replace `.run()` with `.start()` |
| S1860 | Synchronization on `String` or boxed primitive | Introduce dedicated `Object` lock and synchronize on it |
| S2095 | Resource not properly closed | Wrap relevant block in `try-with-resources` |
| S2111 | `new BigDecimal(doubleValue)` | Use `BigDecimal.valueOf(doubleValue)` or convert first argument to `String` |
| S2116 | `array.toString()` or `array.hashCode()` | Replace with `Arrays.toString(array)` or `Arrays.hashCode(array)` |
| S2142 | Ignored `InterruptedException` | Insert `Thread.currentThread().interrupt();` |
| S2184 | Arithmetic overflow risk before widening assignment | Cast the leftmost operand to the target type |
| S2225 | `toString()` returns `null` | Replace `return null;` with `return "";` |
| S2272 | `Iterator.next()` lacks `NoSuchElementException` guard | Prepend `if (!hasNext()) { throw new NoSuchElementException(); }` |
| S4973 | `==` or `!=` on `String` or boxed types | Replace with `.equals()` or `!.equals()` |

These templates are deliberately local and syntactic, although some are context-sensitive. For example, the S2184 repair uses Spoon type information to detect operand types and the target variable type; the S1860 repair may add a new field and a getter method depending on how the lock is obtained; and the S2095 repair manipulates enclosing block and `try` structures rather than only replacing a single expression [2103.12033].

A representative limitation appears in rule S2225. SonarJava flags both `toString()` and `clone()` methods that return `null`, but Sorald only repairs `toString()` by returning `""`, because the expected return value for `clone()` is application-specific and not statically obvious [2103.12033]. This exemplifies the distinction between a detectable violation and a violation for which a deterministic, behaviorally plausible repair template can be defined.

A later case study evaluates Sorald on 30 supported SonarQube Java rules in that study: 18 bug rules, 11 code smell rules, and 1 vulnerability rule [2508.15135]. The listed rules include the original bug-oriented repairs as well as additional transformations such as adding a private constructor to utility classes, removing unused fields or assignments, rewriting equality checks to put string literals on the left, and injecting `serialVersionUID` with the constant value `1L`. This broader support set matters because the later evaluation focuses precisely on the side effects of such local rule-specific transformations.

## 4. Empirical performance on GitHub repositories

The original evaluation uses the “TopRepos” dataset of 161 GitHub Java projects selected by the following criteria: at least 50 stars, Maven-based with a root `pom.xml`, active with at least one commit in the last 3 months as of November 2020, PR-friendly with at least one accepted pull request in the last 3 months, healthy in the sense that both `mvn compile` and `mvn test` pass on the latest commit with Java 11, and using CI via `.travis.yml` [2103.12033]. Across these repositories, the reported total is 4,110k LOC, with a median of 9.73k LOC per repository and a sum of 14,842 SonarJava bug-rule violations.

For the 10 supported rules, the aggregate pre-repair count is 1,759 violations, of which 1,307 satisfy Sorald’s repair preconditions and are therefore target violations [2103.12033]. Sorald fixes 852 of these target violations. This yields an overall TDR of 74%, an FTR of approximately 65% \((852 / 1{,}307)\), and an FDR of approximately 48% \((852 / 1{,}759)\). The paper characterizes these results as evidence that notable SonarJava bug-rule violations can be fixed automatically at scale.

Performance varies by rule. S2142, concerning ignored `InterruptedException`, reaches TDR 99% \((315/316)\), FTR 95% \((300/315)\), and FDR 94% \((300/316)\). S2184, concerning casting of math operands before assignment, reaches TDR 97% \((431/440)\), FTR 85% \((368/431)\), and FDR 83% \((368/440)\). S2272 and S4973 both have TDR values around 97–98% and FTR/FDR values around 82–85%. By contrast, S2095, concerning resource closure, has TDR 46% \((361/782)\), FTR 9% \((34/361)\), and FDR 4% \((34/782)\), making it the hardest rule in the study to repair automatically [2103.12033].

The paper also estimates a total remediation-time saving of 7,340 minutes, approximately 122 hours, across the 852 fixed violations, using SonarJava’s per-violation remediation estimates [2103.12033]. Median time per rule per project ranges between 4.4 and 6.3 seconds, and the overall repair time is described as typically dwarfed by test execution and human review.

For safety assessment, the authors apply Sorald patches and run `mvn compile` and `mvn test` using Java 11, then manually inspect failures [2103.12033]. Only 7 out of 1,610 patch applications cause a test suite to start failing. Reported causes include intentionally used violations in tests, limitations in the S2095 resource-closure logic, and tests that encode buggy behavior rather than the desired behavior. The paper explicitly notes that tests do not fully guarantee behavioral correctness, but it treats the 0.4% breakage rate as low enough for Sorald to be considered reasonably safe in practice.

## 5. Continuous integration workflow, developer reception, and relation to APR research

Sorald is available as a Java command-line tool that takes a Java source directory and one or more target SonarJava rules, invokes SonarJava to detect violations and positions, parses the code with Spoon, applies rule-specific templates to AST nodes that meet repair preconditions, and writes the repaired source back to disk [2103.12033]. The intended post-processing sequence is to compile, test, inspect diffs, and decide whether to keep or adjust the patches. Because the pretty-printer is designed to minimize formatting changes, the resulting patches are meant to be reviewable in ordinary Git workflows.

SoraldBot extends this workflow to GitHub. It monitors commits in configured repositories; for each new commit, it runs Sorald on changed files, and for each rule with violations introduced by that commit, it generates a patch fixing those violations and submits a pull request if any fixes were made [2103.12033]. In a retrospective analysis over 350 days, from December 12, 2020 to November 27, 2021, 126 of the 161 projects had at least one commit, totaling 6,888 commits. Among them, 21 projects, or 16%, introduced new target violations in changed files, and 46 commits introduced violations of the 10 supported rules. SoraldBot generated fixes for four rules: S2095, S2111, S2142, and S2184, producing 80 fixed violations, 54 patches, and touching 21 projects. Most patches were small: 41 fixed exactly one violation, and 13 fixed 2–5 violations [2103.12033].

To study acceptability, the authors manually submitted 29 pull requests to 21 projects, each focused on one rule and aggregating patches over multiple commits where possible [2103.12033]. As of December 2021, 17 PRs had been accepted, 10 declined, and 2 remained pending. The accepted PRs were often small, focused, and associated with severe or clear-cut bug rules such as S2142, S2184, S2111, and S2095. Declined PRs exposed several limitations: some S2184 findings were viewed as false positives because overflow was impossible in context; some S2095 changes conflicted with preferred resource-management designs; and some maintainers were concerned that automated patches could be merged too readily if tests passed, even when a different repair idiom would be preferable.

The authors extract four operational lessons: simple is better, severity matters, context matters, and non-functional requirements matter [2103.12033]. These observations place Sorald in a broader APR and automated maintenance landscape. The paper contrasts Sorald with data-driven systems such as Getafix, Phoenix, AVATAR, and TFix, and with template-based systems such as SpongeBugs. On a shared dataset of 12 projects and the intersection rules S4973 and S2111, Sorald fixes 94% of violations versus 54% for SpongeBugs, namely 106 of 113 compared with 62 of 113 [2103.12033]. The stated distinction is that SpongeBugs re-implements rule detection logic, whereas Sorald reuses SonarJava’s official detection, which the authors argue improves reliability and extensibility.

## 6. Comprehensive reevaluation, limitations, and controversy

A later study argues that APR evaluation should not stop at counting cleared violations and uses Sorald as a case study for a four-dimensional framework: fixing capability, introduction of new violations, functional correctness, and structural quality [2508.15135]. The benchmark is built from Stack Overflow Java answer snippets. Starting from 8,010 `.java` files, the authors filter to 7,828 compilable files under JDK 17 and then retain 2,393 files containing at least one violation among 30 Sorald-fixable SonarQube rules. These 2,393 files contain 3,529 SonarQube violations across the 30 rules.

On the targeted-violation dimension, Sorald performs strongly. The study reports that 3,423 of the 3,529 violations are removed, corresponding to an overall fix rate of approximately 97% [2508.15135]. Selected rule-level results are correspondingly high: S1118 has 1,684 violations and 99.9% fixed; S1068 has 509 violations and 98.2% fixed; S1481, S1132, S1444, S2184, and S2142 each reach 100%; and S2095 reaches 95.7%. Only S2164 and S1948 show comparatively low fix rates, at 36.3% and 45.3%, respectively.

The same study, however, reports substantial side effects. Using a wide 673-rule SonarQube profile pre- and post-repair, together with a code-fragment matching algorithm and manual validation, it identifies 2,120 newly introduced violations [2508.15135]. The abstract summarizes these as 32 bugs and 2,088 code smells, with no new vulnerabilities detected in the sample. The precision of the “new violation” detection algorithm is evaluated on a stratified sample of 326 cases; 250 are judged true positives and 76 false positives, and a binomial test yields \(p = 0.0043\), which the authors interpret as statistically significant evidence that the true precision exceeds 70%.

The functional-correctness assessment uses EvoSuite-generated tests. For the original 2,393 files, the study generates 8,274 tests over 2,422 classes, with average 89% coverage; after removing tests that fail on the original code, 8,212 passing tests remain as the baseline [2508.15135]. When run on the Sorald-repaired code, 1,962 of these tests fail, corresponding to a 23.9% failure rate, reported as 24%. The dominant failure mode is `IllegalAccessError` with 1,694 failures, caused by private constructors added to classes; other failures include 189 `NoClassDefFoundError` cases associated with 61 non-compilable files, and 78 `AssertionError` cases due to changed behavior such as removed exceptions, eliminated method bodies, altered arithmetic results, or modified string-comparison logic.

Structural-quality analysis employs CK metrics aggregated per file and Wilcoxon signed-rank tests on pre/post values [2508.15135]. The study reports significant changes for LCOM1, WMC, CBO, RFC, and LOC, all with \(p = 0.000\); DIT is not significant with \(p = 0.3173\), NPA is borderline with \(p = 0.0633\), and NOC is undefined. The direction of change is reported as increased LCOM1, WMC, RFC, and LOC, with slightly decreased CBO. The authors interpret this as degradation of structural quality: code becomes longer, less cohesive, and more complex, even if direct coupling decreases slightly.

The tension between the 2021 and 2025 results is not a contradiction so much as a change in evaluation lens. The earlier study emphasizes target-rule repairability, low regression counts in Maven-tested GitHub repositories, and developer acceptance for small focused PRs [2103.12033]. The later case study emphasizes that high violation-clearance rates can coexist with newly introduced faults, semantic instability, compilation failures, and structural degradation [2508.15135]. A plausible implication is that Sorald is best understood as a precise rule-centric repair engine whose utility depends strongly on the evaluation criterion, the supported rules, the dataset, and the degree of post-repair review and testing.

Source: https://www.emergentmind.com/topics/sorald