---
title: Automated Code Revision (ACR)
url: https://www.emergentmind.com/topics/automated-code-revision-acr
type: topic
---

# Automated Code Revision (ACR)

Searching arXiv for the cited ACR papers to ground the article and citations.
Search query: 2602.14595 Automated Code Revision consistency semantics-preserving perturbations
Automated Code Revision (ACR) denotes the automation of producing revised source code during or around code review, either by predicting revisions likely to be requested before review, generating revised code from reviewer comments after review, or operationalizing recurring historical change patterns as quick fixes in development environments [2101.02518] [2303.07221] [2108.11199]. In the reviewer-feedback-driven setting, an ACR system takes code and associated natural-language feedback and generates revised code intended to implement the requested change; one formalization uses a method \(c_i\), a review comment \(r_i\), a generated revision \(\tilde{c}_i\), and a human-written target \(\hat{c}_i\) [2602.14595]. Recent work treats ACR not only as sequence-to-sequence generation, but also as a problem of robustness under harmless code variation, partial-progress assistance, retrieval of historical review knowledge, calibrated confidence, and workflow-aware evaluation [2602.14595] [2604.06723] [2604.24525].

## 1. Scope and task structure

The most direct ACR formulations separate **Code Revision Before Review (CRB)** from **Code Revision After Review (CRA)**. CRB automates author-side revision before submission, while CRA automates revision after reviewer feedback. In the notation used for generation-based code review automation, these are
\[
f(Code)\xrightarrow[]{} Revised \, Code
\]
and
\[
f(Code, Comment)\xrightarrow[]{} Revised \, Code,
\]
respectively [2303.07221].

A broader survey of automated code review identifies **Revised Code Generation** as the core ACR category and further divides it into two tasks: **Implementing the Code Change Requested by a Reviewer** and **Predicting the Code Output of the Review Process** [2503.09510]. This taxonomy is useful because it distinguishes comment-conditioned refinement from proactive revision prediction. The same survey also places adjacent tasks—such as generating review comments, predicting problematic code elements, mining code improvement patterns, and reviewing via static analysis—in neighboring categories rather than conflating them with code revision itself [2503.09510].

| Task mode | Formalization | Representative role |
|---|---|---|
| CRB | \(f(Code)\to Revised\,Code\) | Pre-review revision prediction |
| CRA | \(f(Code, Comment)\to Revised\,Code\) | Comment-conditioned refinement |
| Pattern-based IDE revision | Historical before/after change pattern \(\to\) quick fix | Interactive editor assistance |
| Static-analysis-guided revision | File + issue metadata \(\to\) revised file | Repository-level quality improvement |

ACR is therefore narrower than automated code review in general. Review-comment generation, review-outcome prediction, and reviewer recommendation all support the review process, but they do not themselves synthesize revised code. Conversely, ACR is broader than classical automated program repair because its targets include readability, style, refactoring, API modernization, design consistency, and reviewer-requested changes that are not reducible to failing tests [2101.02518] [2503.09510].

## 2. Data representations and supervision signals

A recurring design choice in ACR is the extraction of aligned supervision from historical review traces. One early formulation mines review rounds from Gerrit and GitHub, extracts method-level triplets \(\langle m_s, r_{nl}\rangle \rightarrow m_r\), and uses those triplets to learn both pre-review revision suggestion and comment-conditioned revision generation [2101.02518]. In that setting, code is abstracted with `src2abs`, comments are normalized with the same abstraction map, and singleton review comments are linked to the method and to the affected span via `<START>` and `<END>` markers in the comment-conditioned dataset [2101.02518].

Generation-based ACR studies later unified three downstream tasks—CRB, CRA, and review comment generation—under a single seq2seq view, while keeping CRA distinct because it jointly conditions on code and natural-language review feedback [2303.07221]. Their benchmark construction emphasizes that realistic code-revision data often contains new target tokens absent from the input, which explains why abstraction-only pipelines and open-vocabulary tokenization behave differently across datasets [2303.07221].

A more recent robustness-focused formulation defines the evaluation set as
\[
\mathcal{D} = \{(c_i, r_i, \hat{c}_i)\}_{i=1}^{N}, \quad N=2032,
\]
where each triplet is a real Java code-review instance from the method-level dataset of Tufano et al. [2602.14595]. That work is important because it makes explicit that reviewer-feedback-driven revision is harder than plain code completion or unconstrained patch generation: the system must interpret the comment, localize the referenced code fragment, and synthesize the requested fix within the existing program structure [2602.14595].

Outside purely neural supervision, ACR has also been grounded in mined change patterns. Revizor mines recurring graph-based Python change patterns from repository history, preprocesses them into plugin resources, and applies them online as PyCharm quick fixes through subgraph isomorphism over fine-grained program dependence graphs and replay of GumTree edit actions on PSI [2108.11199]. This suggests a second data regime for ACR: not paired “input-output” examples for neural sequence generation, but reusable, interpretable before/after change patterns mined from version history.

## 3. Principal methodological families

The dominant direct ACR family is Transformer-based seq2seq generation. Comparative work evaluates **Trans-Review**, **AutoTransform**, **T5-Review**, **CodeBERT**, **GraphCodeBERT**, and **CodeT5** on CRB and CRA, and reports that CodeT5 is the strongest code revision generator tested for exact accepted revisions, while also showing that rankings can change when partial-progress metrics are used instead of exact match alone [2303.07221]. The architectural distinction emphasized there is between pretrained encoder-decoder models and encoder-only models paired with a randomly initialized decoder; the former perform better for generation-based code review tasks [2303.07221].

Another neural direction reframes ACR as a staged, multi-agent review-and-revision workflow. CodeAgent receives a submitted commit, commit message, and original files; decomposes work into Basic Info Sync, Code Review, Code Alignment, and Document phases; and uses a supervisory QA-Checker to refine inter-agent communication up to a maximum of 10 conversation rounds [2402.02172]. In its ACR role, the relevant output is revised code or revision suggestions produced in the Code Alignment phase, not merely review text [2402.02172].

Pattern-based ACR follows a different logic. Revizor uses graph-based localization rather than generation from scratch: it mines recurrent change patterns, matches the “before” graph into a developer’s current function by subgraph isomorphism, and applies a stored edit sequence over PSI if the developer accepts the quick fix [2108.11199]. Because its representation is an fgPDG rather than raw text or isolated AST fragments, it is designed to support distributed edits connected by data or control dependencies, such as replacing `range(len(...))` idioms with `enumerate`-style rewrites in Python [2108.11199].

Repository-level LLM pipelines introduce yet another methodology. One system starts from SonarQube findings, extracts structured issue metadata such as file, line, message, and type, then prompts GPT-3.5 Turbo first and GPT-4o on unresolved cases, optionally with retrieval-augmented generation (RAG), and uses a custom “Code Comparison App” plus rescanning and manual verification to filter hallucinated or unnecessary edits before final application [2506.10330]. Although this is not reviewer-comment-conditioned CRA, it is still ACR in the sense that the model revises existing code in response to explicit issue specifications.

Retrieval-augmented review generation has also been argued to transfer naturally to ACR. RARe studies automated review-comment generation rather than code editing, but its retrieve-then-generate architecture—retrieve the most relevant historical review evidence, then condition the generator on that evidence—has been presented as directly reusable for revision systems if the retrieval corpus is replaced with review-comment-to-fix pairs, before/after diffs, or accepted historical patches [2511.05302]. This suggests that retrieval can serve as an external memory for edit intent as well as review phrasing.

## 4. Evaluation metrics and what they measure

ACR evaluation has historically centered on exact reproduction of the human-written revised code. Early Transformer-based work evaluates success by counting **perfect predictions** under beam search, together with BLEU-4 and normalized token-level Levenshtein distance [2101.02518]. On the contributor side, exact reproduction reaches 15.76% with beam size 10; on the reviewer-comment-conditioned side, it reaches 30.72% with beam size 10, indicating that explicit reviewer intent materially improves revision generation [2101.02518].

Later comparative work argues that **Exact Match (EM)** is too coarse because it gives the same score to a prediction that fixes most of the problem and to one that is entirely wrong. It therefore introduces **Edit Progress (EP)**, defined as the relative reduction in token-level edit distance from the submitted code to the accepted revised code:
\[
Progress = \frac{|D_{\widetilde{C} \xrightarrow[]{} \bar{C}}| - |D_{\hat{C} \xrightarrow[]{} \bar{C}}|}{|D_{\widetilde{C} \xrightarrow[]{} \bar{C}}|}.
\]
EP can be negative if the generated revision is farther from the accepted code than the original submitted code was [2303.07221]. This metric is especially important for ACR-as-assistance, because it measures drafting value even when the system does not hit the exact accepted revision.

Robustness-oriented evaluation extends the metric set further. In the consistency study, correctness and consistency are measured with **Exact Match (EXM)**, **CodeBLEU**, **Edit Match (EM)**, and **Relative Edit Error (REE)**. REE measures excess editing when the necessary edits are present:
\[
\text{REE} = \frac{|\text{edits}(c_i^{(k)} \rightarrow \tilde{c}_i^{(k)})| - |\text{edits}(c_i^{(k)} \rightarrow \hat{c}_i^{(k)})|}{|\text{edits}(c_i^{(k)} \rightarrow \hat{c}_i^{(k)})|}.
\]
This combination distinguishes total failure, structurally incorrect but partially useful outputs, and over-editing behavior [2602.14595].

The literature review on code review automation treats current generative-task evaluation as a major weakness. For **Implementing the Code Change Requested by a Reviewer**, the most common metrics are **Accuracy**, **BLEU**, and **CodeBLEU**; for **Predicting the Code Output of the Review Process**, they are **Accuracy**, **BLEU**, and **Levenshtein Distance** [2503.09510]. The same review explicitly argues that lexical-overlap metrics are inadequate for semantically valid alternative edits and that curated benchmarks and better usefulness-oriented metrics remain an open need [2503.09510].

A practical implication is that ACR evaluation already contains two incompatible notions of success: exact accepted-code reproduction and useful partial revision. The comparative study of generation-based code review automation makes this tension explicit by showing that the model with the most exact hits is not always the model with the most useful draft on average [2303.07221].

## 5. Reliability under variation and calibrated confidence

A major recent development is the reframing of ACR as a **reliability-under-variation** problem. The consistency study defines **consistency** as the ability to generate semantically equivalent revisions for semantically equivalent inputs and probes this with nine families of **semantics-preserving perturbations (SPP)** over control flow, data flow, and identifier naming [2602.14595]. The perturbations include if-else swapping, dead exception insertion, dead variable assignment insertion, try-catch wrapping with rethrow, independent line swaps, return-via-variable rewrites, def-use breaking, random variable renaming, and shuffled variable renaming, while preserving comment-code alignment when comments mention renamed identifiers [2602.14595].

Using a JavaParser-based parse-transform-serialize pipeline, that work generates over 10K semantically equivalent variants from 2032 Java methods and evaluates five transformer-based ACR tools: fine-tuned T5-small, LoRA-tuned LLaMA-7B, LLaMA 3.3-70B-Instruct-Turbo, ChatGPT-3.5 Turbo, and DeepSeek-V3 [2602.14595]. The headline result is that semantics-preserving changes can reduce correct revision performance by up to **45.3%**, with the largest drop on LoRA-tuned LLaMA on its own solvable subset; even on the common intersection subset of instances solved by all models on the original input, the worst observed drop is **40.9%** for GPT-3.5 Turbo [2602.14595].

The paper’s regression analysis makes locality central. Perturbation distance from the reviewer-tagged span is the strongest positive predictor of preserved exact match, with estimate **0.118**, odds ratio **1.125**, \(95\%\) CI \((1.076,1.176)\), and \(p=1.85\times10^{-7}\); perturbations inside the tagged span have the strongest negative effect, with estimate **-0.690**, OR **0.502**, and \(p=4.46\times10^{-9}\) [2602.14595]. This supports the claim that current transformer-based ACR systems remain highly dependent on surface form, local token patterns, and structurally aligned contexts.

The same study tests attention-guiding input heuristics—**Code Repetition (CR)**, **Inline Commenting (IC)**, and **Chain-of-Thought (CoT)**—and finds only marginal or inconsistent improvement. In many settings they hurt EXM, with particularly strong degradations under Inline Commenting, leaving consistency under semantics-preserving variation as an open research problem [2602.14595].

Confidence estimation forms a complementary reliability layer. Fine-grained calibration work argues that conventional sequence-level confidence is too coarse for ACR because correctness is often determined by a few local edit decisions rather than by the average confidence of the whole generated sequence [2604.06723]. It studies automated program repair, vulnerability repair, and automated code refinement, and compares normalized sequence likelihood and average token probability against three finer-grained scores: **minimum token probability**, **lowest-\(K\) token probability**, and **attention-weighted uncertainty** [2604.06723]. Across 14 models and three tasks, the strongest practical conclusion is that **minimum token probability** is the best raw confidence score, **global Platt-scaling** is often sufficient for program repair and vulnerability repair, and **local Platt-scaling plus minimum token probability** is necessary for automated code refinement [2604.06723].

## 6. Deployment forms, evaluation limits, and open directions

ACR already appears in several deployment-oriented forms. Revizor packages mined patterns into PyCharm plugins and, in a user evaluation with nine experienced developers, receives average scores of **3.66/4** for correctness of edit operations, **3.77/4** for overall usability, and **3.88/4** for impact on IDE performance, with pattern visualization rated lowest at **3.33/4** [2108.11199]. A static-analysis-guided LLM pipeline reports that in a comprehensive repository-level setting it reduces issues from **7,599** to **1,058**, fixing **5,445** issues in less than three hours and for less than **\$35**, but it still depends on manual verification and manual correction of hallucinated edits, and it does not report build success, unit tests, integration tests, runtime validation, or semantic equivalence checks [2506.10330].

Evaluation in practice remains contested. Industrial work on automated code review bot evaluation shows that automated judges achieve only moderate agreement with developer-provided fixed/wontFix labels, with agreement ratios ranging from approximately **0.44** to **0.62** across Gemini-2.5-pro, GPT-4.1-mini, and GPT-5.2, under both G-Eval and LLM-as-a-Judge formulations [2604.24525]. The paper’s central point is that action labels are operational outcomes shaped by timing, ownership, workflow pressure, and organizational constraints, not objective measures of technical usefulness [2604.24525]. For ACR, this implies that patch acceptance, comment resolution, or developer action cannot be treated as clean gold standards without qualification.

Repository-level review benchmarks sharpen the same point from the upstream side. AACR-Bench introduces multilingual repository-level context and expert-verified defect comments, increasing defect coverage by **285%**, and argues that issue detection quality depends strongly on context granularity, retrieval strategy, base model, language, and whether the system is agentic [2601.19494]. SWRBench, a PR-centric benchmark with full project context, reports that current LLM-based review systems remain weak at realistic issue finding, with the best overall \(F1\) only **19.38** and the best functional-only \(F1\) **26.26**, while a simple multi-review aggregation strategy can raise \(F1\) by up to **43.67%** [2509.01494]. These are review benchmarks rather than code-editing benchmarks, but the associated analyses explicitly matter to ACR because revision systems depend on correct issue identification and localization before patch synthesis.

A broad literature review of code review automation identifies the main open problems for direct ACR as noisy triplet extraction from review history, weak generalization across datasets, limited support for low-resource languages, suboptimal metrics for generative tasks, and insufficiently realistic deployment studies [2503.09510]. The consistency study adds a more specific agenda: future ACR systems may need stronger structural representations, explicit localization mechanisms, training with semantics-preserving augmentation, and objectives that enforce invariance or equivariance under code transformations, because simple prompting heuristics were not enough [2602.14595].

Taken together, the literature suggests a stable characterization of the field. ACR has progressed from method-level seq2seq revision and mined pattern application to retrieval-augmented, agentic, and repository-level pipelines. Yet the same literature also shows that benchmark success can overstate practical reliability, that exact-match metrics can understate partial usefulness, and that harmless structural rewrites can sharply degrade revision quality. A plausible implication is that mature ACR will require a layered architecture: issue localization with repository context, revision generation or pattern application, execution- or rule-grounded validation, calibrated confidence, and workflow-aware human oversight [2602.14595] [2604.06723] [2604.24525].

Source: https://www.emergentmind.com/topics/automated-code-revision-acr