---
title: 'RefactoringMirror: Safe Automated Code Refactoring'
url: https://www.emergentmind.com/topics/refactoringmirror
type: topic
---

# RefactoringMirror: Safe Automated Code Refactoring

RefactoringMirror is a detect-and-reapply tactic for LLM-based automated software refactoring: given an original piece of code \(c\) and an “improved” version \(c'\) generated by an LLM, it detects which refactorings have been performed between \(c\) and \(c'\), and then re-applies these refactorings to \(c\) using a conventional refactoring engine, producing \(\hat{c}\). Its stated goal is that \(\hat{c}\) contains all the genuine refactorings but none of the unsafe edits that the LLM may have introduced. The tactic was introduced in a study of Java refactorings that found strong potential in LLM-generated refactoring suggestions, together with non-trivial rates of semantic changes and syntax errors when raw LLM patches were accepted directly [2411.04444].

## 1. Origin and problem setting

RefactoringMirror emerged from an empirical study on the potential of GPT-4 and Gemini-1.0 Pro in automated software refactoring. That study constructed a dataset of **180 real-world refactorings** from **20 projects**, restricted to **9 within-document types**: Extract Class, Extract Method, Extract Variable, Inline Method, Inline Variable, Rename Attribute, Rename Method, Rename Parameter, and Rename Variable. To make the task realistic, the study used reversed refactorings: for each refactoring with before-version \(V_{n-1}\) and after-version \(V_n\), the authors manually applied the reverse refactoring to \(V_n\), producing \(V_n'\), and then asked the LLM to recover the refactoring from \(V_n'\) back to \(V_n\) [2411.04444].

The study separated two tasks. In the **opportunity-identification** task, the LLM received Java code containing a known opportunity and was asked to refactor it. With a generic prompt, GPT identified **28/180** opportunities and Gemini **7/180**. When the prompt explicitly specified the refactoring type, GPT rose to **94/180** (**52.2%**) and Gemini to **38/180** (**21.1%**). With refactoring subcategories and few-shot examples, GPT increased to **120/180** (**66.7%**). With refactoring subcategories plus search-space limitation, GPT reached **156/180**, or **86.7%**. The study also reported a negative correlation between LOC and success rate: longer files reduced success [2411.04444].

In the **solution-recommendation** task, the LLM was told exactly which entity to refactor and which type to perform. GPT recommended **176** refactoring solutions for the **180** refactorings, and **63.6%** of those solutions were rated “Good” or “Excellent” by experts. Gemini recommended **137** solutions, with **56.2%** rated “Good” or “Excellent.” The central difficulty was safety: **13 out of the 176** GPT solutions and **9 out of the 137** Gemini solutions were unsafe, with **18** semantic bugs and **4** syntax errors. RefactoringMirror was proposed as the mechanism for preserving the LLM’s high-level refactoring intelligence while avoiding those unsafe direct edits [2411.04444].

## 2. Detect-and-reapply workflow

The RefactoringMirror workflow begins with an LLM producing a refactored version \(c'\) from original code \(c\). The system does not require the LLM to emit a structured command language. Instead, it treats the LLM output as a textual Java file and reconstructs structure post hoc by comparing \(c\) and \(c'\) [2411.04444].

The second step is **refactoring detection**. The study used **ReExtractor**, described as a state-of-the-art refactoring detection system, to analyze \(c\) and \(c'\). ReExtractor outputs both a list of detected refactorings and a list of low-level differences such as inserted, deleted, and changed AST nodes. At this stage the system knows which refactoring types occurred and which entities they involve, but not yet all parameters needed to invoke a refactoring engine [2411.04444].

The third step is **refactoring parameter extraction**. For each detected refactoring instance, the authors implemented customized logic to recover all required parameters in structured form. For Rename Method, for example, the parameters include the original method signature and the new method name. For Extract Variable, they include the target method, the expression range to extract, and the new variable name. The paper does not present pseudocode, but it explicitly states that, for each refactoring type, a customized algorithm was manually designed and implemented to extract the detailed refactoring solutions as a sequence of parameters [2411.04444].

The fourth step is **reapplication via IntelliJ IDEA**. The system loads the original code into IntelliJ’s program model, locates the target entities, binds the recovered parameters to IntelliJ’s refactoring operations, and invokes IntelliJ’s refactoring support to perform the transformation. Because IntelliJ performs precondition checks, unsupported or invalid transformations are not forced. If a refactoring fails, RefactoringMirror simply does not apply it. This is the core safety move: only detected refactorings that also satisfy the refactoring engine’s validity constraints are mirrored into \(\hat{c}\) [2411.04444].

The paper describes this as a twofold mirroring process. **Transformation mirroring** converts an ad hoc LLM patch into explicit refactoring operations. **Safety mirroring** ensures that unsafe non-refactoring edits are not carried over. A plausible implication is that RefactoringMirror should be understood less as a code-generation system than as a refactoring-intent recovery pipeline grounded in conventional tool support [2411.04444].

## 3. Representation and canonical examples

A defining property of RefactoringMirror is that it does **not** constrain the LLM to output a structured API call, JSON schema, or catalog command. The LLM receives a textual Java file and returns another textual Java file. Structure is then inferred from the diff between the two versions by ReExtractor and the subsequent parameter-recovery algorithms. This design preserves LLM flexibility, but transfers responsibility for safe program transformation to the refactoring detector and the refactoring engine [2411.04444].

One illustrative case came from `CreateBranchCommand.java` in JGit. GPT produced a refactoring in which the inline of `refToCheck` into the assignment for `exists` was valid, but it also rewrote a guarded `if` block into a ternary expression of the form `startPoint != null ? repo.findRef(startPoint).getName() : null`. That rewrite introduced a potential `NullPointerException`, because `repo.findRef(startPoint)` could return `null`, after which `.getName()` would fail. RefactoringMirror preserved the valid **Inline Variable** refactoring and discarded or failed to mirror the semantics-changing ternary rewrite. The result kept the behavior-preserving inline but retained the original null-safe control structure [2411.04444].

A second example came from `Checker.java` in Checkstyle. GPT performed an **Extract Variable** refactoring for `file.getPath()`, but declared the new variable inside a `try` block while using it in `catch` blocks, producing a scope error and uncompilable code. RefactoringMirror detected the Extract Variable operation, reconstructed the intended extraction parameters, and invoked IntelliJ’s Extract Variable refactoring on the original code. IntelliJ then chose a declaration location with a scope that covered both the `try` body and the `catch` clauses, thereby avoiding the compilation failure. This example shows the specific advantage of detect-and-reapply: the high-level idea of the refactoring may be acceptable even when the LLM’s concrete patch is not [2411.04444].

These examples also clarify the system’s epistemic division of labor. The LLM proposes a plausible transformation pattern. ReExtractor identifies whether that pattern matches known refactoring types. IntelliJ enforces operational correctness. This separation is the conceptual center of RefactoringMirror [2411.04444].

## 4. Evaluation and observed effectiveness

The study evaluated RefactoringMirror on the subset of **22 buggy solutions** produced by GPT and Gemini. Across those 22 cases, the LLMs had performed **35 refactorings**, including both explicitly requested refactorings and additional refactorings the models introduced. ReExtractor detected **33/35** of those refactorings, a **94.3%** detection rate on that subset. RefactoringMirror then attempted to reapply the **33** detected refactorings with IntelliJ and successfully reapplied **32/33**. The remaining one was an “if–else to ternary” transformation that IntelliJ does not support as a corresponding refactoring [2411.04444].

The safety result was the paper’s principal finding: manual expert inspection found that RefactoringMirror’s output \(\hat{c}\) contained **no semantics-changing edits or syntax errors in any of the 22 cases**. Expressed another way, the tactic preserved **32/35** of the LLM’s refactorings in safe form, or **91.4%**, while avoiding all observed unsafe changes. The dropped transformations were false negatives for mirroring rather than unsafe false positives [2411.04444].

The broader empirical study used three experienced Java developers to rate LLM-generated solutions on a five-level scale: Excellent, Good, Poor, Failed, and Buggy. The final score was the median of the three ratings, and the reported inter-rater agreement was **Fleiss’ kappa \(= 0.82\)**. The study also constructed a test suite of **102 unit tests**, consisting of **59 existing tests** and **43 auto-generated using TestMe**, to help identify semantic issues in the raw LLM outputs [2411.04444].

For range-based refactorings such as Extract Method and Extract Class, the study treated the opportunity as identified when the Dice-style overlap with the oracle refactoring satisfied

$$
tolerance = \frac{2 \times \#commons}{\#extracted + \#oracle}
$$

with success defined by \(tolerance \ge 0.5\). This evaluation criterion matters because it formalizes how close an LLM-generated extraction must be to the developer’s original refactoring to count as the same opportunity [2411.04444].

## 5. Relation to adjacent “mirror” paradigms

The explicit tactic called RefactoringMirror belongs to a broader family of “mirror-like” refactoring systems in which software structure is reflected back to developers through another representation. In one adjacent line of work, “RefactoringMirror” is described as a good label for turning the IDE into a live mirror that constantly reflects refactoring opportunities and their impact on code quality back to the developer. That IntelliJ IDEA plugin for Java visually identifies, recommends, and applies **Extract Method** refactorings in real time, uses color-coded gutter overlays with a severity scale from **1** to **10**, and empirically showed that a full live visualization condition achieved stronger improvements than manual refactoring without tool support [2307.11010].

A second adjacent line is post hoc refactoring detection across revisions. **RefModel** treats refactoring detection as an LLM-based classification and explanation problem over code changes. It uses one-sentence natural-language definitions for refactoring types, compares foundation models against RefactoringMiner, RefDiff, and ReExtractor+, and reports that in real-world settings **Claude 3.5 Sonnet and Gemini 2.5 Pro jointly identified 97% of all refactorings**, while also showing encouraging generalization to Python and Golang [2507.11346]. This suggests that the detector component in a RefactoringMirror-style pipeline need not be limited to static rule-based detectors.

Cross-language refactoring detection is also relevant. **RefactoringMiner++** is described as the first open-source, publicly available refactoring detection tool for C++, built by mapping Clang/libClang output into a model consumable by RefactoringMiner. It can detect refactorings and report behavior-altering changes for C++ programs, although the current version supports only two versions of a single C++ file at a time [2502.17716]. A plausible implication is that detect-and-reapply mirroring could be extended beyond Java when both a reliable detector and a mature refactoring engine exist.

The practical need for such systems is supported by empirical evidence on developer demand. An analysis of Stack Overflow refactoring discussions identified five dominant domains in which developers ask for refactoring assistance: **Code Optimization**, **Tools and IDEs**, **Architecture and Design Patterns**, **Unit Testing**, and **Database** [2110.12229]. That distribution helps explain the appeal of RefactoringMirror’s hybrid design: it addresses not only raw code transformation, but also trust, tooling, and workflow integration.

## 6. Limitations, trade-offs, and future directions

RefactoringMirror’s main limitations are inherited from its dependencies. Its coverage is bounded by **ReExtractor’s ability to detect refactorings** and by **IntelliJ IDEA’s ability to reapply them**. If ReExtractor misses a refactoring, the system cannot mirror it. If IntelliJ lacks support for a detected transformation, the system drops it rather than forcing a risky edit. This conservative bias lowers recall of the LLM’s ideas, but it is central to the safety argument [2411.04444].

The technique was evaluated only for **within-document refactorings** in **Java**. Cross-file refactorings, multi-module changes, and languages without robust refactoring engines are outside the scope of the reported implementation. The study also notes that RefactoringMirror did not include an automatic validation layer on \(\hat{c}\): validation of the mirrored outputs was manual rather than systematic recompilation and regression testing. In practice, the tactic is compatible with compilation and test execution as an additional safety layer, but that layer was not integrated into the study’s reported pipeline [2411.04444].

Another limitation is semantic selectivity. RefactoringMirror filters out changes that are not recognized as refactorings by the detector, even when those changes may be useful in some broader maintenance sense. Conversely, it guarantees safety rather than utility: the paper explicitly notes that RefactoringMirror does not distinguish beneficial from cosmetic or even detrimental refactorings, only whether they can be mirrored safely [2411.04444].

The paper nevertheless proposes several extensions. It describes possible **IDE plugins**, **CI pipelines**, and **code review bots** that would use LLMs to suggest refactorings, pass the resulting patches through detect-and-reapply, and then present only the engine-applied results. It also points to improved prompt engineering, automated compilation and regression testing, support for composite or multi-step refactorings, semantic quality filtering, and cross-language adaptation as future directions [2411.04444]. Related results on multilingual foundation-model detection [2507.11346] and C++ refactoring detection [2502.17716] suggest that these extensions are technically plausible, though they remain extensions rather than demonstrated properties of the original Java implementation.

In this sense, RefactoringMirror occupies a specific position in the refactoring-tool landscape. It is neither a pure refactoring recommender nor a pure refactoring engine. It is a mediation tactic: LLMs generate the refactoring idea, refactoring detection reconstructs the intended operation, and a conventional engine executes only those transformations that can be justified and applied safely [2411.04444].

Source: https://www.emergentmind.com/topics/refactoringmirror