RefactoringMirror: Safe Automated Code Refactoring

• RefactoringMirror is a detect-and-reapply tactic that identifies intended refactorings from LLM-generated code by comparing original and modified Java files.
• It employs a four-step workflow—LLM refactoring, diff detection with ReExtractor, parameter extraction, and reapplication via IntelliJ—to enforce safety and correctness.
• Empirical evaluation showed that RefactoringMirror preserves over 91% of beneficial refactorings while filtering out unsafe semantic and syntax changes.

RefactoringMirror is a detect-and-reapply tactic for LLM-based automated software refactoring: given an original piece of code $c$ and an “improved” version $c'$ generated by an LLM, it detects which refactorings have been performed between $c$ and $c'$, and then re-applies these refactorings to $c$ using a conventional refactoring engine, producing $\hat{c}$. Its stated goal is that $\hat{c}$ contains all the genuine refactorings but none of the unsafe edits that the LLM may have introduced. The tactic was introduced in a study of Java refactorings that found strong potential in LLM-generated refactoring suggestions, together with non-trivial rates of semantic changes and syntax errors when raw LLM patches were accepted directly (Liu et al., 2024).

1. Origin and problem setting

RefactoringMirror emerged from an empirical study on the potential of GPT-4 and Gemini-1.0 Pro in automated software refactoring. That study constructed a dataset of 180 real-world refactorings from 20 projects, restricted to 9 within-document types: Extract Class, Extract Method, Extract Variable, Inline Method, Inline Variable, Rename Attribute, Rename Method, Rename Parameter, and Rename Variable. To make the task realistic, the study used reversed refactorings: for each refactoring with before-version $V_{n-1}$ and after-version $V_n$, the authors manually applied the reverse refactoring to $V_n$, producing $c'$0, and then asked the LLM to recover the refactoring from $c'$1 back to $c'$2 (Liu et al., 2024).

The study separated two tasks. In the opportunity-identification task, the LLM received Java code containing a known opportunity and was asked to refactor it. With a generic prompt, GPT identified 28/180 opportunities and Gemini 7/180. When the prompt explicitly specified the refactoring type, GPT rose to 94/180 (52.2%) and Gemini to 38/180 (21.1%). With refactoring subcategories and few-shot examples, GPT increased to 120/180 (66.7%). With refactoring subcategories plus search-space limitation, GPT reached 156/180, or 86.7%. The study also reported a negative correlation between LOC and success rate: longer files reduced success (Liu et al., 2024).

In the solution-recommendation task, the LLM was told exactly which entity to refactor and which type to perform. GPT recommended 176 refactoring solutions for the 180 refactorings, and 63.6% of those solutions were rated “Good” or “Excellent” by experts. Gemini recommended 137 solutions, with 56.2% rated “Good” or “Excellent.” The central difficulty was safety: 13 out of the 176 GPT solutions and 9 out of the 137 Gemini solutions were unsafe, with 18 semantic bugs and 4 syntax errors. RefactoringMirror was proposed as the mechanism for preserving the LLM’s high-level refactoring intelligence while avoiding those unsafe direct edits (Liu et al., 2024).

2. Detect-and-reapply workflow

The RefactoringMirror workflow begins with an LLM producing a refactored version $c'$3 from original code $c'$4. The system does not require the LLM to emit a structured command language. Instead, it treats the LLM output as a textual Java file and reconstructs structure post hoc by comparing $c'$5 and $c'$6 (Liu et al., 2024).

The second step is refactoring detection. The study used ReExtractor, described as a state-of-the-art refactoring detection system, to analyze $c'$7 and $c'$8. ReExtractor outputs both a list of detected refactorings and a list of low-level differences such as inserted, deleted, and changed AST nodes. At this stage the system knows which refactoring types occurred and which entities they involve, but not yet all parameters needed to invoke a refactoring engine (Liu et al., 2024).

The third step is refactoring parameter extraction. For each detected refactoring instance, the authors implemented customized logic to recover all required parameters in structured form. For Rename Method, for example, the parameters include the original method signature and the new method name. For Extract Variable, they include the target method, the expression range to extract, and the new variable name. The paper does not present pseudocode, but it explicitly states that, for each refactoring type, a customized algorithm was manually designed and implemented to extract the detailed refactoring solutions as a sequence of parameters (Liu et al., 2024).

The fourth step is reapplication via IntelliJ IDEA. The system loads the original code into IntelliJ’s program model, locates the target entities, binds the recovered parameters to IntelliJ’s refactoring operations, and invokes IntelliJ’s refactoring support to perform the transformation. Because IntelliJ performs precondition checks, unsupported or invalid transformations are not forced. If a refactoring fails, RefactoringMirror simply does not apply it. This is the core safety move: only detected refactorings that also satisfy the refactoring engine’s validity constraints are mirrored into $c'$9 (Liu et al., 2024).

The paper describes this as a twofold mirroring process. Transformation mirroring converts an ad hoc LLM patch into explicit refactoring operations. Safety mirroring ensures that unsafe non-refactoring edits are not carried over. A plausible implication is that RefactoringMirror should be understood less as a code-generation system than as a refactoring-intent recovery pipeline grounded in conventional tool support (Liu et al., 2024).

3. Representation and canonical examples

A defining property of RefactoringMirror is that it does not constrain the LLM to output a structured API call, JSON schema, or catalog command. The LLM receives a textual Java file and returns another textual Java file. Structure is then inferred from the diff between the two versions by ReExtractor and the subsequent parameter-recovery algorithms. This design preserves LLM flexibility, but transfers responsibility for safe program transformation to the refactoring detector and the refactoring engine (Liu et al., 2024).

One illustrative case came from CreateBranchCommand.java in JGit. GPT produced a refactoring in which the inline of refToCheck into the assignment for exists was valid, but it also rewrote a guarded if block into a ternary expression of the form startPoint != null ? repo.findRef(startPoint).getName() : null. That rewrite introduced a potential NullPointerException, because repo.findRef(startPoint) could return null, after which .getName() would fail. RefactoringMirror preserved the valid Inline Variable refactoring and discarded or failed to mirror the semantics-changing ternary rewrite. The result kept the behavior-preserving inline but retained the original null-safe control structure (Liu et al., 2024).

A second example came from Checker.java in Checkstyle. GPT performed an Extract Variable refactoring for file.getPath(), but declared the new variable inside a try block while using it in catch blocks, producing a scope error and uncompilable code. RefactoringMirror detected the Extract Variable operation, reconstructed the intended extraction parameters, and invoked IntelliJ’s Extract Variable refactoring on the original code. IntelliJ then chose a declaration location with a scope that covered both the try body and the catch clauses, thereby avoiding the compilation failure. This example shows the specific advantage of detect-and-reapply: the high-level idea of the refactoring may be acceptable even when the LLM’s concrete patch is not (Liu et al., 2024).

These examples also clarify the system’s epistemic division of labor. The LLM proposes a plausible transformation pattern. ReExtractor identifies whether that pattern matches known refactoring types. IntelliJ enforces operational correctness. This separation is the conceptual center of RefactoringMirror (Liu et al., 2024).

4. Evaluation and observed effectiveness

The study evaluated RefactoringMirror on the subset of 22 buggy solutions produced by GPT and Gemini. Across those 22 cases, the LLMs had performed 35 refactorings, including both explicitly requested refactorings and additional refactorings the models introduced. ReExtractor detected 33/35 of those refactorings, a 94.3% detection rate on that subset. RefactoringMirror then attempted to reapply the 33 detected refactorings with IntelliJ and successfully reapplied 32/33. The remaining one was an “if–else to ternary” transformation that IntelliJ does not support as a corresponding refactoring (Liu et al., 2024).

The safety result was the paper’s principal finding: manual expert inspection found that RefactoringMirror’s output $c$0 contained no semantics-changing edits or syntax errors in any of the 22 cases. Expressed another way, the tactic preserved 32/35 of the LLM’s refactorings in safe form, or 91.4%, while avoiding all observed unsafe changes. The dropped transformations were false negatives for mirroring rather than unsafe false positives (Liu et al., 2024).

The broader empirical study used three experienced Java developers to rate LLM-generated solutions on a five-level scale: Excellent, Good, Poor, Failed, and Buggy. The final score was the median of the three ratings, and the reported inter-rater agreement was Fleiss’ kappa $c$1. The study also constructed a test suite of 102 unit tests, consisting of 59 existing tests and 43 auto-generated using TestMe, to help identify semantic issues in the raw LLM outputs (Liu et al., 2024).

For range-based refactorings such as Extract Method and Extract Class, the study treated the opportunity as identified when the Dice-style overlap with the oracle refactoring satisfied

$c$2

with success defined by $c$3. This evaluation criterion matters because it formalizes how close an LLM-generated extraction must be to the developer’s original refactoring to count as the same opportunity (Liu et al., 2024).

5. Relation to adjacent “mirror” paradigms

The explicit tactic called RefactoringMirror belongs to a broader family of “mirror-like” refactoring systems in which software structure is reflected back to developers through another representation. In one adjacent line of work, “RefactoringMirror” is described as a good label for turning the IDE into a live mirror that constantly reflects refactoring opportunities and their impact on code quality back to the developer. That IntelliJ IDEA plugin for Java visually identifies, recommends, and applies Extract Method refactorings in real time, uses color-coded gutter overlays with a severity scale from 1 to 10, and empirically showed that a full live visualization condition achieved stronger improvements than manual refactoring without tool support (Fernandes et al., 2023).

A second adjacent line is post hoc refactoring detection across revisions. RefModel treats refactoring detection as an LLM-based classification and explanation problem over code changes. It uses one-sentence natural-language definitions for refactoring types, compares foundation models against RefactoringMiner, RefDiff, and ReExtractor+, and reports that in real-world settings Claude 3.5 Sonnet and Gemini 2.5 Pro jointly identified 97% of all refactorings, while also showing encouraging generalization to Python and Golang (Simões et al., 15 Jul 2025). This suggests that the detector component in a RefactoringMirror-style pipeline need not be limited to static rule-based detectors.

Cross-language refactoring detection is also relevant. RefactoringMiner++ is described as the first open-source, publicly available refactoring detection tool for C++, built by mapping Clang/libClang output into a model consumable by RefactoringMiner. It can detect refactorings and report behavior-altering changes for C++ programs, although the current version supports only two versions of a single C++ file at a time (Ritz et al., 24 Feb 2025). A plausible implication is that detect-and-reapply mirroring could be extended beyond Java when both a reliable detector and a mature refactoring engine exist.

The practical need for such systems is supported by empirical evidence on developer demand. An analysis of Stack Overflow refactoring discussions identified five dominant domains in which developers ask for refactoring assistance: Code Optimization, Tools and IDEs, Architecture and Design Patterns, Unit Testing, and Database (Peruma et al., 2021). That distribution helps explain the appeal of RefactoringMirror’s hybrid design: it addresses not only raw code transformation, but also trust, tooling, and workflow integration.

6. Limitations, trade-offs, and future directions

RefactoringMirror’s main limitations are inherited from its dependencies. Its coverage is bounded by ReExtractor’s ability to detect refactorings and by IntelliJ IDEA’s ability to reapply them. If ReExtractor misses a refactoring, the system cannot mirror it. If IntelliJ lacks support for a detected transformation, the system drops it rather than forcing a risky edit. This conservative bias lowers recall of the LLM’s ideas, but it is central to the safety argument (Liu et al., 2024).

The technique was evaluated only for within-document refactorings in Java. Cross-file refactorings, multi-module changes, and languages without robust refactoring engines are outside the scope of the reported implementation. The study also notes that RefactoringMirror did not include an automatic validation layer on $c$4: validation of the mirrored outputs was manual rather than systematic recompilation and regression testing. In practice, the tactic is compatible with compilation and test execution as an additional safety layer, but that layer was not integrated into the study’s reported pipeline (Liu et al., 2024).

Another limitation is semantic selectivity. RefactoringMirror filters out changes that are not recognized as refactorings by the detector, even when those changes may be useful in some broader maintenance sense. Conversely, it guarantees safety rather than utility: the paper explicitly notes that RefactoringMirror does not distinguish beneficial from cosmetic or even detrimental refactorings, only whether they can be mirrored safely (Liu et al., 2024).

The paper nevertheless proposes several extensions. It describes possible IDE plugins, CI pipelines, and code review bots that would use LLMs to suggest refactorings, pass the resulting patches through detect-and-reapply, and then present only the engine-applied results. It also points to improved prompt engineering, automated compilation and regression testing, support for composite or multi-step refactorings, semantic quality filtering, and cross-language adaptation as future directions (Liu et al., 2024). Related results on multilingual foundation-model detection (Simões et al., 15 Jul 2025) and C++ refactoring detection (Ritz et al., 24 Feb 2025) suggest that these extensions are technically plausible, though they remain extensions rather than demonstrated properties of the original Java implementation.

In this sense, RefactoringMirror occupies a specific position in the refactoring-tool landscape. It is neither a pure refactoring recommender nor a pure refactoring engine. It is a mediation tactic: LLMs generate the refactoring idea, refactoring detection reconstructs the intended operation, and a conventional engine executes only those transformations that can be justified and applied safely (Liu et al., 2024).