ReFIne: Patch Refinement for APR

• ReFIne is a patch-refinement framework for APR that converts near-correct draft patches into fully correct repairs by addressing context ambiguities and test overfitting.
• It employs a three-module architecture—context disambiguation, test-time scaling for delta diversification, and LLM-powered aggregation via code review—to systematically enhance patch quality.
• Empirical evaluations on SWE-Bench benchmarks show notable improvements in resolution rates, underscoring its effectiveness in refining APR outputs.

ReFIne is a patch-refinement framework for repository-level automatic program repair (APR) with LLMs, introduced to transform “Draft Patches” into correct patches by addressing limited context understanding, ambiguities in natural-language issue descriptions and large codebases, and over-reliance on incomplete regression tests. In the formulation of the method, a repair instance is $x=(I,D,T)$, where $I$ is the issue description, $D$ the codebase, and $T$ the regression test suite; an existing APR tool $R_{\mathrm{init}}(\cdot)$ produces a draft patch $P_{\mathrm{init}}=R_{\mathrm{init}}(x)$, and refinement applies a function $F$ to obtain a refined patch $P_r$ that maximizes semantic correctness (Pabba et al., 4 Oct 2025).

1. Problem setting and formal objective

ReFIne is motivated by a recurrent failure mode in LLM-based APR: the generation of Draft Patches that are “near-correct” but either incomplete or overfit to the available test suite. The paper characterizes three causes. First, natural-language issue descriptions $I$ and large codebases $D$ contain ambiguities that lead to misinterpretation of root cause. Second, regression tests $I$0 are necessary but not sufficient for semantic correctness, so a patch that passes $I$1 may still be invalid. Third, current APR pipelines often lack an explicit refinement stage capable of converting a partially correct candidate into a semantically correct repair (Pabba et al., 4 Oct 2025).

The refinement objective is defined as

$I$2

such that

$I$3

where

$I$4

This formulation is significant because it makes explicit that ReFIne is not an APR seed generator. It is a post-generation refinement module applied to the output of an existing repair system. The paper states that it is implemented as a general refinement module that can be integrated into both open-agent-based and workflow-based APR systems (Pabba et al., 4 Oct 2025).

A central misconception addressed by this formulation is that test-passing is equivalent to repair correctness. ReFIne is built precisely around the opposite claim: passing the regression suite does not guarantee that the issue semantics have been satisfied.

2. Three-module architecture

ReFIne comprises three LLM-agentic modules: Context Disambiguation, Test-time Scaling for Candidate Diversification, and LLM-powered Aggregation via Code Review. The architecture is explicitly staged so that ambiguity reduction precedes candidate expansion, and candidate expansion precedes synthesis and validation (Pabba et al., 4 Oct 2025).

Module	Core representation	Function
Context Disambiguation	$I$5 and $I$6	Disambiguates issue and code context
Test-time Scaling	$I$7	Diversifies targeted patch deltas
Aggregation via Code Review	$I$8	Reviews, groups, merges, and validates candidates

The first module performs issue and code context extraction. Issue context extraction maps the issue description $I$9 to a structured 5-tuple

$D$0

where $D$1 denotes affected components, $D$2 the intended behavioral change, $D$3 a high-level edit plan, $D$4 invariants to preserve, and $D$5 the scope beyond the example. Code context extraction is defined per patch location $D$6 in $D$7 as

$D$8

with $D$9 for data dependencies, $T$0 for control dependencies, $T$1 for invariant constraints, and $T$2 for call-graph context. The prompt-engineering strategy includes templates such as “Given $T$3 and $T$4, enumerate target modules and invariants,” and analogous prompts for code context (Pabba et al., 4 Oct 2025).

This stage gives ReFIne a structured intermediate representation absent from many direct patch-generation workflows. A plausible implication is that refinement quality depends not only on the initial patch diff, but on whether issue semantics and surrounding program structure are made explicit before further editing.

3. Candidate diversification through test-time scaling

The second module is based on the philosophy of generating $T$5 diverse “delta patches” $T$6 by repeated sampling from the LLM in sampling mode with temperature $T$7. The token sampling distribution is written as

$T$8

where a higher $T$9 yields more exploratory token choices (Pabba et al., 4 Oct 2025).

Delta-patch generation constructs a prompt combining the structured issue context $R_{\mathrm{init}}(\cdot)$0, the code context $R_{\mathrm{init}}(\cdot)$1, and the diff-style $R_{\mathrm{init}}(\cdot)$2, then queries the LLM $R_{\mathrm{init}}(\cdot)$3 times to produce small, targeted diffs. In the reported protocol, the default hyperparameters are $R_{\mathrm{init}}(\cdot)$4, $R_{\mathrm{init}}(\cdot)$5 samples, and $R_{\mathrm{init}}(\cdot)$6. Claude 3.7-Sonnet is used by default for context and delta generation (Pabba et al., 4 Oct 2025).

The design choice is notable because ReFIne does not attempt to replace the seed patch with an entirely new patch family. Instead, it produces localized deltas around the original draft patch. This suggests a refinement-centric view of APR in which diversity is applied to patch edits rather than to full repair trajectories.

The hyperparameter study directly supports this interpretation. More retries and higher $R_{\mathrm{init}}(\cdot)$7 increase diversity and resolved rate: $R_{\mathrm{init}}(\cdot)$8 retry with $R_{\mathrm{init}}(\cdot)$9 yields $P_{\mathrm{init}}=R_{\mathrm{init}}(x)$0, while $P_{\mathrm{init}}=R_{\mathrm{init}}(x)$1 retries with $P_{\mathrm{init}}=R_{\mathrm{init}}(x)$2 yields $P_{\mathrm{init}}=R_{\mathrm{init}}(x)$3; the comparison of $P_{\mathrm{init}}=R_{\mathrm{init}}(x)$4 versus $P_{\mathrm{init}}=R_{\mathrm{init}}(x)$5 versus $P_{\mathrm{init}}=R_{\mathrm{init}}(x)$6 shows monotonic improvement with diversity (Pabba et al., 4 Oct 2025).

4. Aggregation, review, and iterative validation

The third module uses an LLM-powered code review process to retain, reconcile, and validate partial fixes. For each candidate

$P_{\mathrm{init}}=R_{\mathrm{init}}(x)$7

a Code Reviewer Agent judges relevance via

$P_{\mathrm{init}}=R_{\mathrm{init}}(x)$8

and only candidates with $P_{\mathrm{init}}=R_{\mathrm{init}}(x)$9 are retained (Pabba et al., 4 Oct 2025).

Aggregation proceeds in several stages. ReFIne de-duplicates candidates and groups mutually conflicting patches, where two patches conflict if they modify the same line differently. Within each group $F$0, an LLM is prompted to merge or select the best combined diff based on $F$1. All non-conflicting unified diffs are then merged into a single patch $F$2. A Patch Validator executes $F$3 on $F$4; if $F$5 passes, the system returns $F$6, otherwise it sets $F$7 and repeats up to $F$8 (Pabba et al., 4 Oct 2025).

Gemini 2.5-Pro is the default reviewer model in the reported setup. Backend selection is not treated as incidental. The paper reports three configurations: homogeneous Claude 3.7 at $F$9 $P_r$0, homogeneous Gemini 2.5 at $P_r$1 $P_r$2, and a mixed setup using Claude 3.7 for Context/Delta and Gemini 2.5 for Reviewer at $P_r$3 $P_r$4, which is the best reported combination (Pabba et al., 4 Oct 2025).

This module encodes the paper’s broader claim that agentic collaboration is a missing component in current APR pipelines. The improvement is not attributed to a single stronger model call, but to a division of labor among extraction, diversification, review, and validation.

5. Experimental evaluation and empirical behavior

The evaluation is conducted on SWE-Bench Lite and SWE-Bench Verified. SWE-Bench Lite contains 300 real-world Python GitHub issues, while SWE-Bench Verified contains 500 issues with private test suites for final evaluation. The reported metrics are Resolved Issue Rate, Correct Location Rate, and cost and token usage per issue. The evaluation protocol is single-pass, with no external Internet and hidden test suites (Pabba et al., 4 Oct 2025).

Evaluation setting	Baseline or change	Result
SWE-Bench Lite (AutoCodeRover seed)	AutoCodeRover alone	37.00%
SWE-Bench Lite (AutoCodeRover seed)	REFINE	51.67%
SWE-Bench Verified	Resolution rate	51.6% $P_r$5 63.8%

On SWE-Bench Lite, ReFIne boosts AutoCodeRover by $P_r$6 percentage points, from $P_r$7 to $P_r$8, surpassing all prior baselines and achieving state-of-the-art results among workflow-based approaches. On SWE-Bench Verified, it improves the resolution rate by $P_r$9 percentage points, from $I$0 to $I$1. When integrated across multiple APR systems, it yields cross-tool gains of $I$2 to $I$3 percentage points, with an average improvement of $I$4 percentage points. The paper also reports that ReFIne fixes $I$5 unique issues that no other SOTA tool resolves (Pabba et al., 4 Oct 2025).

The baseline set spans workflow-based systems—AutoCodeRover, Agentless-1.5, SpecRover, ExpeRepair, and OrcaLoca+Agentless—and agentic systems including SWE-agent, OpenHands, DARS, devlo, BlackBoxAI, Globant Code Fixer, CodeV, Codart AI, CodeStory, Lingxi, and Moatless Tools. This breadth is important because the reported average gain is presented as evidence of broad effectiveness and generalizability rather than seed-specific tuning (Pabba et al., 4 Oct 2025).

Ablation studies show that removing any module among Context, Delta Generation, and Reviewer reduces resolved rate by $I$6 to $I$7 percentage points, and that all three modules together yield the best result. This is consistent with the system design: context extraction alone does not substitute for patch diversity, and diversity alone does not substitute for review-driven aggregation (Pabba et al., 4 Oct 2025).

6. Interpretation, scope, and limitations

ReFIne is presented as a general refinement module rather than a monolithic APR agent. Its intended contribution is to close the gap between near-correct and correct patches by systematically transforming draft patches through context disambiguation, diversified delta patch generation via test-time LLM sampling, and LLM-powered aggregation (Pabba et al., 4 Oct 2025).

The reported results support two methodological conclusions. First, refinement functions as a distinct stage in APR rather than as a by-product of generation. Second, the combination of structured context extraction and agentic collaboration improves both workflow-based and open-agent-based seeds. The paper’s summary claim is that these results highlight “the effectiveness of refinement as a missing component in current APR pipelines” and “the potential of agentic collaboration in closing the gap between near-correct and correct patches” (Pabba et al., 4 Oct 2025).

The paper also states several threats to validity and limitations. External validity is constrained because SWE-Bench issues may be higher quality than noisy real-world reports. Model dependency remains substantial because the results are tied to Claude and Gemini, and future LLMs may differ. Scalability is a concern because very large codebases may exceed context windows and cost budgets (Pabba et al., 4 Oct 2025).

Potential extensions identified in the paper include hierarchical context extraction at multi-file and pull-request level, a dynamic retriever to select minimal code slices for context, and integration with fine-tuned code-specialist LLMs. The implementation and replication assets are reported as open source via the anonymous GitHub repository linked in the paper (Pabba et al., 4 Oct 2025).