---
title: Self-Refinement With Execution Feedback
url: https://www.emergentmind.com/topics/self-refinement-with-execution-feedback
type: topic
---

# Self-Refinement With Execution Feedback

Self-refinement with execution feedback is an iterative paradigm in which a model generates a candidate artifact, executes it against an external environment or verifier, converts the resulting runtime signal into corrective feedback, and then revises the artifact under that feedback. In recent work, the artifact may be a Python function, a SQL query, a competitive-programming solution, a long-horizon plan, a trajectory, or a Behavior Tree; the feedback may be failed unit tests, compiler or runtime errors, result mismatches, structured diffs, symbolic verification failures, or continuous state-evaluation traces [2412.03578][2306.14898][2502.00675][2604.00790][2605.11225][2508.15501].

## 1. Formal definitions and objective functions

A central formulation appears in PerfCodeGen, which defines code generation as a correctness-constrained runtime minimization problem. For a programming problem $x$ with unit-test suite $U(x)=\{u_x^j\}_{j=1}^J$, an LLM defines a conditional distribution $p_{\theta}(y\mid x)$ over candidate solutions $y$. Functional correctness requires that a candidate pass all tests in $U(x)$. Runtime is estimated per test by
$$
\hat t(y,u_x^j)=\frac{1}{E-2}\sum_{e=2}^{E-1} t(y,u_x^j)[e],
$$
where $E$ independent timing measurements are sorted and the minimum and maximum are discarded. Total cost is
$$
T(y)\triangleq \sum_{j=1}^J \hat t(y,u_x^j),
$$
and the target is
$$
y^*=\arg\min_{y:\,\forall j\,\,\text{pass}(y,u_x^j)} T(y).
$$
Because directly optimizing $p_{\theta}(y\mid x)$ for minimal $T(y)$ is intractable, PerfCodeGen uses an iterative generate–execute–feedback–refine procedure [2412.03578].

A broader formalization appears in InterCode, which treats interactive code generation as a Markov Decision Process
$$
\mathcal{M}=(\mathcal{S},\mathcal{A},\mathcal{P},\mathcal{O},\mathcal{R},\gamma),
$$
with code snippets as actions and execution feedback as observations. The observation at step $t$ is represented as
$$
o_t=(\texttt{stdout}_t,\texttt{stderr}_t,\texttt{exit\_code}_t,\Delta_t),
$$
where $\Delta_t$ may denote filesystem diffs, SQL result rows, or Python unit-test outcomes. In this view, iterative refinement is a policy over interaction histories rather than a one-shot transduction problem [2306.14898].

RefineRL gives a third formalization for competitive programming. The state is the concatenation of problem statement $x$ and previous feedback $r_{\text{fed}}$, the action is the generated trajectory $(r_{\text{cot}},r_{\text{sol}},r_{\text{code}})$, and the reward uses a dense squared-incentive based on public-test pass count:
$$
R=\Bigl(\tfrac{k^*}{|T|}\Bigr)^2,
$$
where $k^*$ is either the number of public tests passed or the index of the first failed test minus one. This reward gives nonzero credit for partial success and explicitly couples self-refinement to reinforcement learning [2604.00790].

## 2. Iterative refinement architectures

Despite domain differences, recent systems converge on a common outer loop: generate an initial candidate, execute it, summarize execution into a targeted feedback object, produce a revised candidate, and either accept the revision, continue refining, or fall back to an earlier version.

PerfCodeGen separates this loop into correctness refinement and performance refinement. It first samples $K$ seed solutions from the base prompt. Any seed that fails a test receives verbal feedback consisting of the failing test case and error message, prompting the LLM to “reflect” and then “plan” a fix. Correct solutions are collected into $C_{\mathrm{correct}}$. For each correct solution, the framework then measures runtime on all tests, identifies the single “hottest” test
$$
f=\arg\max_{j=1..J}\hat t(y,u_x^j),
$$
and prompts the model to optimize with respect to that test. If the refined candidate still passes all tests, it is retained; otherwise it is discarded, and the original candidate remains eligible. Final selection is the fastest correct program among seeds and refined variants, and the performance-refinement prompt is decoded greedily with temperature $0$ [2412.03578].

PairCoder introduces a two-agent decomposition of the same pattern. A Navigator agent reflects on the problem, generates multiple high-level plans, clusters them using embeddings and k-means++, and selects a representative plan. A Driver agent implements the selected plan, executes the resulting code against the public test suite, and returns structured feedback in $\{\text{Pass},\text{Runtime Error},\text{Wrong Answer},\text{TLE}\}$. The Navigator then decides whether to issue a repair strategy under the same plan or abandon the plan and select another. Plan abandonment is triggered when repeated $(C,F)$ pairs appear in the long-term memory formed by `HistCode` and `HistFeed` [2409.05001].

ReFoRCE adapts the loop to Text-to-SQL. After schema compression and expected-answer-format construction, the agent generates SQL, executes it against the actual database, checks validity, non-emptiness, correct columns, and format conformance, and then patches failures by feeding the error message or “empty result” back into the LLM. The loop stops early when the same valid result appears twice in a single thread, and a parallel majority-vote stage over $T=3$ threads provides a second level of consensus. ReFoRCE also uses execution feedback earlier in the pipeline during column exploration, where 10–20 exploratory `SELECT DISTINCT` queries are executed in batch and failing queries are retried up to 3 times [2502.00675].

RefineRL alters the stopping logic itself. Its Skeptical-Agent always constructs refinement feedback, even when public tests pass. Failure cases invoke an error-type-specific prompt containing test input, expected output, predicted output, and failure class. Passing cases invoke a checklist-driven skepticism prompt asking the model to re-verify correctness, complexity, and anti-hard-coding. A restart rule drops feedback after $M_1$ consecutive failures to avoid local minima, while a termination rule stops after $M_2$ consecutive “all-pass+skeptical-pass” iterations [2604.00790].

## 3. Feedback representations and verification mechanisms

Execution feedback is not a single object. Different systems expose different slices of runtime behavior, and the choice of representation materially affects refinement quality.

InterCode standardizes feedback as textual observations containing `stdout`, `stderr`, `exit_code`, and a task-dependent diff. Unlocking LLM Code Correction with Iterative Feedback Loops uses compiler error messages, failing testcases with expected and actual outputs, and `TLE` or `MLE` warnings. Feedback+ in conversational business analytics executes candidate code in a sandbox and returns a record containing success or failure, `stderr` trace, `stdout`, and result-mismatch information; the next candidate is generated from $(Q,c^{(t)},r^{(t)})$ rather than from a separate learned discriminator [2306.14898][2606.17514][2601.00224].

PerfCodeGen shows that feedback can be non-numeric even when the underlying objective is numeric. The framework measures runtime latency over repeated executions, but it does not inject an explicit numeric scoring function into the LLM. Instead, it verbalizes that a single test case is “the most time-consuming” and asks the model to optimize the code with respect to that test. This hot-spot hint is intended to focus the model on the slowest code path while final selection remains purely runtime-based [2412.03578].

In planning domains, execution feedback is often mediated by symbolic or structured verifiers rather than raw logs. The symbolic feedback-driven planning framework uses a verifier $V$ that performs syntactic checks, semantic checks with a PDDL validator, and goal-reachability checks via a plan recognizer. The verifier’s outputs are translated into corrective natural-language instructions before the next planning iteration. PIVOT similarly transforms plan-execution discrepancies into a structured loss and a “textual gradient” that identifies outcome failure, execution divergence, the earliest break point, and actionable repair instructions for suffix rewriting [2606.27757][2605.11225].

Embodied systems require yet another feedback form. SRDrone replaces single-frame final-state checks with continuous state evaluation: high-frequency sensor streams are filtered at each action boundary, converted into action-annotated states, transformed by the Continuous Motion and Spatial Reasoning module into a semantic trajectory, and then judged by the LLM for success or failure with an explanatory narrative [2508.15501].

| System | Executed artifact | Feedback signal |
|---|---|---|
| PerfCodeGen | Python function | hottest unit test by $\hat t(y,u)$ |
| InterCode | Bash / SQL / Python code | `stdout`, `stderr`, `exit_code`, diffs |
| ReFoRCE | SQL candidate | error message, empty result, format mismatch |
| PIVOT | trajectory | structured loss and textual gradient |
| SRDrone | Behavior Tree execution | semantic trajectory and failure explanation |

## 4. Training-free and trained variants

The literature divides sharply between training-free refinement and refinement internalized through supervised or reinforcement learning.

Training-free systems preserve the base model and alter only the inference loop. PerfCodeGen is explicitly described as a training-free framework. PairCoder relies on prompt-based collaboration between Navigator and Driver. ReFoRCE combines self-refinement, consensus enforcement, and column exploration without parameter updates. Feedback+ replaces semantic discrimination with execution-driven correction prompts inside a generator-discriminator scaffold. PIVOT treats trajectories as optimizable objects and performs PLAN–INSPECT–EVOLVE–VERIFY without model fine-tuning. CP-Agent likewise raises performance “without updating any parameters” by combining Dual-Granularity Verification, Test Augmentation, and Experience-Driven Self-Evolving under a calibrated stopped-process model [2412.03578][2409.05001][2502.00675][2601.00224][2605.11225][2605.24693].

CYCLE represents a supervised alternative. It first fine-tunes a code LM on guaranteed-correct canonical solutions, then probes the model to collect faulty generations and execution feedback, and finally trains on prompts that concatenate natural-language problem descriptions, faulty code, and execution output before predicting the ground-truth solution. To prevent trivial copying from faulty code to corrected code, it introduces a Past Generation Mask with a small masking ratio, and it mixes self-refinement data with one-shot data at a tuned ratio such as 25% self-refinement and 75% one-shot. Test-time inference repeats the refinement loop up to $K=4$ [2403.18746].

RefineRL extends refinement into offline reinforcement learning. It synthesizes skeptical refinement trajectories using the Skeptical-Agent, forms
$$
D_{\mathrm{RL}}=D_{\mathrm{CP}}\cup D_{\mathrm{Err}}\cup D_{\mathrm{Skep}},
$$
and fine-tunes with GRPO using the squared-incentive reward. The design goal is to internalize the agentic loop so that both single-attempt and multi-attempt behavior improve [2604.00790].

Self-Execution Simulation Improves Coding Models pushes the internalization further by training code LLMs to simulate program execution in natural language. The model is first supervised on natural-language execution traces and then optimized with verifiable rewards for output prediction and competitive-programming solving. At inference time, iterative self-fixing can use predicted execution feedback instead of a real interpreter, as in Self-RLEF, where the model simulates the output of its current code on public tests and then decides whether to `SUBMIT` or produce a refactored solution [2604.03253].

BanglaForge combines retrieval, translation, dual-model collaboration, and execution-driven refinement in a low-resource setting. A Coder LLM generates code and synthetic tests, a Reviewer LLM refines both code and tests, and any non-`OK` error signal among `Syntax Error`, `Runtime Error`, `Assertion Failure`, `Timeout`, or `System Exit` triggers guided feedback hints. The maximum number of iterations is $M=5$ [2512.19122].

## 5. Reported empirical behavior

Across the cited systems, execution feedback is repeatedly associated with substantial gains, although the gains are task-dependent and concentrated on specific error classes.

PerfCodeGen reports state-of-the-art runtime efficiency on HumanEval, MBPP, and APPS. Under Best@8 sampling on HumanEval, GPT-3.5 improves `%Opt` from 29.6% to 38.9%, GPT-4 improves from 39.3% to 46.6%, and Phi-3-mini improves from 35.98% to 40.85%. The paper also states that on MBPP and APPS there are similar consistent gains in `%Opt` and `%Correct`, and that hot-spot feedback outperforms few-shot prompts, strategy lists, multi-round planning prompts, and naive slower/faster execution-feedback schemes, typically adding 3–6 points in `%Opt` over the base [2412.03578].

PairCoder reports large pass@1 gains from coupling multi-plan exploration with feedback-driven repair. On GPT-3.5-Turbo and DeepSeek-Coder-Instruct(33B), HumanEval rises from 67.7% to 87.8%, and CodeContest-test rises from 6.06% to 15.15%. Removing multi-plan exploration lowers HumanEval to 81.1% and CodeContest-test to 10.91%, while removing feedback-driven repair lowers them to 74.4% and 9.69%, respectively [2409.05001].

ReFoRCE reports leaderboard-level Text-to-SQL results, with scores of 35.83 on Spider 2.0-Snow and 36.56 on Spider 2.0-Lite. On execution accuracy, Spider-Agent records 20.29% on Spider 2.0-Snow and 20.66% on Spider 2.0-Lite, whereas ReFoRCE records 26.69% and 24.50%. The mechanism attributed to these gains is the combination of early detection of syntax and lookup errors during column exploration, on-the-fly self-correction of failing SQL, and self-consistency filtering [2502.00675].

Several systems report that refinement training can partially substitute for model scaling. CYCLE improves self-refinement pass rates by up to 63.5% across model sizes and benchmarks; for the 350M variant, HumanEval rises from 14.0% one-time to 20.7% after self-refinement, MBPP-S rises from 19.9% to 32.6%, and APPS rises from 7.5% to 8.7%. The paper further states that Cycle-350M outperforms StarCoder-1B in self-refinement ability and that Cycle-1B matches or beats StarCoder-3B [2403.18746].

RefineRL reports that a compact 4B model can benefit strongly from skeptical refinement and RL. On LiveCodeBench v5/v6, RefineRL-4B + Skeptical@16 reaches 64.07% and 56.54%, compared with 62.69% and 59.25% for Qwen3-32B, while RefineRL-4B-2507 + Skeptical@16 reaches 75.97% and 67.82%. The same study reports that Skeptical@16 consistently leads Random@16, LongCoT@16, RejSamp@16, and Reflexion@16 by 2–5 points, and that removing skepticism or replacing the squared-incentive reward with binary reward degrades both Pass@1 and multi-attempt performance [2604.00790].

BanglaForge reports a Pass@1 of 84.00% on the BLP-2025 Bangla Code Generation benchmark. In its appendix ablation, the full pipeline records 95.5% Pass@1 on the development setting, whereas removing the feedback loop lowers Pass@1 to 69.8%, and removing the Reviewer lowers it to 90.4% [2512.19122].

At the level of error analysis, the gains are uneven. Unlocking LLM Code Correction with Iterative Feedback Loops defines `ISR@k` and `MIS` and shows that reasoning models improve substantially more than non-reasoning models. It also reports per-error-type fix rates aggregated across languages: `Compile Error` 88.2%, `Runtime Error` 60.9%, `Wrong Answer` 34.4%, `Time Limit Exceeded` 20.0%, and `Memory Limit Exceeded` 25.0%. The same study states that syntactic and runtime errors are far more tractable than logical or algorithmic failures, and that a simple optimization prompt reduces `Time-Limit` errors by more than 30% [2606.17514].

Feedback Over Form sharpens this point for 1–3B local models. It reports that self-refinement with execution feedback improves code generation by more than 4 standard deviations on HumanEval and sanitized MBPP, but that the mechanism is narrow: refinement fixes many runtime errors, especially `NameError` and `SyntaxError`, while rarely fixing logic errors such as `AssertionError`. It also finds that, in the tested search space, evolutionary search mostly rediscovers a simple generate–execute–refine loop, with no clearly significant gain from added topology [2604.21950].

## 6. Limitations, debates, and extensions

A recurring limitation is execution cost. PerfCodeGen explicitly notes that measuring runtimes incurs substantial compute cost proportional to $E$ runs, $J$ tests, and the number of candidates; it also notes that memory usage is not measured, and that the method does not yet consider readability or security. The same paper states that results are focused on Python functions, may vary on larger codebases, and remain sensitive to runtime noise and LLM reasoning quality [2412.03578].

Another limitation is that passing visible tests is not equivalent to correctness. RefineRL frames this as a false-positive problem and answers it with skepticism prompts that continue refinement even after all public tests pass. CP-Agent formalizes the issue as false-admission risk, program-level evidence against bad programs, and active-state success hazard, and derives a certificate
$$
C_T(\pi)=\Bigl[\prod_{t=1}^T(1-\bar q_t)-\prod_{t=1}^T(1-\underline h_t)\Bigr]_+
$$
as a lower bound on clean-before-false success under held-out trace calibration and a frozen controller manifest [2604.00790][2605.24693].

A further debate concerns how much the improvement comes from feedback itself rather than from architectural elaboration. Feedback Over Form argues that, at 1–3B scale, execution feedback mattered more than added pipeline complexity, that early stopping is essential because forced extra iterations are net-negative, and that text-only pipeline experiments without execution feedback did not show gains at this scale [2604.21950]. This suggests that the presence of a machine-checkable external signal may matter more than whether the loop is implemented as a single model, a dual-agent pair, or an evolved pipeline.

The paradigm is also expanding beyond conventional code generation. The symbolic planning framework uses verifier-derived natural-language corrections to improve feasibility and correctness in long-horizon planning. PIVOT turns plan refinement into trajectory optimization with a monotonic acceptance rule that preserves non-decreasing solution quality. SRDrone couples continuous state evaluation with hierarchical Behavior Tree modification and reports a 44.87% improvement in Success Rate over baseline methods, while real-world deployment reaches a 96.25% Success Rate [2606.27757][2605.11225][2508.15501].

Finally, execution need not always be literal execution. Self-Execution Simulation Improves Coding Models trains LLMs to simulate execution in text, uses that simulated feedback for self-verification and self-fixing, and reports consistent but modest gains. The same work notes limitations in simulation fidelity, especially on heavy numeric operations, and restricts its scope to single-file competitive-programming tasks [2604.03253].

Taken together, these results define self-refinement with execution feedback not as a single algorithm but as a family of tightly coupled generate–verify–revise procedures. The unifying property is the substitution of external, machine-checkable evidence for purely introspective critique. The main open variables are the granularity of the executed signal, the reliability of the verifier, the cost of repeated interaction, and the extent to which refinement should remain an inference-time scaffold or be internalized through supervised or reinforcement learning.

Source: https://www.emergentmind.com/topics/self-refinement-with-execution-feedback