From Failing to Passing: Evolving Natural Language Prompt Optimization Rules for LLM Code Generation

Abstract: LLMs are known to be sensitive to prompt formulation. Even minor variations in wording can substantially degrade performance. This sensitivity reveals an opportunity: if prompt phrasing can harm performance, can it be used to improve it? To investigate this question, we introduce a search-based approach that identifies and evolves a set of natural language transformation rules with strong downstream effects on coding performance. We then propose DUALFIX, a staged repair pipeline that combines the evolved transformation rules with execution-feedback repair, addressing both specification-level and implementation-level failures. A key strength of our approach lies in its generality: the evolved rules are error-agnostic, reusable across problems, and transferable across models. We evaluate DUALFIX against execution-feedback repair baselines across three models on two challenging benchmarks, LiveCodeBench and APPS. Our results show that the evolved transformations fix from 10-30% of failing cases, including 12-17% of failures that execution-based repair alone cannot resolve. Overall, DualFix recovers up to 30% of baseline failures and fixes 3-5 times more failing cases than Self-Fix across all evaluated settings. Furthermore, we also show that rules evolved on one model transfer zero-shot to other models, outperforming execution-feedback repair without any re-optimization.

• The paper introduces DualFix, a dual-stage pipeline that integrates error- and rule-based prompt optimizations to recover failing LLM code generation cases.
• It employs a genetic algorithm (RuleEvol) to evolve natural language transformation rules that resolve specification ambiguities in problem statements.
• Empirical results show DualFix achieves up to 30% recovery on LCB and 21.3% on APPS benchmarks, delivering additive improvements over single-channel repairs.

Evolving Prompt Optimization Rules for LLM Code Generation: A Summary of DualFix

Motivation and Problem Statement

LLMs exhibit high sensitivity to prompt phrasing, resulting in significant variance in downstream code generation quality—even minor linguistic perturbations can yield divergent execution outcomes. While much research into LLM-based code generation has focused on correcting implementation-level errors through execution-feedback mechanisms, there has been minimal systematic exploration of prompt formulation as a lever for post-hoc repair. This work proposes that specification-level failures—arising from ambiguous, underspecified, or unintuitive task descriptions—represent a fundamentally distinct and underexplored category of code generation failures. Consequently, this paper introduces DualFix (2607.05121), a staged repair pipeline that optimizes failure recovery by integrating both error-based and rule-based task specification rewrites.

DualFix: Dual-Channel Prompt Repair

Architecture and Workflow

DualFix is structured as a two-stage process. Upon a code generation failure (i.e., failing test cases), it first attempts error-based repair by leveraging execution feedback: the error message and failing test case are provided as contextual signal to an external LLM tasked with rewriting the original problem statement. If this approach does not rectify the issue, DualFix reverts to a rule-based strategy, rewriting the specification using a fixed set of natural language transformation rules evolved offline by a genetic search.

Figure 1: The DualFix inference pipeline showing prioritized error-based repair followed by rule-based rewriting; only a single failing test is necessary as feedback.

RuleEvol: Genetic Search for Natural Language Transformation Rules

The rule-based channel is centered on RuleEvol, a metaheuristic evolutionary optimization that iteratively discovers, mutates, and selects sets of IF–THEN–EXCEPT rules. These rules identify and transform surface-level features of task text (e.g., mathematical symbols, ambiguous terminology, function signatures), aiming to maximize the pass rate on downstream code generation when applied as a pre-processing filter to problem specifications. The search employs three mutation operators—reflection (LLM-guided editing based on categorized feedback), deletion (dropping potentially harmful rules), and merge (combining complementary rule sets). Population diversity is preserved by Pareto-based parent selection targeting unique problem fixes across generations. The resulting rule sets are compact, semantically preserving, and error-agnostic.

Figure 2: RuleEvol’s genetic optimization loop, evolving rule sets through mutation, evaluation, and selection under a Pareto frontier.

Empirical Evaluation

Benchmarks and Experimentation

The study evaluates DualFix on LiveCodeBench (LCB) and APPS, covering diverse competitive programming tasks. Multiple LLMs are used as generators (Qwen2.5-Coder-7B-Instruct, Codestral-22B-v0.1, Claude Haiku 4.5). The central metric is the acceptance rate: the percentage of previously failing cases that are rendered passing by each intervention.

Performance Results

Across all model-benchmark settings, DualFix outperforms state-of-the-art execution-feedback baselines (Self-Fix, error-based rewriting). On Codestral-22B, DualFix increases recovery from failing cases by up to 30% on LCB and 21.3% on APPS—representing a 3–5× improvement over error feedback alone. Rule-based specification rewriting alone achieves 11.9–21.6% recovery on LCB and 8.2–13.8% on APPS, often matching or exceeding iterative error-based repair, despite having no access to execution signal.

Critically, ablation and overlap analyses reveal that error-based and rule-based repair channels address largely disjoint classes of failures. Rule-based rewriting often resolves specification-level ambiguities that error feedback cannot, while error-based repair addresses implementation-level misunderstandings. As a result, DualFix provides strictly additive improvements by pairing the two orthogonal approaches.

Figure 3: Overlap analysis of failure sets fixed by different repair approaches, confirming that rule- and error-based rewriting are largely complementary.

Rule Evolution Dynamics

The convergence of RuleEvol is analyzed by tracking validation accuracy across evolutionary generations. The optimization exhibits rapid initial improvement before plateauing, suggesting efficient adaptation to core specification ambiguities but also indicating diminishing returns with the current mutation space.

Figure 4: Validation accuracy of the best-evolved rule set across generations for both benchmarks and generators.

Rule Nature and Transferability

The cross-model and cross-benchmark transferability of the evolved rules is a central claim of the paper. Rules discovered for one generator (e.g., Codestral-22B) zero-shot transfer to other models (e.g., Claude Haiku 4.5), yielding 36.1–44.4% recovery on LCB and 27.8–36.1% on APPS—consistently surpassing execution-feedback repair baselines, without retraining or tuning. Qualitative analysis indicates a set of universal rules (clarification of algorithmic terminology, translation of mathematical symbols to plain English, disambiguation of binary predicates, function signature neutralization), with some rules emerging as benchmark- or model-specific. The universal subset consistently transfers to new LLMs and problem distributions.

Theoretical and Practical Implications

The findings reconceptualize code generation failures in LLMs as a dual-level problem: not all erroneous outputs result from model misimplementation, but are in many cases induced by the structure, notation, or ambiguity in the input specification. By evolving and applying generic, reusable prompt transformation rules, it is possible to systematically “repair” the specification side of the interface, statistically improving overall system correctness without retraining the underlying model or introducing additional manual curation. This has practical implications for integrating such staged pipelines as robust post-processing layers in LLM-based toolchains for automated programming, as well as for automated curriculum curation and instruction generation.

Furthermore, the rule-evolution paradigm can theoretically extend to other LLM-driven domains where semantic precision of natural language input (e.g., mathematical problem solving, data science, procedural task planning) is an essential driver of downstream model performance. The demonstration of cross-model transferability positions the approach as a low-cost, widely reusable solution.

Future Directions

Potential future research avenues include:

• Universal Preprocessing: Deploying evolved rule sets as a default preprocessing step for all prompts, not merely failing cases, with the aim to preemptively prevent failures.
• Rule Diversification: Expanding the mutation space and leveraging model ensemble feedback to evolve more expressive or context-sensitive rewriting strategies.
• Failure Characterization: Systematic study of the majority of failures unreached by either channel (66–83% of cases) to identify specification patterns or capability boundaries currently outside the reach of specification- or error-level repair.
• Formal Guarantees: Investigating properties of rule sets related to semantic invariance and alignment with formal specification standards.

Conclusion

This paper establishes that systematic prompt optimization via evolutionary rule search meaningfully augments LLM code generation, with the key empirical finding that rule-based and error-based repair channels are complementary and largely non-overlapping. The transferability of compact, error-agnostic transformation rules underlines the generality and scalability of the approach. These insights open new directions for prompt engineering—suggesting that improving specification clarity, not just code quality, is crucial for realizing the full potential of LLM-driven program synthesis (2607.05121).