CodeEvo: Iterative Code Evolution

• CodeEvo is a framework that iteratively synthesizes high-quality instruction-code pairs via interaction-driven collaboration between a Coder and a Reviewer.
• It employs keyword-guided instruction evolution to adapt task complexity and semantic grounding, eliminating the need for manually curated corpora.
• The method improves sample efficiency and pass rates by combining compiler determinism with agentic LLM feedback across diverse code generation benchmarks.

Searching arXiv for papers on “CodeEvo” and closely related usages to ground the article. CodeEvo is a label that appears in multiple, non-identical senses across recent literature on LLM-based code generation, evaluation, and program evolution. In its most explicit form, it denotes an interaction-driven framework for synthesizing code-centric training data through iterative collaboration between a Coder and a Reviewer, with keyword-guided instruction evolution and hybrid feedback that combines compiler determinism with agentic review (Sun et al., 25 Jul 2025). In adjacent usages, the same label is attached to the code-space embodiment of CoEvo, a continual evolutionary framework for symbolic solutions (Guo et al., 2024), and to the two-stage self-evolving system introduced as SelfEvolve (Jiang et al., 2023). The term also appears near benchmark-centric work on repository-level and self-evolving coding evaluation (Li et al., 2024, Zhang et al., 10 Feb 2026). This suggests that “CodeEvo” is best understood as a cluster of evolutionary code methodologies rather than a single canonical architecture.

1. Terminology and referential scope

Recent papers use closely related labels—CodeEvo, CodeEvolve, CoEvo, CoCoEvo, and KCoEvo—for systems that share an evolutionary or iterative view of code generation, code transformation, or evaluation. The most direct use is “CodeEvo: Interaction-Driven Synthesis of Code-centric Data through Hybrid and Iterative Feedback,” which frames CodeEvo as a data-synthesis pipeline for instruction-code pairs (Sun et al., 25 Jul 2025). In “CoEvo: Continual Evolution of Symbolic Solutions Using LLMs,” CodeEvo is the code-space embodiment of a broader continual symbolic-search framework (Guo et al., 2024). In “SelfEvolve: A Code Evolution Framework via LLMs,” the summary explicitly states that CodeEvo is the same framework introduced as SelfEvolve (Jiang et al., 2023).

Usage	Core object	Distinguishing mechanism
CodeEvo (Sun et al., 25 Jul 2025)	Synthetic instruction-code data	Coder/Reviewer interaction with hybrid feedback
CodeEvo as CoEvo instantiation (Guo et al., 2024)	Symbolic code solutions	Knowledge-library-enhanced evolutionary search
CodeEvo as SelfEvolve (Jiang et al., 2023)	Single-task code generation	Self-generated knowledge and interpreter-guided self-debugging

A plausible editorial implication is that the label now functions as an umbrella for methods that reject one-shot code generation in favor of iterative search, reflection, validation, or co-evolution. That broad pattern is reinforced by nearby systems such as CodeEvolve for enterprise optimization (Borra et al., 6 May 2026), CodeEvolve for algorithm discovery (Assumpção et al., 15 Oct 2025), CoCoEvo for joint program-and-test evolution (Li et al., 15 Feb 2025), and KCoEvo for API migration via knowledge graphs (Kang et al., 8 Mar 2026).

2. Interaction-driven synthesis architecture

In the 2025 formulation, CodeEvo addresses the problem of acquiring high-quality instruction-code pairs without depending on manually curated corpora, predefined heuristics, or existing code references (Sun et al., 25 Jul 2025). The framework is explicitly interaction-driven and reference-free: it begins from seed instructions and uses two LLM agents with differentiated roles. The Coder generates candidate solutions and test cases; the Reviewer evaluates those candidates, provides feedback, and generates new instructions.

A central mechanism is keyword-guided instruction generation. For a seed instruction $s$ with keyword set $T$, the Reviewer generates a new instruction $s'$ conditioned on a selected subset $t \subseteq T$:

$s' = \operatorname{Reviewer}(s, t)$

The stated purpose is semantic grounding and controllable difficulty. Keywords can be subsetted to increase complexity or, when needed, to simplify a task that proved too difficult.

Candidate generation and validation are then mediated through a hybrid feedback loop. The Coder produces candidate code, tests, and compiler feedback:

$(c, g, f_{\text{comp}}) = \operatorname{Coder}(s')$

The Reviewer reads the instruction, code, execution outputs, and error messages, and combines the compiler-side evidence with its own natural-language assessment:

$$f_{\text{hybrid}} = \operatorname{Reviewer}(f_{\text{comp}}, f_{\mathrm{NL}})$$

The framework emphasizes that compiler feedback alone is insufficient because tests may be incomplete and passing tests do not guarantee semantic correctness, whereas LLM judgment alone is non-deterministic. CodeEvo therefore uses compiler determinism and reviewer reasoning jointly as an automatic quality-control mechanism (Sun et al., 25 Jul 2025).

The per-seed lifecycle is adaptive rather than linear. If the hybrid feedback is valid, the instruction-code pair is retained and the Reviewer generates a new related instruction. If the feedback is invalid, the task is simplified and retried. This implements a trajectory in which difficulty can either escalate or contract depending on the current solvability frontier. The paper’s interpretation is collaborative-programming-inspired: synthesis emerges through iterative interactions rather than one-shot prompting.

3. Data construction, training setup, and empirical behavior

The CodeEvo synthesis pipeline is instantiated with about 5K seed instructions curated from LeetCode, Codeforces, Codewars, GeeksforGeeks, CodeChef, and math/code datasets such as GSM8K and MBPP (Sun et al., 25 Jul 2025). Each seed has an instruction and, on average, about 3 keywords. The main synthesis backbones are Qwen2.5-Coder-32B-Instruct as Coder and Qwen2.5-32B-Instruct as Reviewer. The resulting corpus contains about 17K synthesized samples.

The downstream evaluation studies full fine-tuning of InternLM3-8B-Instruct, StarCoder2-7B, DeepSeek-Coder-6.7B-Instruct, and Qwen2.5-Coder-7B-Instruct on CodeEvo data, with comparisons against zero-shot baselines, Evol-Instruct reproduced with about 25K samples, and OSS-Instruct with 75K samples (Sun et al., 25 Jul 2025). The reported metric is primarily pass@1, on HumanEval, HumanEval+, MBPP, MBPP+, BigCodeBench-Full, BigCodeBench-Hard, and LiveCodeBench v6.

The main empirical claim is sample efficiency with improved functional correctness. For InternLM3-8B-Instruct, CodeEvo reaches 82.3 / 78.0 on HumanEval/HumanEval+, 81.2 / 71.4 on MBPP/MBPP+, 34.9 / 43.2 on BigCodeBench, 15.54 / 15.50 on BigCodeBench-Hard, and 17.1 on LiveCodeBench, exceeding both Evol-Instruct and OSS-Instruct despite using fewer synthetic samples (Sun et al., 25 Jul 2025). For Qwen2.5-Coder-7B-Instruct, CodeEvo yields 85.3 / 79.9 on HumanEval/HumanEval+, 81.2 / 68.5 on MBPP/MBPP+, 41.9 / 52.2 on BigCodeBench, 17.6 / 26.4 on BigCodeBench-Hard, and 22.3 on LiveCodeBench (Sun et al., 25 Jul 2025).

The ablation that removes seed solutions reports little or no performance drop, which the authors interpret as evidence that the synthesized data is itself strong enough to train models (Sun et al., 25 Jul 2025). The analysis sections further report that CodeEvo has the lowest instruction similarity in an embedding-based diversity study, that human raters judge CodeEvo instructions as more difficult than Evol-Instruct variants, and that performance improves as larger subsets of CodeEvo data are used (Sun et al., 25 Jul 2025). The data survival rate declines over rounds, which the paper treats as a consequence of stricter filtering on progressively harder tasks.

4. CodeEvo as continual or self-evolving code search

A second usage of CodeEvo arises in the CoEvo framework, where the code domain is treated as one symbolic space among natural language, mathematical formulas, and logic expressions (Guo et al., 2024). In that formulation, CodeEvo is the executable, code-oriented instantiation of CoEvo, whose broader goal is continual discovery of symbolic solutions with LLMs. The framework combines three components: a versatile representation of solutions, a tree-based idea search process, and a knowledge library-enhanced evolutionary process. The search begins with $N_0$ initial ideas and expands level by level, while the evolutionary stage uses initialization, crossover, mutation, and population management. Two explicit crossover operators—positive-crossover and negative-crossover—and two mutation operators—positive-mutation and negative-mutation—balance exploitation and exploration (Guo et al., 2024).

The knowledge library is the distinctive mechanism. It performs idea summarization, idea management, and idea reuse. Improved solutions are summarized by the LLM, clustered by cosine similarity of sentence embeddings, and later reused either through Random Reuse or Similarity-based Reuse. The paper evaluates CoEvo on four AI Feynman-style problems—Oscillation 1, Oscillation 2, E. coli growth, and Stress-Strain—with gpt-3.5-turbo and gpt-4o-mini backbones, under a budget of up to 2,000 LLM queries and using NMSE as the main metric (Guo et al., 2024). CoEvo is reported to outperform LLM-SR across all tested benchmarks, with GPT-3.5 achieving $4.32e-9$ ID and $8.71e-5$ OOD on Oscillation 1, $T$0 ID and $T$1 OOD on Oscillation 2, $T$2 ID and $T$3 OOD on E. coli growth, and $T$4 ID and $T$5 OOD on Stress-Strain (Guo et al., 2024). The paper also identifies knowledge pollution as a failure mode, where misleading library entries degrade later search.

A third usage is the self-evolving framework introduced as SelfEvolve, where the summary explicitly equates CodeEvo with that system (Jiang et al., 2023). SelfEvolve has two stages. First, the LLM generates task-specific knowledge $T$6 and then draft code $T$7 conditioned on that knowledge:

$T$8

Second, the system executes the draft with authentic example cases from the prompt and iteratively revises the code using interpreter error feedback:

$T$9

The paper evaluates the method on DS-1000, HumanEval, and TransCoder, using ChatGPT / gpt-3.5-turbo as the main backbone and additional scalability tests with GPT-4 (Jiang et al., 2023). Reported gains include 57.10 overall Pass@1 on DS-1000 versus 53.00 for Self-Debugging and 49.30 for ChatGPT, 78.05 / 93.29 Pass@1/Pass@10 on HumanEval, and 94.8 computational accuracy with 92.4 Pass@1 on TransCoder (Jiang et al., 2023). In this usage, “evolution” denotes knowledge discovery, draft synthesis, execution, reflective correction, and iteration until stabilization.

5. Benchmark-centered usages and evaluation frameworks

The CodeEvo label also appears in benchmark-centered contexts, where the object of study is not data synthesis or continual search but evaluation itself. One summary identifies CodeEvo with EvoCodeBench, a repository-level code generation benchmark grounded in real-world open-source Python repositories (Li et al., 2024). EvoCodeBench is constructed through an automatic pipeline with repository selection, function parsing, tests construction, deduplication, and requirement annotation. The first release, EvoCodeBench-2403, contains 275 samples from 25 real-world repositories, with 27% standalone, 73% non-standalone, 3 dependency types, and 3.46 average dependencies per program (Li et al., 2024). Each sample includes a function signature, natural-language requirement, repository context, reference code, reference dependencies, and test cases.

The benchmark formalizes two metrics. Pass@k evaluates functional correctness, and Recall@k evaluates dependency recovery:

$s'$0

$s'$1

The paper reports that gpt-4-turbo-1106 reaches 80% Pass@1 on HumanEval but only 20.73% Pass@1 in the best EvoCodeBench-2403 setting, and that local-file context can raise gpt-4’s Pass@1 from 7.27 without context to 20.73 with local-file infilling (Li et al., 2024). The failure analysis of 50 gpt-4 failures attributes 29 cases to implementation logic errors, 20 cases to missing necessary context, and 1 case to a vague requirement (Li et al., 2024).

A later benchmark, also titled EvoCodeBench, evaluates self-evolving LLM-driven coding systems on LeetCode-style tasks across Python3, C++, Java, Go, and Kotlin (Zhang et al., 10 Feb 2026). It distinguishes one-shot coding agents from self-evolving agents that may revise code for up to 3 reflection/revision rounds. The benchmark records PR, TLE, MLE, CE, RE, WA, TO, RpE, AR, AM, APC, ARB, and AMB, and emphasizes human-relative comparisons through runtime and memory beat statistics (Zhang et al., 10 Feb 2026). The reported finding is that self-evolution improves pass rates by about 10% to 27%, with runtime improvements across all languages, while exposing long-tail language instability in Kotlin (Zhang et al., 10 Feb 2026). In these benchmark-centered usages, CodeEvo denotes a problem setting in which inference-time adaptation itself becomes measurable.

6. Relation to the broader evolutionary coding literature

The broader literature situates CodeEvo within a rapidly expanding space of evolutionary coding systems. CodeEvolve for enterprise optimization extends OpenEvolve with runtime-guided target selection using Java Flight Recorder (JFR), Monte Carlo Tree Search (MCTS), and language-specific evaluation cascades for Java and Salesforce Apex (Borra et al., 6 May 2026). It defines a static Code Component Graph $s'$2, annotates each component with runtime weights $s'$3, and selects targets by thresholding cumulative execution time or call frequency. On a large enterprise Java codebase, it reports an average speedup of $s'$4 across seven hotspot functions, versus $s'$5 for the single-pass LLM bottleneck detector, and in an Apex ablation the final MCTS-augmented configuration produces $s'$6 valid programs out of 20 (Borra et al., 6 May 2026).

A separate CodeEvolve system for algorithm discovery and optimization combines LLMs with an island-based genetic algorithm, inspiration-based crossover, and meta-prompting (Assumpção et al., 15 Oct 2025). It uses 5 parallel islands, migration every 40 epochs, top 10% migration, a Gemini 2.5 FLASH/PRO ensemble with 80/20 sampling, and evaluates on a subset of AlphaEvolve-style mathematical benchmarks. The paper reports that CodeEvolve beats AlphaEvolve on 5 of 6 benchmark instances and matches it on the remaining one (Assumpção et al., 15 Oct 2025).

Other neighboring systems sharpen specific subproblems. CoCoEvo jointly evolves programs and test cases from natural-language descriptions and function headers, reporting pass@1 values of 49.75, 55.75, 45.00, and 76.25 on four backbone models and showing that test evolution improves over program-only evolution (Li et al., 15 Feb 2025). KCoEvo treats API migration as graph-structured reasoning over static and dynamic API graphs, decomposing migration into evolution-path retrieval and path-informed generation (Kang et al., 8 Mar 2026). AMR-Evol focuses on response distillation rather than direct program search, using Modular Decomposition and Adaptive Response Evolution to refine teacher-generated code labels, with more than +3.0 points on HumanEval-Plus and +1.0 point on MBPP-Plus over comparable baselines (Luo et al., 2024). EvoEval extends benchmark evolution itself, reporting an average performance drop of 39.4% relative to HumanEval when tasks are evolved into more difficult or compositional variants (Xia et al., 2024).

Taken together, these systems indicate a general shift away from static prompt-to-code mapping and toward iterative search over code, prompts, tests, paths, or synthetic data. A plausible implication is that CodeEvo now denotes a methodological orientation: correctness is increasingly enforced by execution, search, structured memory, or benchmark dynamics, and not merely by next-token likelihood.