Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks

Abstract: Would experience designing faster GPU kernels also help close in on a long-standing open mathematical conjecture? LLMs integrated into evolutionary search have recently produced state-of-the-art solutions on optimization tasks, including open mathematical conjectures, GPU kernel design, scientific law discovery, and combinatorial puzzles. To achieve this, prior work applied search scaffolds to one target task at a time, so every new problem is approached from scratch and the experience accumulated during search is discarded once the model finishes its attempt. This leaves the capability of iteratively evolving a solution (e.g., knowing which part to mutate and how, deciding when to backtrack) entirely in the scaffold rather than in the model itself. Whether the model itself could acquire this capability and reuse it across different tasks has been largely unexamined. To address this, we introduce Evolution Fine-Tuning (EFT), a mid-training paradigm that teaches LLMs to evolve solutions across tasks by converting evolutionary search trajectories into supervision. We construct Finch Collection, a 156K-trajectory dataset spanning 10 domains and 371 optimization tasks, and fine-tune open-source LLMs from 2B to 9B parameters. Empirically, EFT confers cross-task generalization: across 22 held-out tasks, our models surpass their base counterparts by 10.22% on average. Furthermore, when paired with test-time RL, our model matches state-of-the-art performance on two circle-packing tasks and outperforms its base-model counterpart on the Erdős minimum-overlap problem. EFT thus serves as a "practice phase" for general-purpose discovery agents that do not solve new problems from scratch.

• The paper introduces Evolution Fine-Tuning (EFT) which internalizes evolutionary discovery by training on 156k trajectories from 371 optimization tasks.
• It fine-tunes open-source LLMs using evolutionary trajectories to achieve cross-domain generalization and notable performance gains, including improvements up to +290% on select tasks.
• Integration with reinforcement learning and cross-domain strategy recombination demonstrates EFT’s potential to democratize advanced optimization across diverse problem domains.

Evolution Fine-Tuning: Internalizing Discovery Across 371 Optimization Tasks

Introduction and Motivation

The paper "Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks" (2606.29082) addresses the persistent bottleneck in evolutionary program synthesis with LLMs: that the capacity to evolve solutions is embedded in the evolutionary scaffold rather than the LLM itself. Existing approaches—whether test-time search or test-time learning—leverage LLMs only as mutation operators or adaptors for individual tasks, discarding accumulated experience after each search. Consequently, models fail to internalize the iterative discovery process and cannot transfer optimization acumen between tasks.

This paper proposes Evolution Fine-Tuning (EFT), reframing evolutionary search trajectories themselves as a scalable, cross-domain supervision source. By assembling a dataset of 156,000 filtered trajectories over 371 real-world optimization problems, the authors fine-tune open-source models (2B–9B parameters) to internalize evolutionary improvement patterns. The resultant models demonstrate enhanced capacity for cross-task generalization, approaching the test-time performance of proprietary frontier-scale models and supporting synergistic integration with offline RL and test-time RL frameworks.

Figure 1: EFT as mid-training: The model, after EFT, exhibits improved discovery on hard combinatorial tasks—adopting diverse cross-domain strategies not accessible to base models.

Evolution Fine-Tuning Paradigm

Dataset Construction

The EFT approach begins by systematically harvesting tasks from disparate scientific, mathematical, engineering, and algorithmic sources, including benchmarks like AlphaEvolve, ALE-Bench, FrontierCS, LLM-SRBench, OpenEvolve, and others (Section 3.1). Each task includes specification, initial solution, and an executable, deterministic evaluator. Trajectories are generated by running an evolutionary search scaffold—OpenEvolve coupled with Qwen3.5-397B-A17B—collecting parent–child transitions, prompts, evaluator feedback, and improvement deltas.

Trajectories are filtered to remove systematic failures and misleading signals. The curation yields a high-quality collection spanning 10 domains and covering both local edits and full solution rewrites, balanced between Python and C++.

Figure 3: The construction pipeline of EFT’s trajectory dataset, showing task curation, evolutionary search, and robust trajectory filtering.

Figure 2: Dataset composition: distribution of tasks, programming languages, mutation strategies, trajectory lengths, and popular program libraries.

Supervised Fine-Tuning from Evolutionary Trajectories

For each transition $$(I, x_{t-1}, \mathcal{H}_{t-1}, x_t, \mathcal{E}(x_t), \mathcal{F}_t)$$, the input to the model includes the task instruction, parent solution, selected history, and feedback artifacts; the target is the teacher-generated child. This direct mapping enables the LLM to ingest the full evolutionary state and learn to generate promising mutations.

Crucially, the main training signal is drawn from improvement trajectories ($\Delta>0$), ensuring the model learns trajectories that yield concrete progress. The authors further show that contrastive preference learning using both improved and regressed trajectories via KTO (prospective-theoretic optimization) enhances fine-tuning, providing the LLM with discriminative capability to self-judge promising versus failing solution paths.

Empirical Evaluation

EFT-trained models are benchmarked on a suite of 22 held-out, heterogeneous optimization problems spanning mathematical discovery, algorithm engineering, system performance, and competitive programming. The primary evaluation protocol runs models as mutation operators within evolutionary search scaffolds, measuring maximum task-specific performance within a fixed iteration budget.

Numerical Results and Robustness

EFT yields consistent cross-task gains across all tested model scales. For 9B-parameter models, the average improvement over base models is 10.22%, with individual tasks exhibiting strong numerical gains (e.g., +290% on massive-archival tasks and +74% on a system transaction task). Notably, EFT-equipped 4B models rival or outperform base 8B models, showing the approach's data efficiency and scalability.

In NP-hard competitive programming tasks, gains are larger still: the EFT-9B model achieves an average score of 46.01, vastly exceeding the base 9B's 32.46. Improvements also extend to algorithmic heuristics—models display emergent transfer, recombining optimization patterns across domains and solution modalities.

Figure 4: Competitive programming and algorithmic heuristics: EFT models outperform base models, especially at larger scales and on transfer-heavy tasks.

Synergy with Offline and Test-Time Reinforcement Learning

Applying KTO-based preference learning atop EFT further boosts performance, enabling models (even at 8B scale) to surpass best-human scores in symbolic mathematics (AC1, AC2 tasks). EFT also acts as a mid-training phase for test-time RL frameworks (e.g., nanodiscover), yielding quality improvements over base RL adaptation alone, especially in combinatorial geometry and packing problems.

Scaling Behavior

Performance scales positively with the size of the training task set, demonstrating that more diverse experience consistently improves generalization. This scaling persists regardless of task group, model size, or downstream evaluation scaffold.

Figure 5: Performance scaling with increasing training task diversity, evaluated on mathematically and computationally distinct benchmarks.

Theoretical and Practical Implications

The clear empirical improvements validate the paper's central claim: iterative discovery capability can be internalized into LLMs through supervision from evolutionary trajectories. This shifts capability away from brittle, externally engineered scaffolds to the model itself, with key consequences:

• Model Autonomization: Smaller, public models equipped via EFT approach the utility of proprietary frontier LLMs, obviating dependence on external closed systems for optimization/synthesis.
• Transfer and Compositionality: EFT models manifest strategy recombination—combining, for example, numerical optimization and recommender-system heuristics to solve unseen contest problems—highlighting the emergence of distributed, cross-domain problem-solving skills.
• Synergy with RL: Pretraining on evolutionary trajectories enables more robust and rapid specialization during test-time RL, raising the ceiling for open-ended search and self-improvement without per-task catastrophic forgetting.

Theoretically, the paradigm supports the hypothesis that evolutionary learning signal (rather than static supervised data) constitutes an efficient, scalable route to robust abstraction and generalization in algorithmic domains.

Limitations and Future Directions

The authors acknowledge scope for improved cross-scaffold generalization by incorporating trajectories from multiple evolutionary frameworks (beyond OpenEvolve). Moreover, validation of EFT’s synergy with RL on practical engineering tasks (e.g., GPU kernel optimization) remains an open avenue. Finally, extension to multimodal and multi-turn optimization is critical—many scientific discovery workflows require joint reasoning over code, text, and experimental data, potentially addressable via techniques from recent multi-modal LLM distillation.

Conclusion

Evolution Fine-Tuning demonstrates that the meta-discovery process—iteratively mutating, evaluating, and improving solutions—can be distilled into LLM weights by leveraging large-scale evolutionary trajectories. This enables open-source models to close the performance gap with proprietary frontier LLMs across a wide array of challenging optimization tasks, and facilitates more efficient, domain-transferable scientific and engineering problem solving. Future developments may extend these ideas to cross-modal domains and more autonomous, long-horizon agentic optimization.

---