optimize_anything: A Universal API for Optimizing any Text Parameter

Abstract: Can a single LLM-based optimization system match specialized tools across fundamentally different domains? We show that when optimization problems are formulated as improving a text artifact evaluated by a scoring function, a single AI-based optimization system-supporting single-task search, multi-task search with cross-problem transfer, and generalization to unseen inputs-achieves state-of-the-art results across six diverse tasks. Our system discovers agent architectures that nearly triple Gemini Flash's ARC-AGI accuracy (32.5% to 89.5%), finds scheduling algorithms that cut cloud costs by 40%, generates CUDA kernels where 87% match or beat PyTorch, and outperforms AlphaEvolve's reported circle packing solution (n=26). Ablations across three domains reveal that actionable side information yields faster convergence and substantially higher final scores than score-only feedback, and that multi-task search outperforms independent optimization given equivalent per-problem budget through cross-task transfer, with benefits scaling with the number of related tasks. Together, we show for the first time that text optimization with LLM-based search is a general-purpose problem-solving paradigm, unifying tasks traditionally requiring domain-specific algorithms under a single framework. We open-source optimize_anything with support for multiple backends as part of the GEPA project at https://github.com/gepa-ai/gepa .

• The paper introduces a unified LLM-based evolutionary search framework that optimizes any text parameter across diverse tasks.
• It employs structured side information and Pareto-based candidate selection to accelerate convergence and enhance multi-task performance.
• Empirical results show significant improvements in domains like scheduling, kernel generation, and agent design, highlighting its broad applicability.

Universal LLM-based Text Optimization: The optimize_anything Framework

Motivation and Context

The "optimize_anything: A Universal API for Optimizing any Text Parameter" (2605.19633) paper establishes a unified, domain-agnostic optimization paradigm for any artifact representable as text, leveraging LLM-driven evolutionary search. Prior frameworks, such as AlphaEvolve, GEPA, and FunSearch, achieved strong results within their respective domains—program synthesis, prompt engineering, or code generation—but none demonstrated state-of-the-art performance across claims as diverse as agent design, CUDA kernel optimization, scheduling, and image/CAD generation with a single interface. The work posits that any problem expressible as maximizing an evaluator $f$ over string artifacts $x$ can be handled by LLM-augmented optimization, embedding diagnostic "side information" (SI) into the search loop. This reframes LLMs not only as generative agents but as universal problem solvers via guided text artifact evolution.

System Design and Optimization Modes

The core insight is the reduction of broad optimization problems—ranging from code, policies, and prompts to numeric configurations and even image representations—into a common API contract:

• The user provides a seed artifact (string), an evaluator returning a score and SI, and optionally datasets for multi-task/generalization.
• The framework orchestrates the search, selection, and candidate proposal, decoupled from hand-crafted mutation templates or domain-specific “evolve blocks” common in prior art.

Crucially, SI is treated as a first-class channel: the evaluator emits not just scalars but failure diagnostics (e.g., compiler errors, reasoning traces, visualizations), enabling the LLM proposer to make semantically informed refinements akin to engineer iteration rather than black-box numeric tweaking.

The system supports three distinct search modes under a single API:

• Single-task: Directly optimizes an artifact for a single target metric.
• Multi-task: Joint optimization across batches of related tasks; the Pareto frontier is shared, allowing cross-transfer of optimization patterns. This is uniquely operational in optimize_anything and is critical for domains such as kernel generation, where architectural motifs generalize across instances.
• Generalization: Seeks artifacts that validate robustly on held-out data, extending beyond prompt optimization to agent architectures and scheduling policies.

Key Algorithmic Innovations

The optimization workflow is fundamentally evolutionary, with explicit architectural advances:

• Pareto-based Candidate Selection: Instead of collapsing feedback into aggregate values, the system tracks per-task/subscore metrics over candidates and preserves any artifact excelling in at least one dimension. This maintains a diverse “frontline” of specialist solutions and prevents premature convergence.
• Structured Reflection with SI: LLMs contextualize failures (not just scores) during dedicated “reflection” phases, using SI to target improvements. For multi-module artifacts (e.g., agent + refiner prompt), Pareto leapfrogging enables mutual advancement.
• Seedless and Flexible API: For tasks where even a poor initial seed is infeasible (e.g., 3D modeling), natural language objectives suffice, with LLMs bootstrapping candidate generation.
• Adapter Layer and Backend Agnosticism: The architecture is modular with respect to the optimization backend. GEPA’s reflective mutation/Pareto selection is leveraged by default but can be substituted as externally improved algorithms emerge.

Empirical Results and Claims

The paper reports strong, quantifiable improvements across six diverse domains, validated with cross-framework ablations and matched-budget reruns:

Domain	Optimization Mode	LLM	Key Result
Agent Skills (Bleve repo)	Generalization	Claude Opus	Pass rate: 79.3% → 98.3% (Haiku); transferable skills
Cloud Scheduling	Generalization	Gemini 3 Pro	Cost savings: up to 40.2% (CloudCast benchmark)
ARC-AGI (agent architecture)	Generalization	Gemini Flash	Test accuracy: 32.5% → 89.5%; 4-stage pipeline learned
AIME Prompt Optimization	Generalization	GPT-4.1 mini	Test: 46.7% → 60.0%, +13.3pp over baseline
CUDA Kernel Generation	Multi-task	GPT-5	87% match/beat PyTorch baseline; 25% are 20% + faster
Circle Packing (n=26)	Single-task	GPT-5.1	Sum radii: 2.6360 (beats AlphaEvolve/OpenEvolve)

Ablation studies demonstrate:

• Inclusion of SI (vs. score-only feedback) yields 4–6x faster convergence and large improvements in final quality; for example, mean kernel speedups of 4.11× (with SI) vs. 1.15× (score only).
• Multi-task mode consistently outperforms per-task independent optimization for related problems, confirming the benefit of cross-task pattern transfer via the shared Pareto set.
• Proposer LLM quality directly impacts performance/cost tradeoffs, but even cost-optimized LLMs outperform strong hand-tuned baselines.

The qualitative analysis reveals the system's ability to auto-discover nontrivial algorithmic patterns such as break-even resource allocation in scheduling, verify-then-fallback meta-controllers in agents, and hybridized mathematical solvers for circle packing. Importantly, optimize_anything correctly identifies tasks where multi-task search is counterproductive (e.g., independent circle packing instances), highlighting the nuanced understanding of when cross-task transfer is beneficial.

Implications and Theoretical Ramifications

The results provide concrete evidence for the thesis that LLM-based evolutionary search, when equipped with structured SI and Pareto-based exploration, forms a general-purpose paradigm for programmatic optimization. In effect, this collapses problem-specific optimizer design (i.e., prompt engineers for LLMs, kernel hackers for GPU code, or combinatorialists for algorithmic puzzles) into data/model selection and SI contract design.

Practically, this framework democratizes optimization for any text-serializable domain, abstracting away meta-model selection, algorithmic tuning, and low-level mutation engineering from end-users. The declarative API—comprising artifact, evaluator, and optionally a dataset—suggests future systems where domain experts only specify goals and diagnostic knowledge, delegating the optimization loop to universal LLMs.

Theoretically, the work situates SI as an analog to gradients in continuous optimization, but richer: SI can subsume structured data, textual explanations, and multimodal feedback, enabling LLM-driven evolution to traverse problem landscapes inaccessible to classical numerical search.

Future Directions

Several open questions and promising research trajectories follow:

• Model-based Search: As LLMs and VLMs advance, the coupling of text-driven optimization with perception and control could further unify disparate AI tasks (e.g., hardware, robotics, design synthesis).
• SI Automation: Automating the generation or extraction of high-value SI from evaluators could reduce the remaining domain expertise bottleneck. Learning “what to explain” to the LLM proposer is a critical meta-optimization target.
• Artifact Modality Expansion: Extending the abstraction to non-text artifacts (binaries, graph structures) via serialization and evaluator-provided SI translation.
• Resource-efficient Optimization: As evaluation cost remains a constraint for complex domains, further work is needed in budget-aware candidate prioritization, model distillation for proposal steps, and meta-learning optimal SI extraction.
• Back-end Diversity: Integration of alternative evolutionary or hybrid optimizers and real-time selection/adaptation of backends in response to domain idiosyncrasies.

Conclusion

"optimize_anything" (2605.19633) demonstrates, with robust quantitative and qualitative evidence, that universal LLM-based text optimization is a viable, high-performance alternative to domain-specific optimizers for a broad family of tasks. The system’s advances—particularly the SI-based reflection mechanism, unified multi-mode optimization, and API simplicity—eliminate the need for handcrafted evolutionary infrastructure per task. This constitutes a substantial step toward general-purpose, explainably guided artifact evolution, with deep implications for both the theory and practice of automated program and system design.