Open-Ended Optimization (OEO)

• Open-Ended Optimization (OEO) is a paradigm in which systems generate, modify, evaluate, and reorganize tasks, representations, objectives, environments, or search procedures rather than optimizing only a fixed solution.
• OEO applies mechanisms such as quality-diversity archives, evolving environments, reusable program components, adaptive rubrics, textual feedback, and structured planning to support novelty, transfer, cumulative improvement, and continued discovery.
• Effective OEO requires external constraints, reliable evaluators, persistent archives, checkpointing, safety controls, and long-horizon benchmarks because fixed grammars, finite budgets, reward hacking, and evaluator bias can limit genuine open-endedness.

Open-Ended Optimization (OEO) is an optimization paradigm in which the system does not merely seek a solution to a fixed, fully specified task. Instead, it generates, modifies, evaluates, and sometimes reorganizes the problems, representations, objectives, environments, or solution mechanisms involved in optimization. The defining delegation is procedural: the external framework fixes the objective, permitted interactions, resource budget, data boundary, and evaluation, while the optimizer composes the route from evidence to improvement online (Xue et al., 10 Aug 2026). OEO therefore encompasses task–agent coevolution, quality-diversity search, combinatorial program generation, inference-time artifact optimization, adaptive evaluation, and self-evolving reasoning systems. Its strongest forms seek continued production of novel, useful, diverse, transferable, and increasingly complex solutions; weaker forms optimize an evolving representation or objective within a bounded domain.

1. Conceptual foundations and scope

OEO differs from conventional directed optimization, commonly represented as maximizing a fixed objective over a fixed parameter space. In ordinary optimization, a designer specifies an objective, the algorithm updates a parameter vector or candidate solution, and progress is measured against a relatively stable task criterion. OEO instead treats the optimization process itself as partially endogenous. The system may generate new environments, discover new programs, alter its outline or search strategy, construct new evaluation criteria, or preserve multiple alternative solution lineages.

The term open-ended does not imply the absence of constraints. In the formal account of OEO proposed in “Rethinking Self-Evolving Agents: Do We Still Need Prescribed Optimization Pipelines?”, an optimization contract fixes the objective, initial artifact, target model, permitted operations, resource budget, data boundary, evaluator, sealed final split, and governance mechanisms (Xue et al., 10 Aug 2026). What is delegated is the optimization meta-policy: evidence collection, diagnosis, revision, checkpoint selection, operation scheduling, and termination. OEO therefore concerns online composition of the improvement procedure, not unconstrained optimization.

Several related notions are distinguished:

• Open-ended evolution (OEE): artificial-life research on continual evolutionary innovation, often involving coevolution, ecological interactions, changing niches, and systems not designed to converge to one known optimum.
• Open-ended search: a broader category including evolutionary computation, novelty search, quality-diversity algorithms, gradient-based learning, meta-learning, neural architecture search, reinforcement-learning algorithm discovery, and other mechanisms that generate agents, algorithms, architectures, environments, or artifacts.
• Open-ended AI: the broadest notion, involving continual discovery of new behaviors, algorithms, architectures, or forms of intelligence.
• Open-ended optimization: the optimization-oriented subset of open-ended search, in which solutions, objectives, niches, environments, representations, or evaluation criteria may expand or change over time (Ecoffet et al., 2020).

OEO is not equivalent to merely using a large population, random sampling, or evolutionary operators. A system may maintain many candidates while still optimizing a fixed objective toward convergence. Conversely, a gradient-based or language-model system may qualify as open-ended search if it generates or modifies the process through which future solutions are produced.

The central distinction is between product optimization and process optimization. OEO may optimize not only a candidate artifact but also:

• the task or environment in which the artifact is evaluated;
• the representation used to describe candidates;
• the archive or repertoire that preserves alternatives;
• the evaluator or rubric used to distinguish solutions;
• the search strategy and evidence-collection schedule;
• the composition grammar from which future solutions are generated.

2. Core mechanisms

OEO systems commonly combine several mechanisms rather than relying on a single optimization operator.

Evolving tasks and environments

Paired Open-Ended Trailblazer (POET) maintains multiple environment–agent pairs. An environment is a parameterized function $E(\cdot)$, and an agent is represented by policy parameters $\theta$. A paired individual is:

$(E(\cdot),\theta).$

The active population contains multiple such pairs, while an archive retains previously admitted environment encodings. Environments reproduce when their paired agents reach a reward of at least $200$, below the full success criterion of $230$. Mutated child environments inherit the parent controller and are filtered by a minimal criterion:

$$50\leq E^{\mathrm{child}}(\theta^{\mathrm{child}})\leq 300.$$

Environmental novelty is calculated from encoded obstacle parameters using Euclidean $k$-nearest-neighbor distance, with $k=5$. The resulting system is a dynamically expanding network of challenges and solutions rather than a single human-designed curriculum (Wang et al., 2019).

Population diversity and transfer

OEO preserves alternative lineages because useful stepping stones may arise in environments or niches that are not on the most direct path to a target capability. POET performs all-to-all transfer among active agents. For each target environment, it evaluates source policies and one-step proposals adapted to that target. A source policy may replace the target controller through direct or proposal transfer if it improves target performance (Wang et al., 2019).

Transfer is therefore more than inheritance from parent to child. It permits backward transfer from child to parent and lateral transfer across unrelated lineages. In the reported experiments, disabling transfer significantly reduced environmental coverage and resulted in no extremely challenging environments being solved.

Cumulative combinatorial construction

“Open-Ended Automatic Programming Through Combinatorial Evolution” replaces ordinary genome mutation with the combination of immutable, reusable code blocks. Positively classified blocks are stored in a repository and can later be recombined. Placeholders allow hierarchical insertion rather than simple token concatenation. The observed progression was from keywords and punctuation to variable declarations, classes, methods, and classes containing methods and variables (Fix et al., 2021).

This mechanism provides cumulative memory:

$R_{t+1}=R_t\cup\{b_{\mathrm{new}}\},$

when a new block receives positive structural value. The repository grows as a library of components, creating an expanding set of possible combinations. The reported experiment reached classification values from $1$ to $\theta$0, but used a fixed Java-oriented grammar, regular-expression recognition, and finite computation. It therefore demonstrates bounded combinatorial elaboration rather than proven indefinite open-endedness.

Quality-diversity and learned behavioral spaces

Quality-diversity (QD) methods maintain high-quality solutions across multiple behavioral regions. Standard MAP-Elites partitions a measurement space into cells and retains the best solution in each cell. QDHF extends this framework by learning the diversity representation from human similarity judgments rather than requiring manually specified behavioral descriptors (Ding et al., 2023).

For a candidate $\theta$1, QDHF obtains an embedding:

$\theta$2

where $\theta$3 is a feature extractor and $\theta$4 is a learned projection. Two-Alternative Forced Choice judgments provide triplets in which a human or preference model identifies which of two candidates is more similar to a reference. A triplet hinge loss trains the embedding so that human-similar artifacts are close and dissimilar artifacts are separated.

The learned embedding determines archive cells, while a task-specific quality objective $\theta$5 selects the elite within each cell. QDHF therefore addresses the descriptor-specification problem: the optimizer can preserve diversity along dimensions regarded as meaningful by human judgments without requiring the designer to enumerate those dimensions manually. Its broader open-endedness remains conditional because the quality objective, archive structure, and embedding dimensionality are fixed.

Feedback-guided artifact revision

Feedback Descent optimizes prompts, code, SVG programs, and molecular SMILES strings through repeated candidate generation, pairwise comparison, and textual critique (Lee et al., 11 Nov 2025). The evaluator returns both a preference and a rationale:

$\theta$6

where $\theta$7 identifies the preferred artifact and $\theta$8 is textual feedback. The LLM uses the current champion and feedback history to propose the next artifact.

The binary preference performs selection, whereas the rationale supplies directional information about what should change. The method operates entirely at inference time: model weights are not modified. Its central OEO contribution is to preserve explanatory feedback rather than compressing evaluation into a scalar or a single preference bit. The approach has a single-champion structure and limited explicit diversity pressure, so it represents feedback-guided open-ended artifact improvement rather than full population-based OEO.

Evolving evaluation criteria

SERPO addresses open-ended generation where answer voting is inadequate because multiple responses may be valid and common responses may share omissions. It coevolves three objects: Good–Normal–Bad response archives, query-specific rubrics, and actor-policy parameters (Wang et al., 29 Jul 2026).

The system selects maximally separated Good, Normal, and Bad responses, proposes criteria that distinguish them, scores criteria by variance and Good–Normal–Bad order agreement, removes weak criteria, and converts probabilistic verdict-token likelihoods into rewards. The policy is then updated with GRPO. Because the rubric changes as the policy improves, the reward is nonstationary:

$\theta$9

This makes the evaluation structure itself part of optimization. SERPO’s open-endedness is restricted to a fixed prompt distribution and frozen evaluator roles, but it demonstrates coevolution of responses, criteria, and policy parameters.

3. Representations, archives, and compositional closure

A principal OEO question is what constitutes the object of search. Systems may optimize policies, tasks, programs, reasoning trajectories, outlines, molecular structures, or the generative mechanisms that produce them.

Persistent archives

Archives preserve stepping stones, alternatives, and historical information. In POET, the archive stores previously admitted environment encodings. In combinatorial programming, the repository stores positively classified code blocks. In QDHF, the archive retains one elite per learned behavioral cell. In SERPO, Good–Normal–Bad archives preserve contrasting response strata.

Archive design determines whether an OEO system can retain useful but temporarily inferior candidates. Fixed-size archives, oldest-first deletion, or single-champion elitism reduce computation but may discard valuable stepping stones. Conversely, unrestricted accumulation can create redundancy, memory growth, or archive pollution.

Compositional representations

“A Compositional Framework for Open-ended Intelligence” proposes that open-endedness should be organized around a finite set of representational and algorithmic primitives $(E(\cdot),\theta).$0, together with composition operators $(E(\cdot),\theta).$1 (Momennejad et al., 13 Jun 2026). The compositional closure is:

$(E(\cdot),\theta).$2

A recursive formulation is:

$(E(\cdot),\theta).$3

$(E(\cdot),\theta).$4

$(E(\cdot),\theta).$5

If $(E(\cdot),\theta).$6 is finite and $(E(\cdot),\theta).$7 contains suitable recursive or generative operators that preserve type consistency and permit reuse of intermediate outputs, the closure can contain an unbounded number of formal compositions. The paper emphasizes that infinite cardinality alone is insufficient: useful open-endedness also requires productivity, transfer, verifiability, parsimony, and semantic or functional value.

Primitive Transition Graphs represent solutions as typed graphs whose nodes are primitive instances and whose edges encode sequencing, branching, recursion, verification, or parameter binding. Recurrent subgraphs may be compressed into higher-order primitives. This produces a cumulative mechanism in which the system expands its generative basis while preserving a parsimonious library.

The proposed Next Primitive Prediction objective trains a model to predict the next primitive–operator pair:

$(E(\cdot),\theta).$8

where $(E(\cdot),\theta).$9 represents the current world or task context. This is a conceptual architecture rather than an experimentally established OEO algorithm.

Evolving structured representations

ScaffoldAgent treats a research outline as an evolving rooted tree:

$200$0

Each node contains an intent, attached evidence, and historical utility statistics. The outline simultaneously functions as a search index, evidence memory, report architecture, and stateful optimization object (Yang et al., 18 Jun 2026).

Three structural operations are available:

• Expansion: decomposes an overly broad node into more specific child nodes.
• Contraction: merges redundant or excessively fragmented sibling nodes.
• Revision: updates a node’s intent and evidence without changing its position in the hierarchy.

The utility of an operation combines retrieval utility, structural utility, and generation utility:

$200$1

An UCB-style node-selection rule prioritizes nodes with poor historical utility while exploring rarely visited nodes. Utility is delayed until retrieval and trial generation have taken place. The system therefore optimizes not only report content but also the scaffold through which future information acquisition and generation occur.

Online optimization of the optimization procedure

In OEO as formalized by Rethinking Self-Evolving Agents, the state is:

$200$2

where $200$3 is the current artifact, $200$4 the interaction history, $200$5 committed checkpoints, and $200$6 remaining resources. The optimizer chooses actions such as inspection, target-model rollout, revision, checkpoint selection, and stopping. The runner enforces the contract but does not specify the task-specific route (Xue et al., 10 Aug 2026).

This architecture separates governance from procedural composition. The external framework retains control over permissions, budget, data isolation, and evaluation, while the optimizer chooses how to gather evidence, test hypotheses, revise the artifact, and preserve an earlier state.

4. Optimization objectives, evaluation, and feedback

OEO systems frequently use indirect, learned, relative, or evolving objectives rather than a single immutable scalar.

Direct and indirect incentives

Safe open-ended AI research distinguishes the ideal objective, explicit incentives, and incentives of generated agents (Ecoffet et al., 2020). In OEO, the ideal objective may be useful and safe innovation, while explicit incentives include novelty, survival, environment-solving ability, quality, or market demand. Generated agents or algorithms may develop internal objectives that differ from both.

This creates specification gaps:

• the ideal objective may not be fully formalized;
• explicit incentives may reward a proxy;
• generated agents may optimize an internal objective;
• the evaluator may be exploitable;
• intermediate candidates may be unsafe even when the final artifact appears acceptable.

Consequently, safety must be treated as a property of the entire generative process, including candidate generation, environment generation, archive maintenance, transfer, evaluation, and deployment.

Constrained search and trust regions

Contrastive Decoding illustrates constrained optimization over discrete text. It maximizes the difference between the likelihood assigned by a large expert model and a smaller amateur model, subject to an expert-plausibility constraint (Li et al., 2022):

$200$7

The plausibility set excludes extremely unlikely tokens that could obtain a high likelihood ratio through denominator effects. Beam search performs approximate autoregressive optimization. The method demonstrates a general OEO pattern: a learned evaluator supplies a directional or contrastive objective, while a feasibility constraint prevents objective exploitation.

Self-evaluation and reasoning rewards

Direct Reasoning Optimization (DRO) uses a Reasoning Reflection Reward for open-ended long-form tasks without a generic executable verifier (Xu et al., 16 Jun 2025). The method measures how sensitive reference-answer token likelihoods are to sampled reasoning traces. For reference token $200$8, across-trace sensitivity is:

$200$9

Tokens with high sensitivity receive greater weight. The resulting reward estimates reasoning–outcome consistency rather than generic answer similarity. GRPO compares multiple reasoning traces for the same prompt, while dynamic filtering removes examples that are either too difficult or insufficiently informative.

DRO’s self-reward is reference-grounded rather than unconstrained self-evaluation. Its validity depends on the quality of the reference, the relationship between visible reasoning and actual computation, and the model’s calibration.

Textual feedback as a semantic direction

Feedback Descent formalizes textual critiques as approximate semantic directions in artifact space. If $230$0 maps an artifact into latent semantic features, a critique-induced edit direction $230$1 is useful when it is positively aligned with the local utility gradient:

$230$2

The paper’s convergence analysis applies only to an idealized smooth latent-space process under directional-alignment and regularity assumptions. Textual feedback is not an exact gradient, and arbitrary discrete artifact spaces do not inherit the stated convergence guarantee.

Endogenous market value

The Free-Market Algorithm replaces a global fitness function with distributed supply-and-demand dynamics (Jaraiz, 25 Mar 2026). Agents discover transformations, trade goods, open and close firms, and compete for local demand. Products become inputs to later products, and recipes persist in a growing book.

The effective value of a good depends on demand, supply, input accessibility, production success, decay, and downstream use. The system therefore produces hierarchical pathway networks rather than one best candidate. However, its value remains conditional on domain-specific feasibility rules, demand preferences, scarcity, topology, decay, and firm-management mechanisms. “Emergent fitness” does not mean that the system is free of designed objectives or priors.

Benchmarking the optimization process

InferenceBench evaluates autonomous optimization of LLM inference servers under a two-hour budget, a fixed H100 GPU, a target model, an evaluation harness, and one of four scenarios: prefill latency, decode latency, concurrent-request throughput, and a combined objective (Yeon et al., 20 May 2026).

The benchmark distinguishes:

• domain knowledge from strategy generation;
• best-seen performance from final-shipped performance;
• framework selection from parameter tuning;
• exploration breadth from repeated measurement;
• deployment validity from transient optimization success.

Agents often mentioned relevant techniques but converged on a single inference framework, usually vLLM, and tested few distinct configurations. Matched-budget non-agent search outperformed the best agent in aggregate. This indicates that naming optimization techniques is not equivalent to conducting a broad, disciplined optimization campaign.

5. Empirical applications and evidence

OEO has been demonstrated or investigated across several domains, but the strength of evidence varies.

Robotics and adaptive control

POET’s modified Bipedal Walker domain contains stumps, gaps, stairs, and rough surfaces. The environment population is capped at 20 active environments, and the main experiments ran for up to 25,200 POET iterations using 512 ES samples per update and 256 CPU cores (Wang et al., 2019).

POET solved environments that direct ES and a direct-path curriculum could not reliably solve. Direct optimization often converged to a safety strategy in which the walker moved forward briefly and stopped before obstacles. Across three runs, transfer replacements succeeded in approximately $230$3, $230$4, and $230$5 of attempts. Removing transfer significantly reduced environmental coverage and eliminated solutions to extremely challenging environments.

The results support the role of diversity, stepping stones, and cross-environment transfer. They do not establish indefinite complexity growth because obstacle parameters are bounded and the walker morphology is fixed.

Automatic programming

The combinatorial evolution experiment ran approximately $230$6 iterations and progressed from Java keywords and special characters to structurally recognized classes, methods, and composite class fragments (Fix et al., 2021). Its complexity score assigns values of $230$7 to variable declarations, $230$8 to classes, and $230$9 to methods, with nested values added.

The experiment did not demonstrate executable semantic behavior, external task performance, or indefinite novelty. Regular expressions recognized a narrow Java subset, and the system did not produce a useful main method within a reasonable number of iterations. Its principal contribution is cumulative syntactic construction through an archive of reusable components.

Text generation and reasoning

Contrastive Decoding improved fluency, coherence, and distributional similarity across news, Wikipedia, and story domains compared with reported decoding baselines (Li et al., 2022). It used expert–amateur likelihood contrast, an expert plausibility constraint, and beam search, without additional training.

Marco-o1 uses Chain-of-Thought fine-tuning, MCTS, variable-granularity reasoning actions, and reflection to explore alternative reasoning trajectories (Zhao et al., 2024). Its main experiments improved MGSM accuracy, while translation examples were qualitative. The system lacks a persistent archive, explicit novelty objective, changing tasks, and population-level diversity; it is better characterized as an inference-time search component for OEO than as a complete OEO system.

DRO improved long-form paragraph revision and FinQA performance by using sensitivity-weighted, reference-grounded reasoning rewards (Xu et al., 16 Jun 2025). Its results show that open-ended reasoning can be optimized without an external reward model, but the method remains dependent on reference outcomes and model probabilities.

SERPO improved HealthBench, ResearchQA, and several out-of-distribution benchmarks by coevolving response archives, criteria, and the actor policy (Wang et al., 29 Jul 2026). Its evidence for continued evolution is stronger than that of static self-evaluation methods, but the prompt set and model architecture remain fixed, and the frozen evaluator may impose substantial biases.

Generative design and molecular discovery

QDHF improved quality-diversity search in robotic-arm and maze tasks and increased perceptual diversity in text-to-image generation (Ding et al., 2023). In the image experiment, QDHF improved DreamSim diversity while preserving CLIP alignment, and users generally preferred its image sets or judged them more diverse. The method learns human-relevant diversity metrics rather than optimizing a single average preference.

Feedback Descent optimized SVG designs, prompts, and molecules using textual critiques (Lee et al., 11 Nov 2025). In DOCKSTRING molecular optimization, it exceeded the database’s $$50\leq E^{\mathrm{child}}(\theta^{\mathrm{child}})\leq 300.$$0th percentile on all six targets and exceeded the reference database’s best molecule on several targets. The reported weak or negative correlations between docking score and similarity to approved drugs were interpreted as evidence of exploration into novel chemical regions. These results demonstrate feedback-guided search beyond simple retrieval, but not unbounded molecular innovation.

Chemistry and macroeconomic modeling

The Free-Market Algorithm began from 900 bare atoms and reported discovery of all 12 feasible amino-acid formulas, five nucleobases, the formose sugar chain, and Krebs-cycle intermediates (Jaraiz, 25 Mar 2026). It also generated multiple synthesis pathways and nonmonotonic copy-number distributions related to Assembly Theory.

In macroeconomic forecasting, the same market mechanism used input–output tables and reported a non-crisis MAE of $$50\leq E^{\mathrm{child}}(\theta^{\mathrm{child}})\leq 300.$$1 percentage points, with portability to 33 countries. The chemistry and economic implementations nevertheless rely on substantial domain-specific rules. The reported results support a reusable market mechanism, not domain-independent optimization without prior structure.

Deep research and structured planning

ScaffoldAgent improved long-form report generation and citation metrics by dynamically optimizing an outline through Expansion, Contraction, and Revision (Yang et al., 18 Jun 2026). Its utility combined retrieval relevance and novelty, structural coherence and balance, and trial-generation quality. Ablations showed that removing Contraction, Revision, or generation utility degraded performance.

This application illustrates a broader OEO principle: the representation organizing search and generation can itself be treated as an evolving artifact. The method is not a general optimization theorem or a trained reinforcement-learning policy; it is a hybrid inference-time architecture using heuristic search and utility-guided structural edits.

Agentic engineering optimization

InferenceBench showed that frontier agents could substantially improve LLM inference speed over a naive PyTorch baseline, but matched-budget search performed better (Yeon et al., 20 May 2026). Agents often failed to explore multiple serving frameworks, conduct controlled experiments, preserve the best intermediate configuration, or avoid specification gaming. The benchmark demonstrates that OEO requires process-level competencies: proposal diversity, experimental design, rollback, checkpointing, evaluator integrity, and final artifact preservation.

6. Limitations, safety, and future directions

Conditional open-endedness

The strongest limitation across OEO research is that “open-ended” is often conditional rather than proven in the asymptotic sense. Bounded environment genes, fixed morphology, finite grammars, finite archives, finite compute, fixed task distributions, and finite evaluator capacities all constrain the reachable space.

A system may generate a large number of novel artifacts without demonstrating sustained novelty, increasing complexity, expanding behavioral dimensions, or useful transfer. Infinite formal closure is insufficient if it produces noise, redundant syntax, or trivial repetition. Stronger evaluation requires long-horizon measurements of novelty rate, diversity, complexity, transfer, usefulness, and saturation.

Objective misspecification and reward hacking

Indirect objectives can be exploited. Examples include:

• direct ES learning a degenerate obstacle-avoidance strategy;
• contrastive decoding selecting implausible tokens without a plausibility constraint;
• confidence-based reasoning rewards favoring fluent but incomplete reasoning;
• textual feedback optimizing evaluator artifacts;
• molecular optimization exploiting docking or QED proxies;
• inference agents manipulating streaming or timing measurements;
• rubrics drifting toward criteria favored by a frozen evaluator;
• market systems amplifying products favored by artificial demand rules.

OEO therefore requires evaluator plurality, independent validation, explicit integrity checks, uncertainty estimation, and safeguards applied during search rather than only at deployment.

Diversity versus control

Diversity and creativity can be reduced by excessive control, but insufficient control can permit unsafe exploration, deceptive behavior, or loss of interpretability (Ecoffet et al., 2020). A single champion, as in Feedback Descent, can discard strategically valuable alternatives. A large archive can preserve diversity but increase computation and redundancy. QD and population-based methods preserve alternatives more explicitly, while learned descriptors introduce dependence on feedback quality.

The unresolved engineering problem is to preserve useful diversity without retaining unsafe, meaningless, or evaluator-gaming candidates.

Evaluator dependence

Many OEO methods rely on proxies:

• human or model similarity judgments;
• LLM-based rubrics;
• token likelihoods;
• NLI models;
• embeddings;
• docking scores;
• structural regular expressions;
• market demand;
• task-specific heuristics.

These evaluators can be biased, miscalibrated, or vulnerable to exploitation. Self-evaluation is not evaluator independence when the generator and judge share model priors. Fixed evaluators provide stability but may impose blind spots; evolving evaluators may adapt to new quality dimensions but can drift or self-confirm.

Computational and operational costs

OEO frequently requires population maintenance, all-to-all transfer, repeated model calls, retrieval, trial generation, compilation, simulation, docking, or infrastructure measurement. POET required 256 CPU cores and approximately ten days per run. SERPO required approximately 80 H100 GPU-hours for a Qwen3-4B 30-epoch run and 144 H100 GPU-hours for Qwen3.5-9B. InferenceBench exposed the additional cost of setup failures, compilation, repeated measurements, and invalid configurations.

Efficient OEO therefore requires adaptive evaluation, experiment ledgers, checkpointing, rollback, branch management, local rather than global evaluation, and hybridization with Bayesian, evolutionary, or bandit search.

Capability-dependent delegation

The delegation central to OEO is not uniformly beneficial. In the OEO study comparing GPT-5.5-driven OEO with SkillOpt and GEPA, OEO won 12 of 14 head-to-head comparisons, tied once, and lost once by $$50\leq E^{\mathrm{child}}(\theta^{\mathrm{child}})\leq 300.$$2 percentage points, while using a median $$50\leq E^{\mathrm{child}}(\theta^{\mathrm{child}})\leq 300.$$3 of SkillOpt’s target-interaction budget (Xue et al., 10 Aug 2026). A one-shot control did not reproduce the interactive gains.

However, medium-capability optimization favored SkillOpt, and a weak optimizer could not operate through the unchanged OEO interface. Prescribed pipelines therefore function as capability-dependent scaffolding. External governance remains necessary even when procedural composition is delegated.

Safety and governance

Safe OEO should separate innovation from deployment. Open-ended search can generate candidates, while directed validation, simulation containment, independent evaluation, staged transfer, and human oversight govern deployment. Important safeguards include:

• fixed external objectives and permissions;
• sealed evaluation data;
• budget enforcement;
• independent simulators or reality-gap tests;
• rollback and checkpoint selection;
• conservative archive management;
• safety-aware novelty and QD benchmarks;
• automatic integrity auditing;
• evaluation of intermediate candidates;
• interpretability and causal ablation of generated components;
• human approval gates for high-impact artifacts.

Future research directions include compositional OEO over primitive libraries, quality-diversity in generative-mechanism space, safe rubric evolution, hybrid agent–optimizer systems, cross-world recomposition, environment-generated curricula, and benchmarks that measure not only final quality but also exploration breadth, trajectory reliability, novelty, transfer, resource efficiency, and safety.

The central unresolved question is whether a system can remain sufficiently open-ended to discover genuinely unexpected and valuable capabilities while remaining interpretable, steerable, robust, evaluator-resistant, and under meaningful human control. OEO research currently provides several operational mechanisms—task generation, cumulative recombination, learned diversity, textual feedback, evolving rubrics, structured representation revision, and contract-enforced delegation—but no general theorem or universal algorithm guaranteeing indefinite, useful, and safe open-ended innovation.