Open-Endedness Search in Computational Systems

Updated 9 November 2025

Open-endedness search is a process that continuously generates novel, increasingly complex artifacts without a fixed end point.
It employs evolutionary dynamics, quality diversity algorithms like MAP-Elites, and meta-diversity techniques to avoid premature convergence and foster creativity.
Applications span artificial life, neural architectures, dialogic search, and cultural evolution, providing measurable insights into novelty, complexity, and diversity.

Open-endedness search refers to algorithmic and interactive processes that continually generate novel, increasingly complex, and unanticipated artifacts or behaviors, typically without a fixed endpoint, equilibrium, or externally imposed objective. Unlike classical search, which optimizes for a specified target or loss, open-endedness (OE) seeks unbounded discovery, complexity growth, and the ongoing surfacing of unexpected phenomena. Contemporary research explores open-endedness in evolutionary systems, artificial life, neural architectures, learning environments, question answering, and dialogic information-seeking, and provides formal, empirical, and algorithmic foundations for designing and evaluating such systems.

1. Formal Frameworks and Definitions

Open-endedness frameworks are grounded in formal criteria that distinguish ongoing novelty, unbounded complexity growth, and sustained diversity.

General Formalism:
- Novelty: For all $t$ , for any $T > t$ , there exists $T'>T$ such that
$\mathbb{E}[\ell(t,T')] > \mathbb{E}[\ell(t,T)]$ - Learnability: For all $T$ , for any $t<T$ , there exists $t' \in (t,T)$ such that

$\mathbb{E}[\ell(t',T)] < \mathbb{E}[\ell(t,T)]$ - A system is open-ended with respect to an observer if both hold; it creates ever more surprising, yet ultimately learnable, outputs.
Complexity and Diversity Criteria:
- $|S(t)| \to \infty$ as $t \to \infty$ (unbounded diversity)
- For all $t$ , there exists $x \in S(t)\setminus \cup_{\tau < t} S(\tau)$ (continuous novelty) (Borg et al., 2022).
Taxonomies of Novelty:

Expanding on Banzhaf et al., three kinds of OE are recognized (Taylor, 2018, Soros et al., 28 May 2024): 1. Exploratory: Generation of new instances within the current generative model or state space. 2. Expansive: Discovery of outcomes requiring model expansion but not new primitives. 3. Transformational: Emergence of artifacts demanding changes to the generative meta-model itself.

Quantities and Metrics:
- Novelty: Minimum embedding-space distance to all prior states, e.g., $N_t(\theta) = \min_{t'<t} d(f(RS^t(\theta)),f(RS^{t'}(\theta)))$ (Kumar et al., 23 Dec 2024).
- Complexity: Kolmogorov complexity $K(x)$ , model size, behavioral entropy, or description length.
- Diversity: Population variety, entropy over feature bins, coverage of behavioural niches (Ecoffet et al., 2020, Norstein et al., 2023).
Open-Endedness in Dialogue and Search:

In dialogic exploratory search, OE is linked to multi-turn, iterative sense-making with no predefined query or path, emphasizing process structure over goal alignment (Schneider et al., 2023).

2. Mechanisms and Algorithms for Open-Ended Search

Implementation of open-endedness involves mechanisms that avoid both early convergence and the limitations of static objectives.

Evolutionary Dynamics:

The evolutionary process is decomposed into genotype→phenotype generation $p = M_L(g,c_a,c_b)$ , phenotype evaluation $(\ell, \bar{s}_r, \bar{p}_m) = E_L(p,c_a,c_b)$ , and reproduction with variation $g' = R_L^{\bar{s}_r}(p,\bar{p}_m)$ , with an emphasis on intrinsic (evolvable) over extrinsic (fixed) mechanisms (Taylor, 2018). In biologically inspired OE, mechanisms for expanding the mutability of the genotype–phenotype map, constructing transdomain bridges, and evolving new operators are essential for transcending mere exploratory OE.

Quality Diversity and MAP-Elites:

MAP-Elites algorithms discretize behavior space into bins and maintain a repertoire of elites per bin. By adding novelty as an explicit map dimension (e.g., autoencoder reconstruction error as behavioral diversity), the system performs structured, open-endedly scalable exploration across agent–environment pairs (Norstein et al., 2023).

Meta-Diversity Search:

The “meta-diversity search” paradigm alternates between learning new modular behavior characterizations (via VAEs or self-supervised embeddings) and performing diversity-maximizing search within each, dynamically splitting and growing the BC hierarchy as saturation is detected (Etcheverry et al., 2023). This supports perpetual exploration by evading the constraints of any single descriptor.

Neural and Hybrid Methods:

Open-endedness in neural networks employs mechanisms such as probabilistic outputs, minimum-over-set losses, adversarial (GAN) objectives, dynamic architectures, and meta-learning of update rules or individualities. Alternating evolutionary population-based search with gradient updates enables continual curriculum generation and shifting objectives (Guttenberg et al., 2018).

Foundation Model–Driven Search:

Search for temporally open-ended novelty is operationalized by maximizing the embedding-space distance between each new observed state and the entire search history using pretrained vision-language or LLMs. This approach aligns with human-like perceptions of change and interest by using foundation models’ semantic priors (Kumar et al., 23 Dec 2024, Zhang et al., 2023).

Reinforcement Learning in Open-Ended QA and Reasoning:

Unified RL agents perform open-domain open-ended and closed-ended QA by interleaving search actions (retrieval or querying) with answer synthesis, guided by composite rewards that reflect answer diversity, factuality, and coverage. Condensation modules structure knowledge, and reward shaping ensures the agent “discovers, synthesizes, and presents diverse findings” (Mei et al., 22 May 2025).

3. Empirical Case Studies and Application Domains

Open-endedness search principles are instantiated across diverse empirical settings.

Domain	Key Mechanism	Output/Artifact Types
Artificial Life (ALife)	Evolutionary processes with mutated rule sets	Digital lifeforms, CA rules, morphologies
Neural QD/EA	Novelty/complexity preserving loss functions	Neural architectures, behavioral policies
Cultural evolution	Multi-channel inheritance, social learning	Tool repertoires, traditions, artifacts
Game environments	Open-ended RL/EA, co-evolution	Agents, terrains, strategies
Commonsense reasoning	Iterative multi-hop graph search	Reasoning chains, answers
Conversational search	Multi-loop dialogic exploration	Domain structures, knowledge maps

For example, EvoCraft generates 3D block-based artifacts in Minecraft and reveals that open-ended search can solve growth tasks easily but fails on tasks with deceptive or highly epistatic dependencies unless novelty or hierarchical stepping stones are incorporated (Grbic et al., 2020). In open-domain QA, agents trained via RL on search-and-answer tasks, with type-adaptive reward functions, robustly outperform larger models constrained to static knowledge (Mei et al., 22 May 2025). Meta-diversity techniques applied to generative artifact spaces (e.g., in Minecraft) demonstrate the emergence of fractal-like and motile structures not reachable via traditional, single-descriptor searches (Etcheverry et al., 2023).

4. Quantitative Evaluation and Analysis of Open-Endedness

Assessment of OE requires longitudinal measurement of novelty, diversity, and complexity, as well as coverage and discoverability.

Lifetime Novelty Curve:

For each timestep, $N_t(\theta)$ computes the minimum embedding distance to all prior states (Kumar et al., 23 Dec 2024).

Complexity Growth:

Typically evaluated via model size, behavioral sophistication, or compressed description length; open-endedness requires $\lim_{t \to \infty} C(t) = \infty$ (Soros et al., 28 May 2024).

Archive Dynamics and Coverage:

MAP-Elites–type archives track the proportion of filled behavior niches; continuous expansion or non-saturation indicates structural open-endedness (Norstein et al., 2023).

Subjective Models of Interestingness:

OMNI-type frameworks use foundation model predictions to gate task sampling, quantifying the human-aligned “interestingness” of candidate artifacts and measuring the number and variety of qualitatively novel, learned tasks (Zhang et al., 2023).

Dialog Dynamics:

Process mining on exploratory conversational search uncovers concurrent loops of information-seeking, aggregation, and feedback, suggesting a reusable process model for guiding users through nascent domain space (Schneider et al., 2023).

5. Challenges, Limitations, and Safety Considerations

Open-ended search presents distinctive challenges that differ from classical search, particularly for safety, control, and interpretability.

Tension Between Control and Creativity:

The “specification gap” is the distance between designer-stated incentives and ideal objectives; open-endedness systemically increases the risk of emergent behaviors diverging from designer intent (Ecoffet et al., 2020). Mesasystems may self-modify objectives in unforeseen ways.

Limits of Diversity and Complexity:

Practical systems risk diversity collapse (convergence on narrow solutions), complexity saturation (novelty plateaus), or failure to transcend extant individuality levels. Evolutionary designs require explicit mechanisms for ongoing variation, shifting contexts, and mutation of the genotype–phenotype map (Taylor, 2018, Guttenberg et al., 2018).

Foundation Model Biases:

When novelty is measured by FM embedding distance, the open-endedness score is only as valid as the FM’s alignment with human-understandable change. Missing representation of critical features can stall exploration (Kumar et al., 23 Dec 2024).

Resource Constraints:

Large-scale, temporally open-ended search demands substantial sampling; dynamic management of evaluation resources, memory, and model retraining must be addressed for scalability (Etcheverry et al., 2023, Kumar et al., 23 Dec 2024).

Safety and Unintended Discoveries:

Constraints such as behavior shields, innovation budgets, and rejection sampling in behavior space are proposed to avoid unbounded or harmful behaviors (Ecoffet et al., 2020, Hughes et al., 6 Jun 2024). Automated interpretability, regular human feedback, and governance are recommended for deployment settings.

6. Future Directions and Interdisciplinary Opportunities

Research opportunities span algorithmic and foundational perspectives.

Cross-domain Integration:

Aligning value- and surprise-based computational creativity with ALife’s complexity and adaptive success could yield a more unified theory of artificial innovation (Soros et al., 28 May 2024).

Observer-Dependent OE:

Open-endedness is inherently observer-relative; integrating automated observers with learnability and novelty scores enables system-agnostic, comparative benchmarking (Hughes et al., 6 Jun 2024).

Meta-Search and Representation Learning:

Dynamic expansion of behavior characterizations, meta-learning of novelty functions, and embedding models adaptive to emergent features are active research arenas (Etcheverry et al., 2023).

Human-in-the-Loop and Safety Integration:

Incorporating continual human feedback, hybrid FM–human benchmarks, and explainability modules will be central to both performance and safety (Zhang et al., 2023, Hughes et al., 6 Jun 2024).

Cultural and Societal Mechanisms:

Cultural evolution introduces multi-channel inheritance, teaching, and symbolic recombination as drivers of “tall,” “wide,” and recombinative OE, informing the design of artificial systems targeting emergent unbounded dynamics (Borg et al., 2022).

Open-endedness search thus provides a rigorous, multifaceted approach for the generation and analysis of unbounded, creative exploration in artificial and computational systems, with frameworks, mechanisms, and empirical successes spanning evolutionary biology, artificial life, neural computation, natural language, and interactive search.