Bidirectional Evolutionary Search

• BES is a unified search paradigm that combines forward evolutionary operations with backward goal decomposition to break down global objectives into verifiable subgoals.
• It overcomes sparse feedback by using evolutionary variation to escape entropy shells and applying recursive subgoal verification for enhanced sampling efficiency.
• Its practical applications include self-improving language models and constrained multiobjective optimization, yielding exponential gains in performance and diversity.

Bidirectional Evolutionary Search (BES) is a unified search paradigm designed for efficient exploration and optimization in settings where the objective is to generate high-quality candidate solutions under sparse verification signals, with particular utility for LLMs, agentic systems, and multiobjective evolutionary optimization. BES advances prior approaches by coupling forward candidate evolution—augmented with evolutionary variation operators to escape model-induced entropy shells—with backward goal decomposition that delivers dense intermediate rewards by recursively breaking down global objectives into verifiable subgoals. This dual mechanism has been shown to yield exponential improvements in sampling efficiency for both post-training self-improvement and inference-time search, and can be instantiated for continuous optimization tasks through the integration of bidirectional coevolutionary architectures and differential evolution operators (Xu et al., 27 May 2026, Mendes et al., 2024).

1. Motivating Limitations of Conventional Search

Standard search and inference strategies in high-complexity spaces, such as best-of-N sampling and tree search for LLMs or canonical differential evolution (DE) for multiobjective optimization, are fundamentally limited by sparse, coarse feedback and search confinement to the probabilistic or feasible region induced by the model or constraints. In the domain of LLMs, expansion-only methods generate candidates almost exclusively from high-probability regions of the autoregressive policy $\pi_\theta$, producing an "entropy shell" barrier that makes rare but correct solutions nearly inaccessible. Analogously, classical single-directional evolutionary schemes can drive populations prematurely toward feasibility or optimality fronts, neglecting the rich diversity to be found in infeasible or unexplored regions. BES addresses both issues by integrating two complementary flows: forward evolutionary operations to traverse and recombine regions outside standard model rollouts, and backward decomposition to provide guidance through hierarchically constructed subgoals and dense verification signals (Xu et al., 27 May 2026, Mendes et al., 2024).

2. Bidirectional Evolutionary Search Framework

BES consists of two alternating components:

Forward Evolutionary Search maintains a pool of partial solutions or trajectories and applies a mixture of expansion and recombination operators. For LLMs and agentic systems, operators include autoregressive expansion (sampling several steps from $\pi_\theta$), combination, crossover, translocation, and deletion—each generating new trajectories by recombining or trimming sequences. In continuous optimization, BES is instantiated as bidirectional differential coevolution, where the DE mutation, binomial crossover, and polynomial mutation operators generate the trial population, and selection advances the main population toward feasibility and objectives (Xu et al., 27 May 2026, Mendes et al., 2024).

Backward Goal Decomposition/Search recursively breaks down the primary objective into a tree of local subgoals, each equipped with a verifier $V_g(x,n)\in[0,1]$. Partial and complete solutions are scored by recursively aggregating (with a blending parameter $\alpha$) both raw verifier outputs and the scores of child subgoals. Every fixed number of forward steps, BES adaptively decomposes unsatisfied leaves using the base model or generator (e.g., prompting an LLM), yielding more granular feedback structures.

The backward flow in evolutionary optimization is operationalized as an "archive population" that maintains non-dominated infeasible fronts; this archive feeds diversity and novel genetic material back to the main (feasible) population, broadening exploration (Mendes et al., 2024).

3. Theoretical Analysis: Escape from Entropy/Feasibility Shells

BES provides formal guarantees on escaping both model entropy shells (for sequence generation) and hard constraint-induced feasible regions (for optimization):

• Entropy Shell Escape (LMs, Agentic Systems): Expansion-only search is confined to a shell of size $\exp(H_T + \epsilon T)$, where $H_T$ is the trajectory-level entropy. Evolutionary recombination of independently generated sequence blocks creates candidates outside this shell, with expectation and probability bounds that grow with block total correlation. Theorems precisely quantify how evolution operators break model-induced typicality (Xu et al., 27 May 2026).
• Exponential Sample Efficiency via Goal Decomposition: Terminal-only search, which aims to satisfy all constraints or solve the full objective in one shot, requires $\Omega(1/\prod_{i=1}^m p_i)$ samples to achieve coverage of $m$ subgoals with individual success probabilities $p_i$. By contrast, BES's backward decomposition only requires $O(p_{\min}^{-1} \log(m/\delta))$ samples—a gain that is exponential in $\pi_\theta$0 when success probabilities across subgoals are uniform (Xu et al., 27 May 2026).

4. Bidirectional Coevolution in Constrained Multiobjective Optimization

A major instantiation of BES for continuous domains is Non-Dominated Sorting Bidirectional Differential Coevolution (NSBiDiCo) (Mendes et al., 2024). This framework comprises:

• Two Populations: Main population $\pi_\theta$1 (feasible bias) and archive population $\pi_\theta$2 (infeasible, diverse front).
• Bidirectional Information Flow:
  • Forward: Restricted mating pools draw from both populations; DE-based variation and environmental selection (with crowding distance) advance $\pi_\theta$3.
  • Backward: Archive update selects of infeasible non-dominated solutions using the constraint-dominance principle (CDP), treating constraint violation as an explicit $\pi_\theta$4-th objective during sorting.
• Operator Details: The pipeline uses DE/current/1/bin for mutation and crossover, polynomial mutation, and a non-dominated sorting schema analogous to NSGA-II.
• Impact: Empirically, NSBiDiCo demonstrated strong improvements in hypervolume (HV) and inverted generational distance (IGD) metrics versus both classical BiCo and unidirectional DE across standard benchmarks and real-world tasks.

Parameter	Benchmarks	Real-world
Population size $\pi_\theta$5	100	100
Mutation factor $\pi_\theta$6	0.5	0.45
Crossover probability $\pi_\theta$7	1.0	0.7
Poly. mutation prob.	$\pi_\theta$8	$\pi_\theta$9
Poly. mut. index $V_g(x,n)\in[0,1]$0	20	20
Max. function evals	$V_g(x,n)\in[0,1]$1	$V_g(x,n)\in[0,1]$2

On the DOC and LIRCMOP suites, NSBiDiCo achieved IGD wins in 8/9 and 11/14 cases, and HV wins in 7/8 and 12/14 cases, respectively (Mendes et al., 2024).

5. Empirical Validation and Benchmarks

BES has been extensively evaluated in both the self-improvement of LLMs and constrained multiobjective optimization:

• LLMs and Agentic Systems: On logical reasoning (Knights-and-Knaves) and multi-hop QA (MuSiQue), BES enabled consistent accuracy gains where best-of-N sampling, preference optimization (MaxRL), and tree-based RL (Tree-GRPO) stagnated or degraded. On MuSiQue, BES achieved 7.0% accuracy versus 2.1–4.0% from baselines and demonstrated improvements in both number of valid search actions and finish ratio (Xu et al., 27 May 2026).
• Open Problem Solving (Inference): Using GPT-5 as the backbone with a fixed API budget, BES outperformed open-source evolutionary frameworks (OpenEvolve, GEPA, ShinkaEvolve) on geometric packing benchmarks. BES achieved superior average and best-case objectives with reduced variance across runs.

Strategy	Square Avg	Square Best	Rect Avg	Rect Best	Conv Avg	Conv Best
OpenEvolve	2.531±.018	2.541	2.267±.014	2.276	.025±.005	0.027
GEPA	2.613±.022	2.628	2.326±.023	2.354	.025±.002	0.027
ShinkaEvolve	2.464±.083	2.541	2.335±.026	2.358	.023±.005	0.026
BES	2.623±.014	2.632	2.349±.012	2.360	.026±.001	0.027

• Ablation and Cost: Removing either evolutionary recombination or backward decomposition substantively degraded performance. BES incurred ≤30% additional overhead relative to tree-search baselines while producing substantially better results (Xu et al., 27 May 2026).

6. Practical Considerations and Limitations

BES requires the existence of efficient verifiers for primary and subgoals, which may not be available in all domains. Decomposition quality is contingent on the competence of the model generating subgoals; insufficiently capable models may propose low-quality decompositions, diminishing search efficiency. The increased computational cost due to evolutionary recombination and repeated scoring necessitates careful tuning for latency- or budget-constrained settings.

Potential extensions include learned decomposition modules in place of prompt-based splitting, adaptive weighting of search operators, and integration with differentiable feedback for gradient-augmented search (Xu et al., 27 May 2026).

7. Connections and Extensions

BES generalizes the core search ideas underpinning best-of-N, beam/tree search, classical genetic algorithms, and multiobjective evolutionary strategies. Its bidirectional design is instantiated in multiple modalities:

• In LLM domains, BES structurally surpasses uni-directional sampling and tree search by introducing entropy-shell escape mechanisms and subgoal-guided density.
• In continuous optimization, NSBiDiCo extends BiCo by integrating DE mutation/crossover, non-dominated sorting with crowding, and an archive that explicitly maintains infeasible nondominated fronts for exploration (Mendes et al., 2024).

BES thus unifies exploration and exploitation across both feasible and infeasible directions, under sparse reward or constraint-heavy regimes, and is supported by rigorous theoretical and empirical evidence of superior sampling efficiency, convergence, and diversity across tasks.

References:

• "Self-Improving LLMs with Bidirectional Evolutionary Search" (Xu et al., 27 May 2026)
• "Non-Dominated Sorting Bidirectional Differential Coevolution" (Mendes et al., 2024)