Evo-Policy NeSy: Hybrid Evolutionary Policy Synthesis

• Evo-Policy NeSy is a hybrid framework that evolves a symbolic rule-based policy alongside a neural perceptual module, enabling automatic and interpretable policy synthesis.
• It employs evolutionary operators such as symbolic mutations and neural weight inheritance, guided by fitness-based and LLM-driven selection, to optimize policy representations.
• Empirical results on benchmarks like LunarLander demonstrate enhanced interpretability, compact policy code, and competitive performance compared to traditional end-to-end approaches.

Evo-Policy NeSy denotes a class of neural–symbolic evolutionary systems for automatic policy synthesis in control and decision-making domains. These frameworks combine nondifferentiable symbolic policy representations with neural perceptual modules or leverage foundation models (e.g., LLMs) to evolve executable policies, targeting settings where differentiable end-to-end learning or manual rule specification are inadequate. Evo-Policy NeSy approaches formalize the organism as a composite of a symbolic (rule-based or code-based) policy and a neural or semantic component, evolving these via structured mutation, selection, and fitness-guided search. This paradigm enables the synthesis of interpretable, verifiable, and often human-readable policies, supporting domains with restricted access to domain expertise or complex environment dynamics.

1. Neural–Symbolic Evolutionary Architectures

Evo-Policy NeSy encompasses implementations that model each system as an evolving organism $O = (\pi, w)$, where $\pi$ is a symbolic policy (explicit rule set or program fragment) and $w$ parameterizes a neural module, e.g., a feed-forward network mapping input $x\in X$ to atom probabilities $[0,1]^{|A|}$ (Thoma et al., 8 Jan 2026). These organisms are evolved over a population via mutation and fitness-based selection mechanisms, allowing for the concurrent optimization of symbolically encoded logic and neural feature extractors.

Architectures such as those introduced in (Thoma et al., 8 Jan 2026) extend the NeuroLog framework to enable mutable symbolic policies and nondifferentiable target concepts. Other approaches, as in EvoToolkit (Guo et al., 11 Jan 2026), operate in code space, synthesizing interpretable Python policy functions using evolutionary search driven by LLMs. Both variants embed the evolutionary loop within a broader neural-symbolic context, allowing for systematic discovery of hybrid policy representations.

2. Evolutionary Operators, Mutation, and Selection Schemes

In Evo-Policy NeSy, evolution is formulated as the application of symbolic and neural mutations to the organism population, typically combining all symbolic and neural mutations for each parent in parallel at every generation (Thoma et al., 8 Jan 2026), or using fitness-proportional selection and LLM-driven code mutation/crossover (Guo et al., 11 Jan 2026).

Symbolic mutations include cloning, rule addition (“coaching” by appending a rule $x \Rightarrow y$ for misclassified examples), and rule simplification (removing a literal from the last-added rule). Neural mutations are applied as either weight inheritance (preserving parent weights) or random re-initialization (Xavier initialization) (Thoma et al., 8 Jan 2026). The complete offspring set per parent thus covers the full cross product of mutation operators.

Selection employs fitness-based or tournament schemes. For example, (Thoma et al., 8 Jan 2026) partitions offspring into Beneficial, Neutral, or Detrimental sets based on relative fitness; selection proceeds stochastically over beneficial mutants or otherwise by maximum relative fitness, with thresholds and exponentiation for selection pressure. EvoToolkit (Guo et al., 11 Jan 2026) uses roulette-wheel (fitness-proportional) sampling and elitist generational replacement: top-performing offspring and parents are retained for the next generation.

3. Policy Representations: Symbolic Rules and Executable Code

Symbolic policies $\pi$ in Evo-Policy NeSy are either prioritized rule sets under Machine-Coaching semantics (Thoma et al., 8 Jan 2026) or directly executable code within a subset of Python (Guo et al., 11 Jan 2026). A typical symbolic policy consists of an ordered list of propositional rules of the form

$$r: (\ell_1 \wedge \cdots \wedge \ell_k) \Rightarrow h$$

where literals $\ell_j$ denote atomic properties (e.g., $+a_i$, $\pi$0), and $\pi$1 is the consequent.

Code-based policies are self-contained Python functions permitting arithmetic expressions, discrete returns, and bounded control flow (e.g., if–then–else), with explicit restrictions to avoid unbounded loops and unsafe operations. Validity is enforced by AST parsing and sandboxed execution. This representation supports human inspection, formal analysis, and direct modification (Guo et al., 11 Jan 2026).

Example evolved policy (LunarLander, (Guo et al., 11 Jan 2026)):

$w$8 This program encapsulates a three-phase descent heuristic with explicit, interpretable decision boundaries.

4. Fitness Evaluation, Abductive Training, and Semantic Loss

Fitness assignment in Evo-Policy NeSy is domain-specific. For reinforcement learning environments (e.g., LunarLander), fitness is the average cumulative reward across $\pi$2 episodes of horizon $\pi$3:

$\pi$4

where $\pi$5 encodes task-defined rewards and penalties (Guo et al., 11 Jan 2026). Success is thresholded (e.g., reward $\pi$6).

For symbolic NeSy systems, fitness is defined as validation accuracy of the deduced output $\pi$7 from the combination of the neural and symbolic modules (Thoma et al., 8 Jan 2026). Training optimizes the neural module using a semantic loss:

$\pi$8

where $\pi$9 are the neural outputs and $w$0 is the set of atom assignments sufficient for $w$1 to deduce $w$2. This abductive training signal does not require $w$3 to be differentiable.

For multi-objective optimization (e.g., code length, cyclomatic complexity), composite fitness $w$4 may penalize these terms, although experimental emphasis is typically on pure reward or accuracy (Guo et al., 11 Jan 2026).

5. Experimental Methodology and Empirical Results

Experimental protocols span simulated reinforcement learning benchmarks and full ablation suites for policy induction. On LunarLander-v3 (Guo et al., 11 Jan 2026), Evo-Policy NeSy (EvoEngineer) achieves 66.6 ± 56.1 average reward and 40% success with $w$545 LLM calls; scaling to EvoEngineer⁺ (200 calls) reaches 143.6 reward and 70% success. Success is defined as achieving $w$6 reward per episode. Baseline performance for PPO at 1M environment steps is 214 ± 27 reward (60% success). Variants (FunSearch, EoH) underperform Evo-Policy NeSy, and ablations confirm increased LLM calls yield both improved performance and more compact code (down from ∼80 lines to 59).

For symbolic NeSy settings (Thoma et al., 8 Jan 2026), experiments use 30 randomly generated shallow policies over $w$7, with each atom represented as MNIST digits. 20,000 training, 2,000 validation, and 2,000 test samples are randomly sampled. The evolved organisms achieve a median test accuracy of 0.992, mean 0.944, and median “abstain” rate near zero. End-to-end neural baselines reach 98% median test accuracy but lack explicit rule extraction and exhibit higher variance.

6. Interpretability, Verifiability, and Limitations

Evo-Policy NeSy produces human-interpretable policies—either as concise Python functions with explicit states and decision boundaries (Guo et al., 11 Jan 2026), or as prioritized rule sets that can be directly analyzed and modified (Thoma et al., 8 Jan 2026). These representations enable formal verification, including AST-based static analysis for code policies (to guarantee properties such as absence of unbounded loops and bounded control flow) and formal safety checks on cyclomatic complexity and threshold logic.

Domain experts can inspect, stress-test, or directly adjust policy heuristics (e.g., changing landing thresholds), without necessitating retraining or loss of policy transparency. The underlying neural modules remain amenable to abductive and semantic reasoning.

Limitations include:

• High variance in convergence quality across seeds
• Scalability to high-dimensional, continuous action spaces, or complex RL domains remains open
• Heavy reliance on the pretrained knowledge of LLMs (for LLM-driven code evolution)
• Nontrivial computational cost from LLM inference and environment simulation, though often still less expensive than real-world system rollouts (Guo et al., 11 Jan 2026, Thoma et al., 8 Jan 2026)

7. Theoretical Context and Integration with Existing Paradigms

Evo-Policy NeSy concretely instantiates Valiant’s Evolvability model in the neural–symbolic setting (Thoma et al., 8 Jan 2026), constructing evolvable representations via rule addition/simplification and weight inheritance across generations, and empirically demonstrating that shallow propositional policy structures coupled with trainable neural components are evolvable under population-based search.

The approach extends the NeuroLog paradigm from pure black-box module composition to direct mutation and learning in the symbolic field, facilitated by Machine Coaching semantics for prioritized mutable rule sets and abductive learning signals for neural training. In code evolution, EvoToolkit closes the neural–symbolic loop by using LLMs as semantically-aware mutation and crossover engines, guiding the search by program patterns and control domain priors (Guo et al., 11 Jan 2026).

Evo-Policy NeSy thus integrates symbolic interpretability, neural perceptual modeling, evolutionary search, and foundation model priors into a unified, sample-efficient, and human-interpretable policy synthesis methodology.