The Time Value of Evolution

Abstract: In evolutionary search, a weak child can be a valuable ancestor that makes high-fitness regions reachable. Immediate-return control is blind to this delayed utility, penalizing mutations through their immediate offspring even when they open productive future lineages. We formalize this hidden dynamic as the time value of evolution within a finite-horizon Markov decision process. To exploit it, we introduce Lineage-Value Policy Gradients (LVPG), a long-horizon actor-critic framework for automated trading policy discovery. Our architecture decouples search control into specialized policy heads over a shared generative backbone: a bootstrapped critic head estimates the value of finite-horizon lineage potential from multi-step mutation trees, while an actor head dynamically modulates mutation intensity over the remaining search budget. We isolate the impact of long-horizon credit assignment against immediate-return optimization across 90 paired runs under matched operators, lineage supervision, folds, seeds, and budgets. Path-based credit assignment substantially accelerates finite-budget search, increasing validation best-so-far AUC by 0.394 Sharpe units. LVPG also produces fewer temporary regressions than immediate-return optimization and recovers from them more often. Finite-horizon lineage value yields more selective non-monotonic search and stronger policies within identical resource constraints.

• The paper formalizes delayed mutation utility as the time value of evolution and introduces Lineage-Value Policy Gradients, which credit mutations for downstream best-so-far improvements over an eight-step search horizon.
• PPO-Path improved validation AUC by 0.394 Sharpe units over PPO-Immediate and increased mean sealed-test Sharpe from 0.862 to 1.321 across 90 matched paired runs.
• The method produced fewer temporary regressions, faster recoveries, and more selective mutation control, but its generalization remains limited by fixed mutation operators, historical trading data, and domain-specific evaluation.

The paper "The Time Value of Evolution" (2608.13297) addresses a specific credit-assignment deficiency in evolutionary search: mutations are typically valued by the immediate fitness of their offspring, even though a weak child can open a lineage whose descendants exceed the previous best before a finite search budget expires. The authors formalize this delayed utility as the time value of evolution (TVE) within a finite-horizon Markov decision process and introduce Lineage-Value Policy Gradients (LVPG), an actor-critic framework applied to automated trading-policy discovery.

Formalizing delayed utility in evolutionary search

The evolutionary process is cast as an MDP $$\mathcal{M}_{\mathrm{evo}}=(\mathcal{S},\mathcal{Z},P,R,H)$$ with horizon $H=8$. A state comprises the current policy in natural-language and executable forms, training-window execution metrics and feedback, the best-so-far fitness $B_t$, and normalized time $t/H$. The action space contains three behaviorally calibrated mutation radii—Refine, Interpolate, and Explore. The key quantity is the expected best-so-far improvement over the remaining horizon, $Q_h^\pi(s,z)$, and the residual

$$\operatorname{TVE}_h(s,z) = Q_h^\pi(s,z) - Q_1^\pi(s,z),$$

which is positive whenever one-step evaluation understates a mutation's finite-budget utility. With $\gamma=1$, the path return telescopes to $\widetilde{B}_H - \widetilde{B}_t$, so an action can accrue positive credit after an immediate fitness decline if a later descendant establishes a new path best. Invalid children incur a penalty of 0.25 standardized units. The authors are explicit that PPO-Path optimizes the finite-horizon return directly and does not separately estimate the TVE residual; the quantity serves as a conceptual bridge rather than an estimated object.

This framing distinguishes the work from adaptive operator selection and from Novelty Search or MAP-Elites. Prior operator-control methods do not establish whether a language-model-generated program mutation should be valued through the executable lineage it opens, and diversity-preservation methods change the objective or archive rule, whereas LVPG leaves parent sampling and archive rules fixed, crediting a lower-fitness child only when later descendants exceed the path best.

Architecture: frozen ELM with LoRA actor and critic

The variation operator is an Evolutionary LLM (ELM): a Qwen3-8B backbone trained offline on 44,736 multitask examples covering strength-conditioned natural-language mutation, compilation into Genetic Programming Trading Language (GPTL), and back-translation, drawn from reversed degradation walks. Behavioral calibration via Offset Direct Preference Optimization (14,735 preference pairs, $\beta=0.3$) produces empirically separated radii: across 536 held-out mutations, median executed behavioral disagreement is 0.095, 0.194, and 0.326 for Refine, Interpolate, and Explore respectively, with strict parent-median ordering in 91.7% of cases and validity above 90% at every strength. The ELM is frozen during all online learning.

Online learning is confined to two LoRA adapters ($r=150$, $H=8$0, roughly 409.2M parameters each, about 5% of the base model) over the frozen 36-layer backbone. The actor maps the state-prompt token to a three-way radius distribution; the critic maps a transition prompt to a scalar estimate of finite-horizon lineage value. The critic is bootstrapped from depth-five ternary mutation trees (243 leaves, 363 child executions per root before memoization) with targets clipped at $H=8$1; these targets are exact for the evaluated deterministic tree but optimistic with respect to the full stochastic transition kernel. The warm start reaches held-out $H=8$2 with mean absolute error 0.564 Sharpe units on 2,109 transitions.

Matched experimental protocol

The evaluation uses hourly continuous futures for the S&P 500 E-mini, Silver, and 30-Year Treasury (roughly 24,000 bars each, ending October 20, 2025), with ten end-anchored rolling-origin folds per market: four months training, one month validation, a three-day embargo, and one month of sealed test. Fitness is standardized per asset-fold against a frozen 64-policy reference set using the median absolute deviation. Seven primary conditions—three fixed radii, Uniform, Schedule, PPO-Immediate, and PPO-Path—yield 90 runs each (three assets, ten folds, three seeds), plus 30-run Treasury ablations, for 720 total runs and 184,320 archive-facing children.

The central contrast holds everything fixed except the return: PPO-Path and PPO-Immediate share the ELM, state and action spaces, critic initialization, mutation-tree supervision, first-action distillation, folds, seeds, optimization settings, and the 256-child budget, differing only in whether credit comes from downstream best-so-far progress ($H=8$3) or clipped one-step fitness change ($H=8$4). Inference uses hierarchical block bootstrap with 10,000 replicates over complete paired runs, Holm-adjusted $H=8$5-values, and probability of superiority, with transitions never treated as independent observations.

Results

The paired results are summarized below.

Baseline	$H=8$6 validation AUC (Sharpe units)	$H=8$7 test Sharpe
PPO-Immediate ($H=8$8)	0.394 [0.261, 0.526], $H=8$9	0.459 [0.113, 0.795], $B_t$0, PS=.633
One-step, $B_t$1 ($B_t$2)	0.599 [0.411, 0.795], $B_t$3	0.657 [−0.045, 1.367], $B_t$4
Schedule ($B_t$5)	0.597 [0.433, 0.758], $B_t$6	0.590 [0.261, 0.924], $B_t$7
Uniform ($B_t$8)	0.654 [0.486, 0.807], $B_t$9	0.702 [0.313, 1.078], $t/H$0
Fixed-Medium ($t/H$1)	0.685 [0.551, 0.825], $t/H$2	0.553 [0.161, 0.952], $t/H$3

All five validation AUC contrasts remain significant after Holm correction. Mean sealed-test Sharpe rises from 0.862 under PPO-Immediate to 1.321 under PPO-Path, with secondary descriptive gains in annualized return (22.8% vs. 15.3%), Sortino (2.462 vs. 1.623), and positive test Sharpe frequency (86.7% vs. 81.1%). The one-step test contrast is positive but statistically uncertain, and the conservative Friedman omnibus over the 30 shared Treasury blocks gives $t/H$4—the authors acknowledge this does not support the broader omnibus hypothesis, though it evaluates a reduced sample.

Horizon ablations on Treasury blocks locate the effect: moving from one-step to four-step credit raises validation AUC by 0.399 (adjusted $t/H$5), and one-step to eight-step raises it by 0.599 ($t/H$6), while the incremental eight- over four-step gain of 0.200 is not significant ($t/H$7). The defensible conclusion is one-step versus multistep credit; the eight-step horizon is descriptively best but not established as universally optimal.

The mechanism results indicate that long-horizon control makes non-monotonic search more selective rather than simply more exploratory. PPO-Path produces fewer temporary regressions than PPO-Immediate (54.2% vs. 59.8% of steps) and recovers from more of them (48.0% vs. 39.9%). Its regressions are deeper (0.94 vs. 0.77 Sharpe units) but recover sooner (1.93 vs. 2.20 steps later) and yield larger post-recovery improvement (0.535 vs. 0.251). A behavioral model shows the actor reads more than the clock: a multinomial model with remaining budget, parent-fitness percentile, stagnation, and recent improvement beats a time-only model (likelihood ratio 544.74, $t/H$8). Interventional forced-action continuations on 720 held-out states show the controller selects the highest-valued action in 51.9% of states against a 33.3% chance rate, with mean regret 0.157 Sharpe units, and that no radius is uniformly best—Refine and Explore have nearly equal mean continuation values (0.538 and 0.533) overall, but Refine dominates among states already in regression (0.727).

On 23,040 states excluded from fitting, the critic achieves mean run-level Spearman correlation 0.597 with realized path returns, 19.3% explained variance, and MAE 0.509, with monotonically ordered calibration quintiles—though the top quintile overpredicts, so the critic is an informative ranker rather than a calibrated oracle. After controlling for parent fitness, path best, budget, stagnation, and recent improvement, the critic retains coefficient 0.361 with incremental partial $t/H$9, indicating that full program context contributes future-yield information beyond simple state proxies.

Limitations and open questions

The paper concedes several constraints that qualify these results. The ELM's content distribution and the three calibrated radii are fixed, preventing online improvement of edits and leaving a coarse, domain-specific action space; the study therefore cannot speak to whether learned mutation content would compound with learned control. Because PPO-Path and PPO-Immediate share critic learning and tree distillation, the matched comparison isolates the return horizon but not the independent necessity of each component. The critic learns optimistic maxima from deterministic depth-five trees rather than expected stochastic returns and is imperfectly calibrated. Fixed, Uniform, and Schedule arms are matched only in archive-facing children, not in the full tree and PPO training cost borne by the PPO arms. The evaluation covers three futures markets, one policy language, horizons through eight steps, and one-month test windows with fixed costs omitting market impact, capacity, latency, and order-book effects; the results establish finite-budget search gains under controlled historical evaluation, not live profitability. Open questions the paper leaves include whether the eight-step horizon advantage generalizes beyond this setting, whether the framework transfers to other program-search domains, and how richer or learned action spaces interact with lineage-based credit.

Conclusion

The paper's central claim is that immediate offspring fitness is an incomplete measure of a mutation's value because it omits the future search opportunities the mutation creates. By framing finite-budget evolutionary search as an investment problem with a $Q_h^\pi(s,z)$0 path return, and by pairing a frozen language-model variation engine with LoRA actor and critic adapters trained via PPO and mutation-tree bootstrapping, LVPG demonstrates across 90 matched paired runs that lineage-based credit accelerates search (validation AUC +0.394 Sharpe units), improves sealed-test performance (mean Sharpe 1.321 vs. 0.862), and produces fewer, more recoverable temporary regressions. The evidence supports credit assignment over reachable descendants as a general design principle for generative program search, with its generality beyond trading-policy discovery remaining to be tested.