InterEvolve: Test-Time Controller Adaptation

• InterEvolve is a framework for adapting a pretrained controller to new tasks at test time, evolving executable, staged reward programs to ensure effective task execution without retraining the controller.
• The framework combines a large language model (LLM) to revise task strategies and reward structures with CMA-ES to tune numerical constants, using an object-aware forward-backward behavioral foundation model to repurpose existing skills.
• InterEvolve validates and evaluates candidate programs across parallel simulation scenarios and retains successful programs in a skill library for future tasks, enhancing efficient learning and adaptation.

InterEvolve is a test-time evolution framework for humanoid loco-manipulation that adapts a pretrained controller to tasks absent from its training distribution without retraining the controller. It evolves executable, staged reward programs rather than policy parameters: a LLM revises task strategies and reward structure from execution feedback, while CMA-ES tunes numerical constants. An object-aware forward-backward behavioral foundation model converts each reward into a latent behavioral prompt, allowing a frozen controller to repurpose and compose existing locomotion and object-interaction competence. Candidate programs are validated and evaluated across parallel simulation scenarios, and successful programs are retained in a skill library for subsequent tasks (Lin et al., 1 Oct 2026).

1. Conceptual foundations and problem formulation

InterEvolve addresses test-time adaptation for contact-rich, multi-stage tasks such as lifting, carrying, tipping, pushing, kicking, placing, and long-horizon composition. The target controller is assumed to possess a broad motor repertoire but not an explicit policy for every task. The central premise is that a new behavior can often be induced by specifying an appropriate reward interface that exposes and sequences existing competence.

The framework keeps the low-level controller fixed during task evolution. It evolves three elements:

1. Task strategies: the objectives to pursue, their temporal ordering, and their completion conditions.
2. Reward constants: weights, thresholds, tolerances, target values, kernel widths, and dwell times.
3. Accumulated experience: verified programs retained for later reuse.

The task is specified by a language request $\ell$ and scene context $c$, including object poses, targets, supports, and obstacles. The fixed controller is represented as

$$\pi(\boldsymbol a_t\mid \boldsymbol o_t,\boldsymbol z),$$

where $\boldsymbol o_t$ is the controller observation, $\boldsymbol a_t$ is the action, and $\boldsymbol z$ is a latent prompt inferred from the current reward.

Execution produces a body-object trajectory

$$\tau=(\boldsymbol s_0,\boldsymbol a_0,\ldots,\boldsymbol s_T),$$

where $\boldsymbol s_t$ is a privileged simulation state used by reward programs and verification. A fixed verifier $\mathcal C=\{C_i\}_{i=1}^{N_C}$ checks task success and physical constraints, including object proximity to a target, required contact, uprightness, object height, release, and final stillness. The verifier is generated once from the task and scene and remains fixed during program evolution, preventing an LLM from weakening the evaluation criteria.

Unlike reward-evolution methods that require retraining a policy for each proposed reward, InterEvolve evaluates a candidate reward immediately through a pretrained behavioral foundation model. Test-time computation is therefore allocated to program search and simulation rather than reinforcement-learning retraining.

The framework is related to several broader evolutionary paradigms. EVOC models cultural change through reciprocal interaction between individual cognition and population-level dynamics (0811.2551). REvolveR transfers policies through a sequence of intermediate robot morphologies rather than directly between dissimilar embodiments (Liu et al., 2022). PINSKY shows that rare inter-species policy transfer can create stepping stones in an emergent task curriculum (Dharna et al., 2022). InterEvolve differs from these systems by evolving task specifications for a fixed humanoid controller at deployment time.

2. Object-aware forward-backward behavioral foundation model

Forward-backward representations

The behavioral foundation model is based on a forward-backward (FB) representation. It factorizes the discounted future-state occupancy measure of a latent-conditioned policy:

$$M^{\pi_{\boldsymbol z}}(\mathrm d\boldsymbol s'\mid \boldsymbol s,\boldsymbol a) \approx F(\boldsymbol s,\boldsymbol a,\boldsymbol z)^{\top} B(\boldsymbol s')\,\rho(\mathrm d\boldsymbol s').$$

Here, $c$0 is the forward representation, $c$1 is the backward representation, $c$2 is a reference state distribution, and $c$3 indexes a behavior or policy. For a state reward $c$4, the value of a latent-conditioned policy is approximated by

$c$5

where

$c$6

A new reward therefore produces a new latent prompt rather than requiring a new policy. This reward-to-latent transformation is the principal mechanism that makes reward-program evolution computationally feasible.

Reward-inference bank

InterEvolve approximates the reward-dependent expectation using a fixed bank of body-object states,

$c$7

The bank is sampled from replay data used during multi-task, multi-object training and includes states associated with contact acquisition, transport, lifting, release, and whole-body object interaction. For each bank state, the system caches backward features and reward features. The bank is not updated with target-task rollouts.

For a reward $c$8, the basic latent estimate is

$c$9

where $$\pi(\boldsymbol a_t\mid \boldsymbol o_t,\boldsymbol z),$$0 rescales the result to the latent sphere used during pretraining. Positive global rescaling of all reward values does not affect the projected prompt; relative reward weighting is therefore the relevant quantity.

InterEvolve improves this aggregation through reward tilting:

$$\pi(\boldsymbol a_t\mid \boldsymbol o_t,\boldsymbol z),$$1

$$\pi(\boldsymbol a_t\mid \boldsymbol o_t,\boldsymbol z),$$2

The parameter $$\pi(\boldsymbol a_t\mid \boldsymbol o_t,\boldsymbol z),$$3 controls concentration on high-reward states. With $$\pi(\boldsymbol a_t\mid \boldsymbol o_t,\boldsymbol z),$$4, aggregation is uniform; moderate values emphasize relevant behaviors; excessive values can collapse the prompt onto too few states.

In one feet-only study using a 50k-state bank, success varied as follows:

$$\pi(\boldsymbol a_t\mid \boldsymbol o_t,\boldsymbol z),$$5	Success
0	0.0%
1	15.6%
3	35.9%
10	59.4%
30	28.1%

The decline at $$\pi(\boldsymbol a_t\mid \boldsymbol o_t,\boldsymbol z),$$6 demonstrates that excessive reward concentration can make the latent dependent on a small number of potentially unrepresentative states.

Stage-conditioned prompting

A reward program contains stage-specific rewards $$\pi(\boldsymbol a_t\mid \boldsymbol o_t,\boldsymbol z),$$7. At control time $$\pi(\boldsymbol a_t\mid \boldsymbol o_t,\boldsymbol z),$$8, the reward is evaluated over the entire bank:

$$\pi(\boldsymbol a_t\mid \boldsymbol o_t,\boldsymbol z),$$9

The stage prompt is

$\boldsymbol o_t$0

with

$\boldsymbol o_t$1

The prompt is recomputed at every control step, after which the frozen actor executes $\boldsymbol o_t$2. Rewards thus select behaviors from the pretrained repertoire rather than being optimized through test-time backpropagation.

Object-conditioned residuals

A body-only FB model may distinguish locomotion but fail to distinguish object outcomes such as pushing, lifting, tipping, or carrying. InterEvolve therefore makes the actor, forward representation, and backward representation object-aware. Each contains a frozen body branch and a trainable object-conditioned residual.

The actor mean is

$\boldsymbol o_t$3

For the G1 with rubber hands, the interaction encoding has 105 dimensions: 15 dimensions describe object position, orientation, linear velocity, and angular velocity, while 90 dimensions represent displacements from 30 body links to the nearest object-surface point. For the Inspire-hand model, 24 additional finger links are included, giving 54 links in total.

The residuals are trained once on OMOMO and GRAB human-object interaction data. The body prior preserves locomotion and whole-body competence, while the object residual provides interaction-specific conditioning. The controller outputs targets for 29 body joints; Inspire-hand control uses a low-dimensional grasp scalar per hand expanded into finger targets through software mimic coupling.

3. Reward programs and executable task structure

A reward program is defined as

$\boldsymbol o_t$4

where $\boldsymbol o_t$5 is the number of stages, $\boldsymbol o_t$6 is the reward for stage $\boldsymbol o_t$7, $\boldsymbol o_t$8 is its completion or transition condition, and $\boldsymbol o_t$9 contains tunable constants. A stage remains active until its transition condition becomes true, after which the program advances to the next stage. The final stage has no transition and runs until episode termination.

Component	Role	Example
Reward code $\boldsymbol a_t$0	Desired interaction	upward or horizontal object motion
Constants $\boldsymbol a_t$1	Numerical calibration	target height or contact threshold
Stages	Temporal decomposition	acquire, transport, release
Completion $\boldsymbol a_t$2	Transition logic	require support before release
Stage memory	Runtime context	displacement since stage entry

Programs can represent “lift, carry, place, and release,” “push, disengage hands, then kick,” or longer compositions involving multiple contact modes.

Feature namespaces

Program code accesses two types of quantities.

Bank features are tensors evaluated over the fixed state bank. Examples include object height, object velocity, object speed, soft hand-object contact, soft foot-object contact, local root velocity, and robot yaw rate. A representative contact feature is

$\boldsymbol a_t$3

Live context consists of values computed from the current rollout frame, including object-target distance, direction from object to target, target-facing status, current object height, resting height, hand-object distance, and episode progress. Live target information is therefore not treated as a property of the fixed bank states.

Programs are validated to use declared features, respect tensor dimensions, return bank-valued rewards, return Boolean transitions, and prevent transition conditions from reading bank features.

Lift–carry–release example

A two-stage program places a box on a $\boldsymbol a_t$4 m table. The first stage rewards raising and transporting the object:

$\boldsymbol a_t$5

$\boldsymbol a_t$6

$\boldsymbol a_t$7

$\boldsymbol a_t$8

The transition condition is

$\boldsymbol a_t$9

The second stage rewards release and settling:

$\boldsymbol z$0

Height is coupled to hand contact so that unrelated airborne states are not rewarded, and release is separated from transport rather than being merged into a single objective.

Verifiers

A verifier is fixed before program evolution. It may require object proximity to a target, a particular contact mode, uprightness, a minimum object height, release, and final stillness. This separation between reward and verification is central: reward programs may evolve, but the success criteria do not.

4. Test-time evolutionary search

LLM program revision

The LLM receives the task language, scene context, verifier, current best program, criterion-level rollout feedback, stage-level traces, and a skill library. It edits discrete and semantic aspects of the program:

• number of stages;
• reward terms;
• stage transitions;
• feature combinations;
• contact-mode changes;
• program composition;
• reuse of stored stages.

The LLM does not update its weights. Its adaptation occurs in context through execution outcomes. Targeted feedback may identify failed criteria, pass counts, median margins, tail failures, stage dwell times, object-height traces, target-distance traces, contact changes, and transitions that fired too early or too late.

CMA-ES constant optimization

For a fixed program structure, CMA-ES tunes $\boldsymbol z$1 within LLM-declared bounds. It adjusts reward weights, thresholds, target values, kernel widths, dwell times, and transition parameters.

The standard search uses $\boldsymbol z$2 candidates per generation, $\boldsymbol z$3 generations, and $\boldsymbol z$4 search scenarios. The rollout cost per candidate program is therefore

$\boldsymbol z$5

search rollouts.

Structural decisions are left to the LLM because numerical tuning cannot compensate for an incorrect temporal decomposition. The ablations show that CMA-ES alone improves constants but does not replace changes to staging, reward-term selection, or transition logic.

Multi-scenario evaluation and confirmation

Candidates and the current best program are evaluated on identical randomized initial conditions. The standard search grid contains 16 scenarios, with object bearing varying by approximately $\boldsymbol z$6 and distance by approximately $\boldsymbol z$7 m around nominal initialization.

Confirmation uses a separate 64-scenario grid with three repeats per scenario:

$\boldsymbol z$8

confirmation rollouts. A candidate is accepted only when its improvement exceeds run-to-run evaluation noise. Paired scenarios reduce confounding from different initial states, while confirmation rejects programs that succeed only under a narrow configuration.

Complete search cycle

The test-time loop is:

1. Generate an initial program from the task, scene, verifier, and skill library.
2. Evaluate it and designate it as the current best.
3. Prompt the LLM with the current best program and execution feedback.
4. Generate, parse, and validate candidate programs.
5. Tune each valid candidate’s constants with CMA-ES.
6. Evaluate candidates and the current best on shared search scenarios.
7. Select the strongest candidate.
8. Confirm it on separate scenarios with repeated rollouts.
9. Accept it only if improvement exceeds evaluation noise.
10. Produce criterion-level and stage-level feedback.
11. Stop when all verifier criteria are satisfied.
12. Add the final program and verifier results to the skill library.

The resulting process evolves task specifications rather than policies. The fixed controller provides the behavioral substrate, reward programs provide task-level control, simulation supplies feedback, and the skill library supplies persistent experience.

5. Skill-library retention and composition

The skill library $\boldsymbol z$9 is a text document containing verified programs from completed tasks. Each entry records the task and scene, verifier criteria, per-criterion pass rates, final success, program source, tuned constants, and reward-design lessons.

Reported reusable lessons include:

• multi-stage programs often benefit from a final release stage;
• release should reward low hand contact and object or robot stillness;
• lift rewards should be coupled to hand contact;
• different contact modes require different stage decompositions.

The library for eight tasks is approximately 28 KB in the prompt. Stored programs are not simply copied unchanged into new tasks. Their main value is to provide program structures, stages, and design lessons that the LLM can adapt and compose.

Three unseen composite tasks were evaluated:

1. relocating an object between supports;
2. stacking one box on another;
3. carrying, placing, and then kicking.

Library condition	Relocate	Stack	Carry–place–kick
Full library	8/10	4/10	5/10
No library	0/10	0/10	1/10
Carry-only library	6/10	1/10	1/10

In an additional carry–place–kick study, stored programs without search solved approximately 6.3% of cases; full library plus search achieved 60.9%; programs without lessons achieved 48.4%; lessons without programs achieved 21.9%; and no library achieved 12.5%. These results indicate that retention is most effective when combined with adaptation and composition.

Programs searched on a large box were also tested unchanged on other objects. Unchanged-program macro-averages were 37.5% for a plastic box, 35.9% for a small box, and 59.0% for a suitcase. After hand adaptation, the corresponding results were 78.5%, 88.7%, and 84.6%. Transfer therefore depends on object geometry, mass, friction, and task type.

6. Experimental results, deployment, and limitations

Experimental setting

The motor model is pretrained on OMOMO whole-body interaction data and GRAB grasping data. Experiments use a Unitree G1 with rubber hands and, in additional experiments, Inspire dexterous hands. The object set contains a large box, plastic box, small box, and suitcase. Simulation uses Isaac Lab, and the LLM is DeepSeek-V4-Flash.

The eight principal task families are:

1. push to a mark;
2. carry at mid height;
3. carry at chest height;
4. tip onto a new face;
5. push through a gate;
6. push around an obstacle;
7. kick to a mark with feet only;
8. lift to a support.

These tasks require distinct contact modes and temporal decompositions. Chest-height carrying requires coordinated lifting and transport; tipping requires rotation around a horizontal axis; gate pushing requires heading control; kicking requires hand disengagement and foot-object contact; obstacle pushing requires deviation and recovery.

Reference tracking

Reference-tracking metrics include mean body-joint error $$\tau=(\boldsymbol s_0,\boldsymbol a_0,\ldots,\boldsymbol s_T),$$0, mean object-surface error $$\tau=(\boldsymbol s_0,\boldsymbol a_0,\ldots,\boldsymbol s_T),$$1, and success rate (SR), defined as the percentage of clips that reach the end without falling, object deviation above $$\tau=(\boldsymbol s_0,\boldsymbol a_0,\ldots,\boldsymbol s_T),$$2 m, or loss of required contact.

Controller	$$\tau=(\boldsymbol s_0,\boldsymbol a_0,\ldots,\boldsymbol s_T),$$3 (cm)	$$\tau=(\boldsymbol s_0,\boldsymbol a_0,\ldots,\boldsymbol s_T),$$4 (cm)	SR
ULTRA	15.68	21.45	66%
Body-only BFM-Zero	28.36	62.33	8%
Object-aware model before reward evolution	27.20	30.80	60%
InterEvolve with evolved tracking reward	19.80	24.92	72%

Reward evolution improves the fixed object-aware controller sufficiently to exceed ULTRA’s success rate, although ULTRA retains lower mean body error. InterEvolve reports approximately 89% contact retention and no falls in the relevant rows. Object orientation error is 23.4° before evolution and 26.2° with evolving reward.

Object-anchored drift correction reduces median root drift from approximately 0.29 m at 4 seconds and 0.32 m at the end without correction to approximately 0.16 m and 0.18 m, respectively, with correction.

General-purpose task results

Across the eight task families:

Program condition	CMA-ES	Evolution	SR	Earned
Inaction	—	—	0.0%	49.1%
Human	no	no	8.0%	65.2%
Initial agent	no	no	32.2%	73.3%
Human	yes	no	18.0%	75.2%
Initial agent	yes	no	34.6%	82.5%
InterEvolve	yes	yes	86.5%	95.6%

The per-task success rates for human-designed rewards with CMA-ES, initial agent programs with CMA-ES, and InterEvolve are:

Task	Human + CMA-ES	Initial agent + CMA-ES	InterEvolve
Push to mark	96.9%	100.0%	100.0%
Carry at mid height	0.0%	40.6%	100.0%
Carry at chest height	0.0%	0.0%	100.0%
Tip onto new face	0.0%	0.0%	95.3%
Push through gate	0.0%	25.0%	73.4%
Push around obstacle	0.0%	4.7%	71.9%
Kick to mark	17.2%	37.5%	59.4%
Lift to support	29.7%	68.8%	92.2%
Macro-average	18.0%	34.6%	86.5%

The results show that numerical tuning alone is insufficient. Structural evolution of stages, reward terms, and transition logic is essential.

Improvement across evolution rounds

Stage	SR	Earned	Tokens
Initial	32.2%	73.3%	0.03 M
Round 1	75.0%	91.7%	0.08 M
Round 2	76.8%	92.6%	0.13 M
Round 3	73.6%	93.1%	0.18 M
Round 4	86.5%	95.6%	0.23 M

Earned criteria generally increase across rounds, whereas full success can temporarily decline because repairing one criterion may disrupt another.

Ablations

The full system achieves 86.5% SR and 95.6% earned criteria using approximately 2.1 GPU-hours and 0.23 million tokens.

Removed component	SR	Earned
CMA-ES tuning	51.6%	89.9%
Targeted edits	68.9%	92.3%
Multi-scenario evaluation	68.4%	89.9%
Scene context	78.7%	93.2%
Multi-stage programs	44.7%	86.9%

The largest degradation results from removing multi-stage programs. Removing CMA-ES also produces a substantial decline, demonstrating that structurally correct programs require numerical calibration. Targeted edits and multi-scenario evaluation improve robustness against ineffective or lucky revisions.

Competence unlocking and novel strategies

Coverage analysis compared evolved motions with 729 OMOMO large-box reference clips. Some behaviors remain close to the training distribution, including ground-height carrying and ordinary pushing. Others form distinct clusters, including lifting onto a table, chest-height carrying, kicking, tipping, and carry–place–kick compositions.

The median evolved rollout is at the 76th percentile of distance to the nearest reference clip. Lifting onto a table, chest-height carrying, and carry–place–kick are around the 98th percentile, suggesting that the method is not merely replaying training motions. Reported strategies include rotating a box around in place through modified whole-body posture, changing contact strategy across evolution rounds for kicking, stabilizing tracking with object-anchored drift correction, combining hand pushing with later foot kicking, and arranging six boxes in a ring through twelve phases from a single reset.

Sim-to-real deployment

Programs are evolved and verified in simulation, then executed on a physical Unitree G1. The perception pipeline uses an egocentric RGB-D torso camera, FastSAM and CLIP for box localization, FoundationPose for six-dimensional pose estimation, torso LiDAR, leg odometry, joint angles, and filtering based on floor and contact constraints.

During real execution:

1. The robot waits for a confident object estimate.
2. Proprioception and estimated object pose are provided to the controller.
3. The reward-program latent is recomputed at each control step.
4. The frozen controller produces commands at 50 Hz.
5. Execution pauses when perception becomes invalid.

Autonomous real-world demonstrations include evolved box-kicking and box-pushing programs. The paper does not report a comprehensive numerical real-robot success table; physical results are presented as autonomous demonstrations.

Computational cost

Each rollout lasts 700 control steps at 50 Hz. CMA-ES requires 192 search rollouts per candidate, and confirmation requires another 192 rollouts for the candidate and current best. A five-round search uses approximately 6.7–14.2 million environment steps, with an average of 9.5 million across the eight task families and approximately 1.9 million steps per round.

Typical per-round costs on one GPU are 1–2 minutes for the LLM and 16–34 minutes for simulation. Simulation is the dominant bottleneck. Five-round searches use approximately 0.13–0.18 million LLM tokens in the reported multi-round runs and 0.23 million tokens in the full benchmark configuration. Reward projection with a 50k-state bank and $$\tau=(\boldsymbol s_0,\boldsymbol a_0,\ldots,\boldsymbol s_T),$$5 requires approximately 0.92 ms and 63 MiB in the reported diagnostic.

Limitations

InterEvolve cannot reliably create a motor capability absent from the pretrained controller. Kicking remains the most difficult task because it is rare in the pretraining data and requires strong whole-body coordination.

The reward-inference bank constrains the behaviors reachable through reward projection. If relevant states are absent, an appropriate reward may still fail to induce the desired behavior. Excessive bank size or reward tilting can also be harmful; a 200k-state bank with $$\tau=(\boldsymbol s_0,\boldsymbol a_0,\ldots,\boldsymbol s_T),$$6 performed worse than a 100k-state bank because the tilted weights became too concentrated.

Programs can be sensitive to object geometry, mass, friction, and scene configuration. The system also depends on fast parallel simulation, currently through Isaac Lab, and does not update controller weights from accumulated task experience.

LLM-generated programs may have unsuitable decompositions, premature transitions, incorrect contact objectives, or overfitting to a small number of scenarios. Validation, CMA-ES, paired scenarios, confirmation evaluation, and targeted feedback mitigate these failures but do not eliminate them. Physical deployment additionally depends on six-dimensional pose estimation, occlusion handling, filtering delays, sim-to-real dynamics, hardware limits, and safe whole-body contact.

InterEvolve is therefore best characterized as a test-time task-specification evolution system rather than a method for universal policy learning. Its principal contribution is the combination of a frozen object-aware behavioral foundation model, executable staged reward programs, LLM-guided structural revision, numerical optimization, scenario-based verification, and persistent skill-library reuse. This architecture separates long-timescale acquisition of motor competence from short-timescale adaptation of task strategies, enabling a humanoid controller to expose, compose, and retain behaviors that were not explicitly represented as individual skills during pretraining.