EEVEE: Towards Test-time Prompt Learning in the Real World for Self-Improving Agents

Abstract: In this paper, we propose EEVEE, the first multi-dataset test-time prompt learning framework for LLM agents, enabling test-time prompt learning under real-world task streams. Existing methods are largely designed for single-dataset settings, while real-world applications require models to handle heterogeneous input streams drawn from multiple datasets, domains, and task distributions, limiting their practical applicability. To mitigate cross-dataset interference, EEVEE introduces a router that partitions incoming inputs into task clusters and assigns them to suitable prompt configurations. This design is optimized via a router-prompt co-evolution strategy, which employs interleaved router and prompt learning phases to address their mutual dependency. Experiments across multiple datasets demonstrate that the framework improves robustness under heterogeneous data streams while maintaining single-benchmark learning capability and efficiency. Specifically, EEVEE improves average multi-benchmark scores by 10.38 and 24.32 points over Qwen3-4B-Instruct and DeepSeek-V3.2, surpassing SOTA methods GEPA and ACE by up to 37.2% and 48.2%.

• The paper introduces Eevee, a framework that co-evolves a router and specialized prompts to enable robust test-time adaptation across diverse datasets.
• It demonstrates significant performance gains in multi-benchmark evaluations, mitigating cross-dataset interference and enhancing retention.
• Empirical results highlight improved accuracy, token efficiency, and generalization compared to state-of-the-art baseline methods.

Eevee: Multi-Dataset Test-Time Prompt Learning for Self-Improving LLM Agents

Motivation and Problem Formulation

Test-time prompt learning is an emerging paradigm for adapting LLMs post-deployment, leveraging iterative prompt updates to extract improved downstream performance without model weight modification. However, contemporary methods operate almost entirely in stationary, single-dataset settings, which is highly restrictive given that real-world input streams are heterogeneous, exhibiting diverse domains, formats, and supervision regimes. Across such distributions, naive prompt adaptation results in cross-dataset interference: adaptation to one distribution often undermines performance on another due to aliasing and mutual contamination in the learned prompt. This phenomenon is quantifiable through the cumulative negative retention observed as more tasks are introduced in multi-benchmark adaptation scenarios, where single-prompt methods exhibit pronounced performance degradation (Figure 1).

Figure 1: Incremental multi-benchmark retention improvement as tasks are added, demonstrating negative retention in baselines and consistent cumulative gain in Eevee.

The paper introduces Eevee, a framework for real-world, multi-dataset test-time prompt learning. Eevee proposes to decouple task generality from task specificity by learning (1) a router that partitions input streams into coherent, behaviorally meaningful clusters, and (2) a set of specialized prompts targeted for each cluster. Critically, Eevee does not fix either component but rather co-evolves them through a joint router-prompt optimization protocol, allowing behavioral specialization and generalization to emerge synergistically.

Eevee Framework

At the core of Eevee is a co-evolutionary loop, wherein the routing policy $R$ and prompt set $\mathcal{P}$ are alternately and iteratively refined. For a fixed model $M$, Eevee routes each input $x$ through $R$, selects a prompt $p_{z}$, and produces the output $\hat{y}=M(x; p_z)$. The crucial departure from prior work is that routing is not static: both the router and prompt set are dynamically optimized using evolutionary strategies that explicitly analyze error attribution, reflect on downstream failures, and maintain Pareto-frontier prompt pools.

Figure 2: Main framework of Eevee, illustrating routing, cluster assignment, and router/prompt co-evolution.

The co-evolution protocol alternates between:

• Router evolution: For a fixed prompt set $\mathcal{P}_T$, a pool of candidate routers is stochastically mutated. Router fitness is measured by downstream accuracy, coverage balance, and behavioral consistency, evaluated only on examples where slot-wise prompt capacity is non-trivial. Secondary LLM reflection is applied to identify cases where alternative routing would repair failed outputs.
• Prompt set evolution: For a fixed router $R_{T+1}$, input clusters induce slot-specific training groups. Prompt optimization uses mutation and reflection on mini-batches, retaining only Pareto-optimal prompts with respect to held-out correctness.

Eevee’s three-stage training comprises (i) prompt initialization via greedy coverage from Pareto pools, (ii) fast exploratory co-evolution with annealed optimizer weights (favoring coverage/consistency early, accuracy late), and (iii) convergence under a stable router, where slot prompts are fully optimized.

Figure 3: Three-stage training and greedy coverage for prompt slot initialization and subsequent co-evolution.

Empirical Evaluation

Multi-Benchmark Test-Time Adaptation

Evaluated on GPQA Diamond, Formula, TheoremQA, and HumanEval, Eevee demonstrates robust advances in average score, surpassing Qwen3-4B-Instruct by +10.38 and DeepSeek-V3.2 by +24.32 points, and outperforming SOTA test-time prompt learning baselines GEPA and ACE by 37.2% and 48.2%, respectively. Notably, cumulative retention remains positive as the benchmark suite expands: Eevee achieves +41.53 retention while GEPA and ACE experience net regressions (Figure 1).

Single-Benchmark Performance and Task Scaling

On individual tasks, Eevee remains competitive, matching or exceeding baselines even when router-based specialization is less consequential. The advantage is accentuated as the number of benchmarks increases. When comparing with FiNER and IFBench, Eevee achieves up to 73.17 on HumanEval and 55.25 on Formula.

Figure 4: Single-benchmark performance across key datasets, displaying parity or gains over baselines.

Ablation and Router–Prompt Coupling

Ablation studies indicate default or manual routers are markedly inferior, introducing up to -4.19 average point decrease. Eliminating co-evolution and using sequential learning substantially reduces gains, confirming that interdependency between routing and prompt specialization is critical for effective adaptation.

Generalization: Cross-Model and Cross-Task

Prompt sets learned on one LLM (Qwen3-4B-Instruct) retain their effect when ported to DeepSeek-V3.2, yielding a +14.35 point increase over the DeepSeek baseline. For held-out benchmarks distinct from in-benchmark domains (MBPP, MMLU-Pro), Eevee exhibits marginal impact (±1.8), outperforming baselines in relative degradation restraint.

Token Efficiency

Eevee’s architectural modularity does not incur excessive inference expense. Average per-example token usage after adaptation is 4.32k, significantly lower than ACE’s 21.3k, and comparable to GEPA (Figure 5).

Figure 5: Average per-test-example token usage for each method, showing Eevee's efficiency advantage over large-context baselines.

Case Study: Error Analysis and Adaptation Phenomenology

Comparative diagnosis reveals that Eevee’s improvements are concentrated in tasks where feedback can be distilled into reusable computational routines or answer contracts, such as code completion and financial formulae. In contrast, fact-centric QA like GPQA Diamond can see negative flips, as the router and prompts reinforce generic reasoning at the expense of rare domain-specific priors. Empirically, code and formula tasks enjoy +23.2 to +48.8 percentage-point improvements, whereas GPQA can exhibit net regressions.

Relation to Prior Work

While prior reflection-based prompt learning frameworks (e.g., GEPA, ACE, Combee) enhance adaptation within static settings, they lack architectural tools for decomposing heterogeneous input streams and cannot jointly specialize and generalize via modular prompt routing. Memory extraction and agentic context evolution methods similarly fall short in the multi-dataset, real-agent adaptation regime. Evolution-inspired works employ co-evolution strategies, yet are typically specialized to search within bounded program or skill spaces rather than test-time behavior induction in LLM agents.

Implications and Future Directions

Eevee constitutes a significant progression toward robust, self-improving LLM agents operating under non-stationary, real-world task distributions. Practically, its router–prompt set modularity enables sample-efficient learning and mitigates catastrophic forgetting driven by task interference. Theoretically, the approach underscores the importance of structured modular adaptation surfaces and joint optimization under mutual inductive coupling.

For future research, anticipated directions include:

• Relaxing the requirement for rule-based or ground-truth feedback, progressing toward reflection-only test-time adaptation
• Expanding the expressivity and interpretability of router architectures, potentially via neural or programmatic routing
• Extending the framework to online streaming, with continual adaptation to tempo-spatial shifts in input data and emergent tasks
• Coupling router specialization with automated task discovery and meta-learning to further enhance agent autonomy

Conclusion

Eevee provides a scalable, practical, and empirically validated framework for test-time prompt learning in the presence of heterogeneity and cross-dataset interference. By jointly evolving routers and prompt sets, it achieves strong multi-task retention, efficiency, and generalization, offering a foundational advance toward autonomous, self-improving agentic LLMs operating reliably in complex, dynamic environments.