Boundary between prompt-based and weight-based learning for compound AI systems

Characterize the boundary between reflective prompt evolution (GEPA) and weight-space reinforcement learning methods (such as Group Relative Policy Optimization with LoRA or full-parameter finetuning) for optimizing compound AI systems, by determining the data and rollout regimes under which prompt-based optimization versus weight updates are expected to outperform one another.

Background

The paper introduces GEPA, a reflective prompt evolution algorithm that optimizes multi-module LLM systems using natural language feedback and Pareto-based candidate selection. Across multiple tasks, GEPA is shown to outperform GRPO (a reinforcement learning algorithm) with far fewer rollouts, highlighting substantial sample-efficiency advantages for prompt-based optimization.

Despite these results, the authors note that when training data or rollout budgets are abundant, standard weight-space reinforcement learning may surpass prompt-based methods. They explicitly state that the boundary between these learning paradigms is not well understood, motivating a precise characterization of when each approach should be preferred for compound AI systems.

References

The boundary between prompt-based and weight-based learning is not well understood—although GEPA excels when rollouts are expensive, it is likely that weight updates will outperform prompting in regimes with abundant data or when large-scale rollouts are feasible.

— GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning  (2507.19457 - Agrawal et al., 25 Jul 2025) in Limitations and Future Work

Although these studies report gains in their respective domains, it remains unclear how the two levers should be coordinated once a harness has become specialized to a particular model, and under what conditions subsequent weight updates will compound rather than disrupt that fit.

— Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails  (2609.09134 - Yu et al., 8 Sep 2026) in Section 2, Related Work

We improve the Environment Designer with gradient updates, but co-adaptation between the Environment Designer and Reasoning Agent might also come from an Environment Designer that improves without weight updates, accumulating and refining design strategies from past attempts in context; whether learned weights or in-context evolution makes the better designer, and at what scale, is an open question.

— SPADE: Self-Play in Adaptive Synthetic Executable Environments  (2608.19197 - Liu et al., 19 Aug 2026) in Section Discussion, Future directions

We state this as an interpretation of the observed regime, not a law: with hundreds of shots the balance may move toward weight space, and we have not measured where.

— Your Model Already Knows Don't Teach It, Learn to Ask It: Soft Prompting for Few-Shot Adaptation of Vision-Language Models  (2609.11310 - Gare et al., 10 Sep 2026) in Section Discussion, paragraph beginning “Why does asking match teaching at 10 shots?”