Bilevel System Prompt Optimization

• Bilevel system prompt optimization is a hierarchical framework that optimizes a global system prompt at the upper level while refining task-specific user prompts at the lower level.
• It employs methods like meta-learning, iterative joint optimization, and Bayesian optimization to enhance prompt efficiency and robustness across diverse tasks.
• Empirical results show significant performance gains and data efficiency improvements, although challenges with convergence and compute demands remain.

Bilevel system prompt optimization refers to the hierarchical optimization of LLM prompts, where the system prompt (task-agnostic, global prompt) is optimized at the upper level, and user prompts (task-specific instructions) are optimized at a lower level. This framework is crucial for producing robust and effective system prompts that generalize across diverse user prompts and tasks, addressing limitations of prior work that typically optimized only user prompts for specific, isolated tasks (Choi et al., 14 May 2025). Recent developments have formalized and addressed this problem using meta-learning, Bayesian optimization, evolutionary algorithms, and sensitivity-based methods.

1. Formal Problem Definition

In bilevel system prompt optimization, the LLM input is decomposed as $x = [s; u; q]$, where $s \in \mathcal{S}$ is the system prompt, $u \in \mathcal{U}$ is the user prompt, and $q$ is the query. The model's output is $\hat{y} = \mathrm{LLM}(s, u, q)$. The key objective is to compute a system prompt $s^*$ that, when paired with per-task optimized user prompts $u_i^*$, maximizes expected performance $f(\cdot, \cdot)$ over a task distribution $\mathcal{T}$:

• Lower-level problem (user prompt optimization for each task $T_i$):

$$u_i^* = \arg\max_{u \in \mathcal{U}} \mathbb{E}_{(q,y)\sim T_i}[f(\mathrm{LLM}(s, u, q), y)]$$

• Upper-level problem (system prompt optimization):

$$s^* = \arg\max_{s \in \mathcal{S}} \mathbb{E}_{T_i \sim \mathcal{T}}\Big[ \mathbb{E}_{(q,y)\sim T_i}[f(\mathrm{LLM}(s, u_i^*(s), q), y)] \Big]$$

This structure captures the nested dependence of optimized user prompts on the current system prompt, and positions system prompt optimization as a meta-level generalization challenge over diverse, lower-level adaptations (Choi et al., 14 May 2025, Zhang et al., 21 Jul 2025).

2. Principal Methodologies

A variety of algorithmic frameworks have been proposed for efficiently solving bilevel system prompt optimization:

Meta-Learning (MetaSPO)

MetaSPO frames system prompt optimization as a two-loop meta-learning process (Choi et al., 14 May 2025):

• Inner loop: For each task, iteratively refine user prompts using LLM-based analysis (Analyzer and Generator) on failure cases.
• Outer loop: Aggregate failures across optimized user prompts, critique the current system prompt, and use LLMs to generate improved system prompt candidates, selecting the empirically best variant across tasks.

Iterative Joint Optimization (P³)

P³ alternates between inner-level user-prompt search (candidate generation, scoring, and refinement per query/task) and outer-level system-prompt improvement, focusing updates on "hard" cases where user prompt optimization fails (Zhang et al., 21 Jul 2025).

• The system prompt is selected to maximize performance on queries that were challenging for current user-prompt strategies, ensuring robustness across a variety of user prompt adaptations.

Information-Theoretic Bayesian Optimization

This method treats both system and user prompt optimization as expensive black-box functions. Gaussian Processes model each level, and a novel acquisition function quantifies information gain about both levels' optima, guiding sample-efficient selection of new system/user prompt pairs (Kanayama et al., 26 Sep 2025).

Evolutionary Surrogates (BLEAQ)

BLEAQ employs a quadratic surrogate to approximate the lower-level (user prompt) optimization mapping, substantially reducing the computational burden of repeated lower-level solves in a population-based upper-level (system prompt) evolutionary optimization loop (Sinha et al., 2013).

Sensitivity-Based Implicit Gradients

Sensitivity-based methods treat the lower-level solution as an implicit mapping, constructing outer-level updates using implicit differentiation and robust augmented Lagrangian techniques. This enables convergent first-order optimization for upper-level (system prompt) parameters, incorporating constraints as needed (Nolasco et al., 1 Oct 2025).

3. Empirical Performance and Evaluation

MetaSPO, P³, and related methods have demonstrated substantial gains over classical baselines (static prompts, Chain-of-Thought, commercial system prompts, and genetic optimization baselines like SPRIG) across a broad suite of LLM benchmarks:

Setting	Default	CoT	SPRIG	MetaSPO	P³
Unseen domains (MetaSPO, avg)	32.2	33.2	37.0	44.5	—
Global (all domains)	32.2	—	35.0	41.4	—
Test-time user-prompt tuning	51.1	—	52.5	54.3	—

MetaSPO reduced the number of user prompt optimization iterations and data required to reach baseline performance by 80% and 75%, respectively. P³ achieved multi-point gains over best alternatives (PAS, BPO) on general QA (Arena-Hard, Alpaca-Eval 2.0), and significantly outperformed all baselines on GSM8K and GPQA reasoning tasks, especially on smaller models (Choi et al., 14 May 2025, Zhang et al., 21 Jul 2025).

4. Theoretical Properties and Analysis

Most bilevel system prompt optimization frameworks (MetaSPO, P³) are derivative-free and operate in discrete prompt spaces using LLMs as oracle optimizers and evaluators. Classical convergence proofs do not apply: algorithm stability is empirically observed but not formally established (Choi et al., 14 May 2025, Zhang et al., 21 Jul 2025).

Population-based and meta-learning approaches justify their hierarchical synergy by analogy to MAML-style adaptation, decoupling generalization across tasks (system prompt) from per-task adaptation (user prompt). Information-theoretic Bayesian optimization explicitly models information gain about both lower- and upper-level optima for joint efficiency (Kanayama et al., 26 Sep 2025).

In contrast, continuous optimization frameworks (BLEAQ, sensitivity-based, and single-step adjoint schemes) provide formal convergence guarantees under smoothness and convexity for continuous prompt or embedding spaces (Nolasco et al., 1 Oct 2025, Suonperä et al., 2022, Sinha et al., 2013).

5. Implementation Paradigms and Practical Considerations

• Prompt Representation: MetaSPO and P³ operate on hard discrete prompt tokens, relying exclusively on black-box LLM evaluations and candidate generation via LLM in-context learning.
• Optimizer LLMs: The size and quality of the LLM acting as Analyzer/Generator or prompt optimizer critically affect performance; larger models yield better system prompts, but strong gains are also possible with moderately-sized (e.g., Llama3.1 8B) optimizers (Choi et al., 14 May 2025).
• Search Efficiency: Both population-based surrogates and meta-learning frameworks balance robustness and efficiency via sample-efficient exploration (e.g., BLEAQ's surrogate or P³-ICL's retrieval-based online adaptation).
• Hyperparameter Sensitivity: Performance is sensitive to candidate sizes, search depths, and system-update intervals. Adaptive selection and meta-learning of these parameters remains an area for improvement (Zhang et al., 21 Jul 2025).
• Data and Compute Efficiency: MetaSPO and P³-ICL achieve significant reductions in data and model size requirements at inference (P³-ICL, for example, shrinks online optimizer footprint by nearly three orders of magnitude while preserving gains) (Zhang et al., 21 Jul 2025).

6. Cross-Domain Transfer, Synergy, and Limitations

Meta-learned system prompts generalize robustly to novel tasks and domains, with performance improving up to 3 source domains in transfer evaluations. Outer-only optimization underperforms setups where inner and outer (user/system) prompts are jointly tuned, highlighting the necessity of full bilevel coupling (Choi et al., 14 May 2025, Zhang et al., 21 Jul 2025).

Concise or summarized system prompts underperform their verbose, base counterparts, suggesting that transferability depends on prompt expressivity as well as content (Choi et al., 14 May 2025).

Known limitations include:

• Absence of formal convergence guarantees in discrete, black-box LLM prompt optimization.
• Residual sensitivity to the quality and scale of the optimizer LLM.
• Substantial offline compute requirements from repeated LLM-based evaluations and candidate generation.
• Potential susceptibility to adversarial prompt crafting, necessitating alignment and safety research for robust system prompt deployment (Choi et al., 14 May 2025).

7. Future Directions

Key directions for advancing bilevel system prompt optimization include:

• Development of continuous relaxation approaches for gradient-based bilevel solvers in prompt tuning (e.g., soft-prompt optimization, differentiable reward proxies).
• Meta-learning frameworks that adapt key hyperparameters and optimization schedules per task.
• Formal analysis of sample complexity and convergence in discrete, black-box, or oracle-based bilevel settings.
• Extensions to multimodal or dialog settings, where prompt components interact more richly.
• Investigation of safety and alignment, particularly defending against adversarially crafted system prompts and establishing principled guardrails (Choi et al., 14 May 2025, Zhang et al., 21 Jul 2025, Nolasco et al., 1 Oct 2025).

Bilevel system prompt optimization constitutes a rapidly evolving framework with wide-reaching implications for efficient, robust, and generalizable steering of LLMs across heterogeneous and previously unseen tasks (Choi et al., 14 May 2025, Zhang et al., 21 Jul 2025, Kanayama et al., 26 Sep 2025, Sinha et al., 2013, Suonperä et al., 2022, Nolasco et al., 1 Oct 2025).