---
title: Declarative Prompt Optimisation
url: https://www.emergentmind.com/topics/declarative-prompt-optimisation
type: topic
---

# Declarative Prompt Optimisation

Declarative Prompt Optimisation denotes a family of methods that move prompt design away from ad hoc string editing and toward explicit specifications of prompt structure, behavior, evaluation, or resource allocation. In this literature, prompts are variously treated as typed programs, componentized schemas, segment-annotation pairs, system–user composites, declarative governance blocks, or context-allocation queries; optimization then operates over those structured objects rather than over undifferentiated text. The field therefore spans prompt programming languages such as PDL and SPL, declarative module compilers such as DSPy, structured search systems such as DelvePO, HAPO, and PSAO, trajectory-level prompt rewriting such as RPO, and merit- or preference-guided optimizers such as MePO and P3 [2410.19135] [2602.21257] [2310.03714] [2510.18257] [2601.02683] [2605.14561] [2510.05921] [2505.09930] [2507.15675].

## 1. Conceptual scope

The term *declarative* is used in this area with a broader meaning than in classical programming languages. Across the recent literature, it usually refers to specifying **what** a prompt, workflow, or generation context should contain or satisfy—roles, constraints, output schemas, memory sources, postchecks, or optimization metrics—while delegating **how** those specifications are realized to a runtime, compiler, optimizer, or LLM-mediated search procedure. PDL explicitly frames this as declarative specification of prompt content, data flow, and orchestration in YAML, while DSPy frames it as signatures and modules that state the transformation to be performed rather than the exact prompt wording [2410.19135] [2310.03714].

The same term also covers lighter-weight control interfaces. Prompt Decorators define a declarative behavioral layer through compact control tokens such as `+++Reasoning`, `+++Tone(style=formal)`, and `+++OutputFormat(format=markdown)`, with the explicit goal of separating task content from execution behavior [2510.19850]. NLD-P similarly reconceptualizes prompting as a modular governance method that separates provenance, constraint logic, task content, and post-generation evaluation in natural language, without relying on external orchestration code [2602.22790]. SPL extends the declarative idea to context management: it treats the context window as a constrained resource and prompt construction as a query-planning problem over memory, retrieval, and input sources [2602.21257].

A narrower but influential sense of declarativity appears in optimization systems that impose explicit prompt structure before rewriting. DelvePO decomposes prompts into named semantic components such as role, task description, output format, workflow, and examples, then performs evolutionary search over those components rather than over whole prompt strings [2510.18257]. PSAO fixes the base prompt and optimizes segment-level annotations attached to prompt segments, while HAPO edits prompt semantic units selected by attribution rather than regenerating the entire prompt [2605.14561] [2601.02683]. DPT, in a multimodal setting, makes the downstream task itself more declarative by rewriting questions into declarative cloze statements and aligning VQA objectives with MLM and ITM pretraining [2205.02456].

Taken together, these works suggest that declarative prompt optimisation is not one algorithmic family but a design stance: prompt behavior is made explicit, structured, inspectable, and therefore more amenable to systematic optimization.

## 2. Representations of prompts, workflows, and control

A major strand of the field concerns the **representation** on which optimization operates. PDL provides a deliberately simple YAML-based representation in which every program is a valid YAML document following the PDL schema, and execution proceeds by accumulating a background context of generated data. Its structured block schema unifies prompt text, variable binding, role assignment, context contribution, parsing, and typing through fields such as `role:`, `contribute:`, `parser:`, and `spec:` [2410.19135]. The compliance-agent case study emphasizes the practical consequence: prompts, model calls, tool schemas, deterministic code, and control flow become a single explicit artifact that can be inspected, modified, and optimized, rather than being hidden in framework internals [2507.06396].

DSPy uses a different representation: signatures, modules, and imperative computation graphs. A signature declares a transformation such as `"question -> answer"` or `"context, question -> search_query"`, while modules such as `Predict`, `ChainOfThought`, `ReAct`, and `Retrieve` realize prompting strategies over those signatures. The program structure is ordinary control flow, but LM calls are invoked through declarative modules whose parameters—especially demonstrations—can be compiled and improved automatically [2310.03714]. The later DSPy use-case study pushes the same representation toward applied prompt optimization, treating prompts as code-like artifacts with typed signatures, compositional modules, and optimizer-driven instruction and demonstration search [2507.03620].

Other systems expose more explicit control layers. Prompt Decorators use the canonical syntax `+++Name(optional_parameters)` and define twenty core decorators across Cognitive & Generative and Expressive & Systemic families, including reasoning, debate, planning, tone, output formatting, scope, and export controls [2510.19850]. NLD-P uses structural blocks such as `IDENT`, `RULE`, `CONTENT`, `INPUT`, and `POSTCHECK`; its minimal compliance criteria require explicit separation of control logic from task instructions, natural-language behavioral constraints, predeclared post-generation validation, and targeted revisability [2602.22790]. SPL adopts a SQL-inspired representation with `PROMPT`, `WITH BUDGET`, `SELECT`, `GENERATE`, `WHERE`, `ORDER BY`, `LIMIT`, CTEs, `EXPLAIN`, and `STORE RESULT IN memory`, thereby turning prompt assembly into a declarative query over context sources [2602.21257].

Structured representations also appear inside optimizers. DelvePO represents a prompt as a template populated by component values such as **Role**, **Task description**, **Output format**, **Workflow**, and **Examples**, marked with HTML-style component tags [2510.18257]. PSAO represents prompts as \(P_{S,A} = \{(s_i,a_i)\}\), where \(S\) is a segmentation and \(A\) an annotation assignment over segment-level descriptors such as importance, context, intent, and priority [2605.14561]. HAPO represents prompts as semantic-unit decompositions \(S(p)=\{u_k\}\), and MePO represents prompt quality through an explicit merit schema—clarity, precision, concise chain-of-thought, and preservation of original information—even though the final optimizer is a neural rewrite model [2601.02683] [2505.09930].

These representations matter because they determine what can be optimized. A prompt programming language exposes parse trees, types, and execution traces; a componentized optimizer exposes mutable semantic slots; a governance schema exposes revision targets; a query language exposes token budgets and context sources. The declarative layer is therefore not incidental metadata but the search space itself.

## 3. Optimization mechanisms

Once prompts are structured, optimization methods differ mainly in the **kind of search or learning signal** they apply. DSPy treats optimization as compilation. Teleprompters such as `BootstrapFewShot`, `BootstrapFewShotWithRandomSearch`, and `BootstrapFinetune` run the program, capture successful traces of internal module calls, and convert those traces into demonstrations or finetuning data, optimizing the compiled program against a user-specified metric rather than a hand-written prompt template [2310.03714]. The later DSPy case study uses `BootstrapFewShotWithRandomSearch`, `MIPROv2`, `InferRules`, and a custom `CustomMIPROv2`, showing how declarative signatures and modules support joint search over instruction wording, demonstrations, and sometimes reasoning structure [2507.03620].

Several systems perform explicit discrete search over structured prompt edits. DelvePO formulates prompt optimization as a population-based search over prompts for a task \(T=(D,A)\), with roulette-wheel sampling, component-aware mutation and crossover, and two memory structures: component memory for before/after value pairs and prompt memory for high-performing prompts. Direction selection occurs before rewriting, so evolution focuses on promising component types rather than perturbing the whole prompt at random [2510.18257]. HAPO formalizes prompt optimization as
\[
p^*=\arg\min_{p\in\mathcal{P}} \mathbb{E}_{(x,y^*)\sim D}\big[\mathcal{L}(f(p,x),y^*)\big],
\]
then approximates a linguistic gradient through semantic-unit attribution, history-aware smoothing, local edit operators, UCB-based edit selection, and an explicit drift metric that measures how often prompt updates break previously solved examples [2601.02683]. PSAO instead defines a joint search over segmentations and annotations,
\[
(S^*,A^*)=\arg\max Q(M(P_{S,A})),
\]
with the original prompt retained as a neutral candidate so that optimization can, in principle, preserve baseline performance [2605.14561].

A second family uses **trajectory- or history-based rewriting**. RPO rewrites the task instruction prompt of an LLM agent using textual feedback derived from multi-turn trajectories. Its basic update is
\[
prompt^{i+1}=LLM_R(prompt^i, feedback^i),
\]
and its replay variant conditions on earlier prompt-feedback pairs as well. The method distinguishes Monte Carlo-style end-of-trajectory feedback from Temporal-Difference-style turn-level feedback, and its central claim is that long-term planning in multi-turn tasks improves when prompt rewriting is guided by temporally grounded critique rather than only by final outcomes [2510.05921]. P3 likewise separates offline and online optimization but does so over prompt components: a global system prompt and query-specific user-prompt complements. Its online formulation is
\[
y=LLM(x_s^*, f(x_u\mid X_u^*)),
\]
where \(x_s^*\) is the offline optimized system prompt and \(f\) generates or retrieves a query-dependent complement using the offline prompt dataset \(X_u^*\) [2507.15675].

A third family optimizes via **preference learning or surrogate evaluation**. MePO defines interpretable prompt merits, constructs a Prompt Optimization Preference dataset of \((P_{\text{silver}}, P_{\text{golden}}, R_{\text{silver}}, R_{\text{golden}})\) tuples, and trains a Qwen2.5-7B-based optimizer with DPO so that merit-aligned rewrites are preferred over raw prompts [2505.09930]. The legal-text classifier paper uses Monte Carlo Tree Search over textual-gradient edits and a proxy prompt evaluator \(\phi\) trained to predict whether a prompt-clause-label triple will be classified correctly; this lets prompt quality be approximated without repeatedly querying the target LLM across a large score set [2510.08524].

Finally, some works optimize by **objective reformulation** rather than by search over instructions. DPT rewrites VQA questions into declarative masked statements and replaces the usual answer-classification objective with MLM- and ITM-style scoring. The total loss is
\[
\mathcal{L}_{DPT}=\mathcal{L}_{MLM}^{VQA}+\mathcal{L}_{ITM}^{VQA},
\]
so declarative prompt optimization here means recasting the task into a form that better matches pretraining objectives rather than iteratively rewriting instructions [2205.02456].

## 4. From prompt engineering to prompt programming

Declarative prompt optimisation is closely tied to the emergence of **prompt programming systems**. In PDL, prompt programs are themselves structured data, which the paper explicitly describes as amenable to program transformations “such as for optimization.” PDL’s interpreter supports streaming, sandboxing, VSCode tooling, Jupyter integration, and a live visualizer that exposes the exact execution trace and accumulated context, making prompt failures inspectable at the level of the program rather than the final output alone [2410.19135]. The compliance-agent case study demonstrates why this matters: a canned ReAct-style architecture in CrewAI was restructured in PDL by splitting one mixed reasoning-plus-JSON step into separate natural-language and structured-output stages, combined with a custom response parser, yielding large gains without changing the underlying models or tools [2507.06396].

DSPy takes the same progression further by treating prompt optimization as compilation of declarative LM calls into self-improving pipelines. Its central abstractions—signatures, modules, metrics, and teleprompters—turn prompt engineering into a pipeline design problem. Instead of hand-writing long prompt strings, users declare the transformations required, such as retrieval query generation or answer synthesis, and the compiler bootstraps demonstrations, reasoning traces, or even finetuned students from successful program executions [2310.03714]. The applied DSPy study extends this compilation view to practical systems such as guardrail enforcement, code hallucination detection, router agents, and contradiction evaluation, reinforcing the idea that prompt optimization is often pipeline optimization rather than string optimization [2507.03620].

SPL makes this connection explicit by introducing a full optimizer stage between semantic analysis and execution. Its core budget model,
\[
\sum_{i=1}^{n} tokens(s_i)+tokens(g)+\beta \leq B,
\]
treats context assembly as an optimization problem over token allocations. The runtime computes an `ExecutionPlan`, supports `EXPLAIN` plans analogous to SQL, integrates RAG and memory sources into `SELECT`, and extends naturally to chunked Map–Reduce style prompting through CTEs [2602.21257]. Prompt optimization thus becomes context-query optimization: deciding how much of each source to include, in what order, at what cost, and under what model.

A governance-oriented variant appears in NLD-P. Instead of program compilation or search, NLD-P structures the prompt lifecycle into authoring, execution, evaluation, and revision. It is explicitly architectural rather than benchmark-driven, but it supplies a declarative control abstraction in which prompt revisions target discrete governance components rather than an undifferentiated text block [2602.22790]. Prompt Decorators occupy a middle position: they provide a composable syntax and a six-stage processing pipeline—parsing, scope resolution, planning and interaction, reasoning and generation, formatting and expression, introspection and export—but rely on either model compliance or middleware to make that pipeline operational [2510.19850].

This systems perspective broadens the meaning of declarative prompt optimisation. The optimized object may be a prompt string, but it may equally be a compiled LM program, a staged tool-use workflow, a context-allocation plan, or a governance template with explicit revision surfaces.

## 5. Empirical evidence and application domains

The empirical literature is heterogeneous, but several patterns recur. DelvePO reports broad task-agnostic gains across 11 datasets and three model backbones. On DeepSeek-R1-Distill-Llama-8B, its average score is **70.48**, compared with **65.55** for EvoPrompt and **61.25** for APE; on GPT-4o-mini it reaches **90.56** versus **85.72** and **85.17**; on Qwen2.5-7B-Instruct it averages **75.25** versus **72.06** and **64.38**. Its memory ablations on GPT-4o-mini show large drops when component memory or prompt memory is removed, especially on SAMSum, SQuAD, and Causal Judgement [2510.18257].

DSPy-based optimization likewise shows substantial task-dependent improvements. In the multi-use-case study, contradiction detection improves from **46.2%** to **64.0%** with zero-shot MIPROv2 and to **76.90%** with `CustomMIPROv2`; a router-agent prompt improves from **85.71%** to **90.47%**; and a weak hallucination-detection prompt on GPT-4o-mini rises from **37.3%** accuracy to **74.0%** after instruction optimization [2507.03620]. The original DSPy paper reports that compiled pipelines outperform standard few-shot prompting “generally by over 25% and 65%” for GPT-3.5 and llama2-13b-chat, respectively, and expert-created demonstrations “by up to 5-46% and 16-40%” [2310.03714].

Trajectory-based optimization shows similar benefits in interactive settings. In text-to-SQL, RPO with TD-style feedback and replay raises mean performance from **0.333** for the sharded baseline to **0.477**, outperforming APO (**0.368**), GPO (**0.376**), and MC-style feedback (**0.388**). In task-oriented dialogue, the same TD+replay variant improves success rate from **0.420** to **0.619** [2510.05921]. P3, which co-optimizes system prompts and user complements, achieves **84.8** on GSM8K and **57.1** on GPQA, exceeding zero-shot CoT, TextGrad, DSPy, and PAS in the reported reasoning benchmarks; on general instruction-following benchmarks, it also improves substantially over BPO and PAS, with particularly large gains for weaker base models [2507.15675].

Merit-guided local optimization addresses a different problem: compatibility across model scales. MePO reports higher average scores than raw prompting, self-optimization, BPO, and FIPO across Qwen2-7B, Tulu2-7B-DPO, LLaMA2-7B-chat-hf, and LLaMA3-8B-instruct, and its downward-compatibility experiments show that some prompts produced by stronger optimizers degrade 1B and 3B inference models, whereas merit-guided rewrites are more robust [2505.09930]. In a legal classification setting, MCTS with direct macro-F1 scoring reaches **0.74** macro F1, while the proxy-guided MLP evaluator reaches **0.73** with an estimated runtime reduction by a factor of **3** and a break-even point after **19** expansions [2510.08524].

Declarative representations also show empirical value even when the “optimization” is architectural rather than search-based. DPT improves GQA balanced test-std accuracy from **60.89** to **63.57**, a **+2.68%** absolute gain, and yields over **31%** gains in zero-shot/few-shot settings on GQA [2205.02456]. PSAO’s proof-of-concept experiments show raw prompts at **29.4%** accuracy, heuristic search over annotations between **44.2%** and **48.6%**, and brute-force annotation search at **58.8%** on a selected reasoning set [2605.14561]. HAPO reports improvements in **11 out of 12** model-benchmark combinations and an average **+13.28% over Zero-Shot CoT** [2601.02683]. PDL’s compliance-agent case study reports up to **4x** performance improvement over a canned agent and prompt pattern, with large reductions in “no tool called” failure modes for both GPT-4o and granite3.2-8b-instruct [2507.06396]. SPL, finally, emphasizes engineering-side results: **65%** average prompt-boilerplate reduction, an exposed **68x** cost spread across model tiers before execution, and identical `.spl` scripts that run at **\$0.002** on OpenRouter or at zero marginal cost on a local Ollama instance [2602.21257].

The domains covered by these studies are correspondingly broad: classical NLP classification, QA, NLG, VQA, text-to-SQL, task-oriented dialogue, medical QA, code generation, safety filtering, router agents, contradiction detection, legal unfairness detection, agentic compliance workflows, and context-budgeted RAG. The breadth of tasks is one reason the field resists reduction to a single algorithmic template.

## 6. Limitations, tensions, and open problems

The literature is also explicit about its unresolved tensions. The most persistent is definitional: many systems are **partially declarative but operationally procedural**. DelvePO is the clearest statement of this point: its representation is structured and close to a declarative specification, but the optimizer itself is an LLM-orchestrated evolutionary pipeline without a formal constraint language, typed schema semantics, or guarantees of semantic preservation [2510.18257]. MePO likewise externalizes prompt merits but does not expose them as explicit constraints or per-merit objective terms at inference time [2505.09930]. P3 separates global and local prompt components but optimizes them through heuristic search and judging rather than through a declarative solver [2507.15675].

A second tension separates **representation** from **optimization**. PDL is deliberately a declarative prompt programming language, but it does not itself provide an optimizer or compiler pass for automatic prompt improvement; it contributes the representation and execution model on which such techniques could be built [2410.19135]. NLD-P is even more architectural: it explicitly does not claim empirical superiority or quantitative drift resistance, but rather offers a governance abstraction for prompt lifecycle control under model drift [2602.22790]. Prompt Decorators make strong claims about modularity and deterministic top-to-bottom processing, yet also acknowledge that precedence rules and conflict handling are not fully specified and that current LLMs may interpret decorators via pattern recognition rather than symbolic parsing [2510.19850].

A third limitation concerns **evaluation dependence**. Many optimizers rely on LLM-as-judge scoring, proxy evaluators, or task-specific metrics defined over development sets. RPO’s RL analogy is deliberately informal: it provides neither a learned critic nor an explicit expected-return objective, and its effectiveness depends strongly on the quality of textual feedback [2510.05921]. The legal MCTS system depends on gold labels to train its correctness proxy and on the representativeness of prompt-clause pairs sampled during proxy construction [2510.08524]. MePO’s merit discovery and filtering depend heavily on DeepSeek-R1 judgments [2505.09930]. HAPO’s claims about reduced prompt drift are more clearly formalized than exhaustively benchmarked [2601.02683].

A fourth open problem is **search efficiency and controllability**. PSAO explicitly presents itself as a proof of concept and leaves efficient identification of optimal segmentations and annotations to future work [2605.14561]. DelvePO, RPO, and legal MCTS all require repeated model evaluations, prompting questions about token cost, optimization-time latency, and evaluator noise [2510.18257] [2510.05921] [2510.08524]. SPL addresses these concerns for context management through explicit budgets and `EXPLAIN`, but it optimizes resource allocation rather than prompt content, so the broader problem of unifying semantic and resource-level declarative optimization remains open [2602.21257].

A final tension concerns **generality versus domain specificity**. DPT’s declarative reformulation is powerful in VQA but depends on good declaration generation and task-specific answer insertion machinery [2205.02456]. PDL’s compliance-agent case study and SPL’s query-planning model show that declarative representations are highly effective when the workflow is explicit and structured, but they do not by themselves solve prompt content search [2507.06396] [2602.21257]. Declarative crowdsourcing, which predates many of these systems, already framed the larger challenge: users should specify high-level data-processing objectives and budgets, while the system should choose prompting strategies, aggregation mechanisms, and hybrid LLM/non-LLM workflows to satisfy those objectives [2308.03854].

The most plausible synthesis is therefore that declarative prompt optimisation is converging on a hybrid architecture. Representation layers—schemas, signatures, decorators, governance blocks, component slots, or query languages—make prompts inspectable and transformable. Optimization layers—compilers, search procedures, preference learners, or feedback-driven rewriters—operate over those representations using task metrics, proxy judges, memory traces, or budgets. The major open question is how far these layers can be integrated into systems that are simultaneously expressive, efficient, controllable, and empirically robust across model drift, task variation, and deployment constraints [2310.03714] [2410.19135] [2602.22790].

Source: https://www.emergentmind.com/topics/declarative-prompt-optimisation