---
title: 'PRESTO: Preimage-Informed Instruction Optimization'
url: https://www.emergentmind.com/topics/preimage-informed-instruction-optimization-presto
type: topic
---

# PRESTO: Preimage-Informed Instruction Optimization

PREimage-informed inSTruction Optimization (PRESTO) is a framework for efficient instruction optimization in prompting black-box large language models (LLMs). It leverages the many-to-one mapping from soft prompts to discrete instructions observed in white-box LLMs, introducing preimage-based strategies to substantially amplify data efficiency under a fixed query budget while optimizing task-specific metrics. PRESTO’s core innovation is to treat the preimage structure—sets of distinct soft prompts that decode to the same natural language instruction—not as a hindrance, but as a useful prior enabling score sharing, diversity-maximized initialization, and unsupervised score consistency regularization [2510.25808].

## 1. Formal Problem Definition and Preimage Structure

Let $f_b$ denote a deterministic black-box LLM (e.g., GPT-4), and $h(\cdot, \cdot)$ a task-specific metric such as accuracy or F₁. The goal is to find a textual instruction $v \in \Omega$ that maximizes expected score on a distribution of input-output pairs $(x, y) \sim D_{val}$:
$$
v^* = \arg\max_{v \in \Omega} \mathbb{E}_{(x, y) \sim D_{val}}[h(f_b(v, x), y)].
$$
Direct optimization over $\Omega$ is intractable due to its combinatorial size. Following recent practice, optimization proceeds over a continuous soft prompt $z \in \mathbb{R}^{N_z \times d}$, mapped to instructions by a white-box LLM $f_w$:
$$
z^* = \arg\max_{z \in \mathcal{Z}} \mathbb{E}_{(x, y) \sim D_{val}} [h(f_b(f_w(z), x), y)].
$$
A candidate pool $Z = \{z_j\}_{j=1}^N$ is constructed (via a scrambled Sobol sequence and random projection). Each $z_j$ is mapped through $f_w$ to a set of instructions $V = \{v_i = f_w(z_j) \mid j=1 \ldots N\}$. The mapping $f_w$ is many-to-one: multiple $z$ may produce the same $v$. The preimage of $v$ is defined as:
$$
\operatorname{Pre}(v) \equiv f_w^{-1}(v) = \{z \in Z \mid f_w(z) = v\}.
$$
This “preimage group” structure is central to PRESTO’s design.

## 2. Black-Box Instruction Optimization as Bandit Problem

Instruction optimization is formulated as a black-box bandit problem, where each $z \in Z$ is an arm with reward
$$
r(z) \triangleq \mathbb{E}_{(x,y)\sim D_{val}}[h(f_b(f_w(z),x), y)].
$$
Under a fixed budget $Q$, only $Q$ queries to $f_b$ are permitted. The objective is to pick $z_1,\ldots,z_Q \in Z$ to maximize the best-observed $r(z)$. Each $z$ is evaluated by querying $f_b$ on $v=f_w(z)$; due to preimage redundancy, multiple $z$'s may represent the same $v$, but only distinct instructions yield new supervision.

## 3. Core Components of PRESTO

### a. Score Sharing

Once $f_b(v_i)$ is evaluated for any $z \in \operatorname{Pre}(v_i)$, its scalar score $s_i$ is distributed to all $z' \in \operatorname{Pre}(v_i)$:
$$
\forall z' \in \operatorname{Pre}(v): \hat r(z') \leftarrow s.
$$
This expands the set of labeled pairs from $Q$ true queries to $\sum_{t=1}^Q |\operatorname{Pre}(v_t)|$ soft prompts.

### b. Preimage-Based Initialization

To initialize the model, $N_{init} \approx 40$ preimage groups are selected to cover the embedding space. For each instruction $v_i$, $G_i = \{g(z) \mid z \in \operatorname{Pre}(v_i)\}$ is constructed, where $g(z)$ is the final token embedding from $f_w$. Groups are greedily chosen using the coverage-score:
$$
S_{cov}(G_i | G_{init}, G_{total}) = S_{size}(G_i) + S_{rep}(G_i | G_{init}, G_{total}),
$$
where $S_{size}(G_i) = |G_i| / \max_j |G_j|$ promotes large preimages, and $S_{rep}$ leverages squared maximum-mean-discrepancy (MMD$^2$) for embedding diversity:
$$
\mathrm{MMD}^2(X, Y) = \mathbb{E}_{x, x' \sim X}[k(x, x')] + \mathbb{E}_{y, y' \sim Y}[k(y, y')] - 2 \mathbb{E}_{x \sim X, y \sim Y}[k(x, y)].
$$

### c. Score Consistency Regularization

For unseen preimages $V_{unseen} = V \setminus \{v_t : t \leq t'\}$, the regression model $m(g(z); \theta)$ is regularized to output identical predictions for all $z, z' \in \operatorname{Pre}(v)$:
$$
\mathcal{L}_{cons}(\theta) = \mathbb{E}_{v \in V_{unseen}} \mathbb{E}_{z, z' \in \operatorname{Pre}(v)} \left( m(g(z); \theta) - m(g(z'); \theta) \right)^2.
$$
The full predictor loss is:
$$
\mathcal{L}(\theta) = \mathcal{L}_{MSE}(\theta) + \gamma(t)\mathcal{L}_{cons}(\theta)
$$
with $\gamma(t)$ linearly annealed from 0 to $\gamma_{max}$ over a warm-up period.

## 4. PRESTO Optimization Algorithm

PRESTO operates in three main stages:

- **Preprocessing:** For each candidate $z \in Z$, compute its embedding $g(z)$ and instruction $f_w(z)$, forming all preimage groups.
- **Initialization:** Greedily select $N_{init}$ preimage groups maximizing $S_{cov}$. For each, query $f_b$ on a representative $z$, then share the obtained score across the group, forming labeled set $D$.
- **NeuralUCB Optimization:** For $t=N_{init}+1$ to $Q$:
    1. Train score predictor $m(\cdot; \theta)$ on $D$ with joint loss.
    2. For each unlabeled $z$, estimate $\mu(z) = m(g(z); \theta)$ and predictive uncertainty $\sigma(z)$.
    3. Select $z_{next} = \arg\max_z [\mu(z) + \sqrt{\beta} \sigma(z)]$.
    4. Query $f_b$ on $z_{next}$, share the score with its preimage, and update $D$.
- **Output:** Return the best observed instruction $v$.

This design allows PRESTO to aggregate supervision over preimages, maximizing the utility of each black-box query.

## 5. Analysis of Query Efficiency and Data Amplification

With query budget $Q=165$, conventional methods yield $Q$ labeled (soft prompt, score) pairs. PRESTO’s score sharing multiplies the number of labeled prompts by the average preimage size. Empirically, across 30 tasks, PRESTO inflates the labeled set to $\sim$2,300, a $\sim$14× increase over the budget:
$$
\#\text{labeled} \approx \sum_{t=1}^Q |\operatorname{Pre}(v_t)| \approx Q \cdot \mathbb{E}[|\operatorname{Pre}(v)|] \approx 165 \cdot 14 = 2,310.
$$
*This suggests that preimage-based score sharing is critical for unlocking efficient regression and improved uncertainty quantification in UCB-based exploration, without additional $f_b$ queries*.

## 6. Experimental Setup and Baselines

Experiments span 30 instruction-induction tasks (Honovich et al., ACL ’23) and 3 math reasoning benchmarks (GSM8K, AQUA-RAT, SVAMP). Metrics include exact match, F₁, and chain-of-thought (CoT) accuracy. The black-box LLM is GPT-4.1 (37B); the white-box LLM is LLaMA 3.1-8B-Instruct. Comparison baselines are APE, InstructZero, INSTINCT, EvoPrompt, ZOPO, and OPRO. Candidate pool size is $N=10,000$ soft prompts, constructed by Sobol sampling and random projections, with grid search over intrinsic dimensions and soft-token counts. The query budget is $Q=165$ and $N_{init}=40$; all experiments use a single NVIDIA A6000 GPU.

## 7. Empirical Results and Ablation Analysis

PRESTO achieves the following empirical performance:

- Wins 18/30 tasks (compared to ZOPO’s 8, next best) with mean rank $\approx$1.97.
- On a 20-task subset, wins 12 tasks (ZOPO: 4), with mean rank 1.90 (vs. 3.05 for ZOPO).
- On CoT benchmarks, PRESTO meets or surpasses hand-crafted and prior soft-prompting approaches on GSM8K, AQUA-RAT, and SVAMP.
- In ablations (20 tasks):

| Configuration                      | Avg. Accuracy | Avg. Rank |
|-------------------------------------|---------------|-----------|
| Vanilla (no preimage modules)       | 51.91         | 4.55      |
| +Score Sharing Only                 | 59.57         | 3.10      |
| +Score Sharing + Score Consistency  | 61.77         | 2.65      |
| +Score Sharing + Preimage Init      | 61.82         | 2.30      |
| All PRESTO components               | 62.91         | 2.20      |

- Score predictor RMSE on a toy task decreases from $\sim$0.27 (vanilla) to $\sim$0.15 (PRESTO).
- t-SNE visualization shows preimage-based initialization achieves denser and more uniform coverage of the embedding space compared to random or sharing-only initialization.

These results substantiate the complementary gains of score sharing, diversity-maximized preimage initialization, and consistency regularization.

## 8. Significance and Broader Implications

PRESTO demonstrates that the many-to-one mapping from soft prompts to instructions—previously viewed as a source of inefficiency—can be systematically exploited to amplify effective data, increase sample efficiency, and improve generalization in instruction optimization for black-box LLMs. The preimage structure enables up to 14× effective labeling, state-of-the-art task performance, and scalable optimization via score sharing, informed candidate selection, and unsupervised regularization. These findings provide a foundation for further advances in black-box LLM instruction engineering and suggest that similar preimage-informed priors may benefit other combinatorial black-box optimization domains [2510.25808].

Source: https://www.emergentmind.com/topics/preimage-informed-instruction-optimization-presto