Self-Supervised Prompt Optimization (2502.06855v2)

Abstract: Well-designed prompts are crucial for enhancing LLMs' (LLMs) reasoning capabilities while aligning their outputs with task requirements across diverse domains. However, manually designed prompts require expertise and iterative experimentation. While existing prompt optimization methods aim to automate this process, they rely heavily on external references such as ground truth or by humans, limiting their applicability in real-world scenarios where such data is unavailable or costly to obtain. To address this, we propose Self-Supervised Prompt Optimization (SPO), a cost-efficient framework that discovers effective prompts for both closed and open-ended tasks without requiring external reference. Motivated by the observations that prompt quality manifests directly in LLM outputs and LLMs can effectively assess adherence to task requirements, we derive evaluation and optimization signals purely from output comparisons. Specifically, SPO selects superior prompts through pairwise output comparisons evaluated by an LLM evaluator, followed by an LLM optimizer that aligns outputs with task requirements. Extensive experiments demonstrate that SPO outperforms state-of-the-art prompt optimization methods, achieving comparable or superior results with significantly lower costs (e.g., 1.1% to 5.6% of existing methods) and fewer samples (e.g., three samples). The code is available at https://github.com/geekan/MetaGPT/blob/main/examples/spo

• The paper introduces a self-supervised framework that optimizes prompts through iterative pairwise output comparisons, eliminating the need for external labels.
• It employs an Optimize-Execute-Evaluate cycle to refine prompts with minimal evaluation samples, significantly cutting computational costs.
• Experimental results demonstrate competitive performance across diverse tasks using only three sample comparisons per iteration.

Introduction and Motivation

Self-Supervised Prompt Optimization (SPO) is a framework designed to automate the discovery of effective prompts for LLMs without reliance on external reference signals such as ground truth labels or human annotations (2502.06855). The primary motivation arises from the limitations inherent in manual prompt engineering, which is labor-intensive and requires domain expertise, and existing automated prompt optimization (PO) techniques. Many contemporary PO methods necessitate reference data for evaluation, posing significant challenges for open-ended tasks or scenarios where labeled data is scarce or expensive to acquire. Furthermore, methods relying on extensive sampling for evaluation incur substantial computational costs, particularly with large-scale LLMs. SPO addresses these constraints by proposing a self-supervised approach that derives optimization signals entirely from the LLM's outputs, leveraging the model's intrinsic capabilities to assess output quality relative to task requirements. The fundamental premise is that prompt efficacy is directly reflected in the generated outputs, and LLMs possess sufficient evaluative capacity to discern superior outputs based on pairwise comparisons.

Methodology: The SPO Framework

SPO operates through an iterative Optimize-Execute-Evaluate cycle, utilizing LLM outputs as the primary information source for both evaluation and refinement signals. The framework integrates three core functional components implemented using LLMs:

1. Optimization Function ($\phi_{opt}$): This function employs an LLM (the "optimizer") to generate improved prompt candidates. It takes the current best-performing prompt ($P^*$) and its corresponding outputs ($A^*$) on a small set of sample questions ($Q$) as input. The optimizer analyzes $A^*$ to identify weaknesses or deviations from the task objectives and proposes modifications to $P^*$, resulting in a new candidate prompt ($P'$). This process relies on the optimizer LLM's understanding of the task and its ability to infer prompt improvements from output analysis, rather than explicit error signals derived from ground truth.
2. Execution Function ($\phi_{exe}$): This function utilizes the target LLM (the "executor") to generate outputs using a given prompt. It takes the candidate prompt ($P'$) and the fixed set of input questions ($Q$) to produce a new set of outputs ($A'$).
3. Evaluation Function ($\phi_{eval}$): This function employs an LLM (the "evaluator") to compare the quality of outputs generated by different prompts. Crucially, SPO utilizes a pairwise comparison strategy termed Output-vs-Output (OvO). Instead of assigning absolute scores, $\phi_{eval}$ directly compares the output set $A'$ (from $P'$) against the output set $A^*$ (from $P^*$) for the same input questions $Q$. The evaluator LLM determines which output set better fulfills the task requirements. This relative judgment avoids the need for large sample sizes often required for stable absolute scoring and is less sensitive to variations in scoring scales.

The SPO process is initialized with a basic prompt template (e.g., a standard Chain-of-Thought prompt) and a small, fixed set of task-specific questions $Q$ (typically 3, drawn from the target dataset without labels). The framework then iterates for a predetermined number of steps (e.g., 10). In each iteration $t$:

1. $\phi_{opt}(P^*_{t-1}, A^*_{t-1}) \rightarrow P'_t$
2. $\phi_{exe}(P'_t, Q) \rightarrow A'_t$
3. $$\phi_{eval}(A'_t, A^*_{t-1}, Q) \rightarrow \text{optimizationSuccess}_t$$ If optimizationSuccess is true, indicating $A'_t$ is superior to $A^*_{t-1}$, the framework updates the best prompt and outputs: $P^*_t \leftarrow P'_t$ and $A^*_t \leftarrow A'_t$. Otherwise, $P^*_t \leftarrow P^*_{t-1}$ and $A^*_t \leftarrow A^*_{t-1}$. The final optimized prompt $P^*$ is returned after the completion of all iterations.

Implementation Details

The practical implementation of SPO involves selecting appropriate LLMs for the optimizer, executor, and evaluator roles. While the executor is typically the target LLM for which the prompt is being optimized, powerful models like GPT-4 are often employed for the optimizer and evaluator functions due to their strong reasoning and instruction-following capabilities. The framework's reliance on a small, fixed set of questions ($Q$, e.g., $N=3$) is a key aspect contributing to its efficiency. These questions are sampled once at the beginning and remain constant throughout the optimization process, ensuring consistent comparison points. The pairwise OvO evaluation prompt typically instructs the evaluator LLM to compare two sets of answers for the same questions based on specified task criteria (e.g., correctness, coherence, adherence to constraints) and determine which set is preferable. The optimization prompt guides the optimizer LLM to analyze the current best prompt and its outputs, identify shortcomings, and suggest concrete revisions to the prompt structure or content to improve performance on the task. The number of optimization iterations serves as a hyperparameter, balancing optimization quality and computational cost.

Performance and Efficiency

Experimental results reported in the paper demonstrate SPO's effectiveness and efficiency across various benchmarks, including both closed-ended (e.g., mathematical reasoning, QA) and open-ended (e.g., creative writing, code generation) tasks (2502.06855). Compared to state-of-the-art PO methods such as OPRO, PromptAgent, PromptBreeder, and TextGrad, SPO achieves comparable or superior task performance. Notably, this performance is achieved with significantly reduced computational overhead. The average optimization cost reported for SPO is approximately $0.15 per dataset, representing merely 1.1% to 5.6% of the costs associated with the baseline methods. This cost efficiency is primarily attributed to the minimal sample requirement (only three samples per iteration for evaluation) and the elimination of reliance on expensive ground truth annotations or human feedback loops. Ablation studies confirmed that using three samples strikes an effective balance, as performance gains plateau or even slightly decrease with more samples, while costs increase linearly. The pairwise OvO evaluation mechanism is identified as crucial for achieving stable and effective optimization with such a small sample size.

Contributions and Implications

The main contributions of the SPO paper include the proposal of a novel self-supervised, reference-free prompt optimization framework, the demonstration of its significant cost-effectiveness and sample efficiency compared to existing methods, and extensive empirical validation showing competitive performance on diverse tasks. SPO's primary implication lies in its potential to make automated prompt optimization significantly more accessible and practical for real-world applications. By circumventing the need for labeled data or human intervention, it becomes particularly valuable for optimizing prompts in domains where such resources are unavailable, such as open-ended generation, specialized low-resource tasks, or scenarios involving proprietary data. The drastic reduction in computational cost and sample complexity lowers the barrier for deploying sophisticated prompt optimization strategies, potentially accelerating the adoption and enhancing the performance consistency of LLMs across a wider range of applications and user groups.

Conclusion

Self-Supervised Prompt Optimization (SPO) presents a cost-efficient and reference-free approach to automated prompt engineering (2502.06855). By leveraging pairwise comparisons of LLM-generated outputs evaluated by an LLM, SPO iteratively refines prompts using minimal samples and computational resources. Its demonstrated performance and efficiency suggest a practical path forward for optimizing LLM behavior in diverse settings, particularly where traditional supervised or human-in-the-loop methods are infeasible.