--- title: Supervisory Prompt Training (SPT) url: https://www.emergentmind.com/topics/supervisory-prompt-training-spt type: topic --- # Supervisory Prompt Training (SPT) Supervisory Prompt Training (SPT) constitutes a family of automated strategies for prompt optimization in large neural language and speech models. Originating in response to the limitations of manual prompt engineering and parameter-intensive fine-tuning, SPT leverages both discrete (textual) and soft (embedding-based) prompt formulations to enhance model generalization, reduce hallucination, and address low-resource or code-switching scenarios with parameter efficiency. Current instantiations of SPT operate in both black-box settings—utilizing dual LLM agents for text prompt evolution—and white-box model access regimes, employing trainable soft prompts integrated with Transformer architectures for multilingual speech recognition. ## 1. Dual-Agent Supervisory Prompt Training for LLMs SPT as proposed in "Supervisory Prompt Training" introduces a collaborative system comprising a generator LLM ($G$) and a corrector LLM ($C$). At each iteration, $G$ generates task responses using its current prompt $p_i$, identifying a set of errors $m_i$. $C$, initialized with its own meta-prompt $c_i$, observes $p_i$ and $m_i$ and synthesizes new prompt candidates $\{p_i^1, \dotsc, p_i^n\}$. These candidates are scored by re-applying $G$ on $m_i$, and the best candidate (maximizing accuracy on $m_i$) is selected as $p_{i+1}$. Optionally, $c_{i+1}$ is refined using feedback on differences between $p_i$, $p_{i+1}$, and the associated errors. This cycle continues until a stopping criterion (e.g., no further accuracy improvement, prompt convergence) is met. SPT operates over explicit, fully-interpretable prompts for LLMs and does not require gradient access, enabling compatibility with proprietary black-box APIs [2403.18051]. ### Algorithmic Variants Two main algorithmic regimes are formalized: - **SPT-p:** Only the generator's prompt $p$ is updated. - **SPT-pc:** Both generator $p$ and corrector meta-prompt $c$ are jointly refined. Extended variants include SPT-cot (where $C$ outputs chain-of-thought rationales) and SPT-imp (impact-score-guided editing). ## 2. Impact Scores and Sentence-Level Prompt Attribution A central feature of SPT is the formal impact score $\Delta(s)$ for each sentence $s$ in a candidate prompt, defined as: $$ \Delta(s) = \mathrm{Acc}\bigl(D_\text{train}\,;\,p_k \oplus \{s\}\bigr) - \mathrm{Acc}(D_\text{train}\,;p_k) $$ where $p_k \oplus \{s\}$ denotes prompt $p_k$ with sentence $s$ appended. These scores quantify incremental training accuracy gain attributable to individual prompt components. In SPT-imp, $C$ incorporates $\{(s, \Delta(s))\}$, biasing the prompt generation process toward high-leverage instructions and demoting low- or negative-impact content [2403.18051]. ## 3. Methodological Comparison to Prior Prompt and Fine-Tuning Approaches SPT distinguishes itself from gradient-based soft prompt methods (which require internal model access and produce non-interpretable embeddings) and prior LLM-based prompt engineering techniques (which lack dual-agent feedback and meta-prompt evolution). Key differentiators: - **Interpretability:** SPT develops textual prompts amenable to direct human analysis. - **Mutual Improvement:** Both generator and corrector can improve, fostering stronger prompt search. - **No Gradient Requirement:** SPT functions in API-only settings. - **Sentence-Level Attribution:** Impact scores enable fine-grained, quantitative prompt assessment. Tradeoffs include higher computational cost (due to repeated LLM calls) and risks of prompt overfitting with long or highly specific instructions [2403.18051]. ## 4. Quantitative Evaluation and Experimental Findings SPT was evaluated on multiple LLMs (GPT-3.5, GPT-4, Llama2-70b) and datasets (TruthfulQA, GSM8K, MMLU, MedQA-US). Notable results include a 28.3% absolute accuracy increase on GSM8K for GPT-4 (65.8% baseline to 94.1% for SPT-pc), and consistent outperformance over Automatic Prompt Optimization (APO), which achieved 68.8% on the same benchmark. On TruthfulQA, SPT-p yielded 89.6% against APO's 87.1%. Similar relative trends were observed across other benchmarks. | Model (G) | Dataset | Baseline | APO | SPT-p | SPT-pc | |-------------|-------------|----------|------|-------|---------| | GPT-4 | GSM8K | 65.8% | 68.8%| 89.6% | 94.1% | | GPT-4 | TruthfulQA | 81.7% | 87.1%| 89.6% | 87.1% | | GPT-4 | MMLU | 79.7% | 79.6%| — | — | | GPT-4 | MedQA | 78.4% | 79.3%| 78.7% | 77.3% | The architecture enables substantial reduction in LLM hallucination and elevated generalization across tasks without traditional model fine-tuning [2403.18051]. ## 5. Soft Prompt Tuning SPT in Multilingual Speech Recognition In whisper-based multilingual ASR, SPT refers to a parameter-efficient class of methods wherein small blocks of learnable prompt embeddings are prepended to input representations of frozen or partially-tuned models [2506.21576][2506.21577]. ### Variants - **Vanilla SPT:** Single prompt matrix prepended to encoder and/or decoder inputs. - **Deep Prompt Tuning (DPT):** Prompts inserted at each transformer block throughout both encoder and decoder (recommended for maximal gains). - **Residual Prompt Tuning (ResPT):** Prompt banks parameterized via shared MLPs, improving convergence and parameter sharing. - **Language Prompt Tuning (LPT):** Employs pre-trained language embeddings as auxiliary prompts to disambiguate code-switching. - **SPT4ASR (Hybrid):** Concatenates DPT, ResPT, and LPT for further error reduction. - **Entire SPT:** Simultaneous prompting of both encoder and decoder, shown to outperform decoder-only SPT in language expansion tasks [2506.21577]. ### Formulation If $X \in \mathbb{R}^{l \times e}$ is the encoder input, SPT prepends a prompt $P \in \mathbb{R}^{n \times e}$: $X' = [P; X]$. In the decoder, prompts $P'$ are prepended analogously. The training objective (autoregressive) is: $$ L(P) = - \sum_{(X,y) \in D} \sum_{t=1}^{T} \log p(y_t \mid y_{