PROMPT-MII: Meta-Learning for LLM Instructions

• PROMPT-MII is a reinforcement learning-based meta-learning framework that synthesizes compact, descriptive instructions to replace lengthy in-context examples.
• It employs a T5-based sequence-to-sequence architecture to transform sparse labeled examples into effective prompts, achieving 10–20% relative F1 score gains.
• Meta-trained on over 3,000 datasets, PROMPT-MII offers scalable, interpretable instruction induction that drastically reduces token usage while maintaining high performance.

PROMPT-MII: Meta-Learning Instruction Induction for LLMs

PROMPT-MII is a reinforcement learning (RL) based meta-learning framework designed to synthesize compact, descriptive prompts—specifically “instructions”—that enable LLMs to perform new classification tasks with high accuracy and substantially reduced inference cost when compared to standard in-context learning. By learning how to induce an instruction from a set of labeled exemplars, PROMPT-MII bridges the efficiency gap between token-heavy in-context learning (ICL) and manual prompt engineering, enabling automated, scalable instruction induction across many tasks using a shared policy.

1. Motivation and Problem Formulation

In-context learning, which prepends training examples to a natural language prompt at inference, allows LLMs to adapt flexibly to new tasks. However, ICL’s reliance on long contexts leads to prohibitively high inference costs—both in terms of memory (quadratic attention scaling) and real-world serving latencies—especially for classification tasks with extensive training sets or long input examples. Traditional instruction prompting, while token-efficient, lacks principled means for automated instruction synthesis and often requires extensive manual engineering.

PROMPT-MII addresses these issues by meta-learning an instruction induction model: Given a support set S_train^i of labeled task exemplars, it outputs an instruction I = πθ(S_train^i) that replaces the explicit enumeration of training examples at inference time. This compact instruction is then used as a prefix for a frozen base LLM, which is referred to as the “instruction follower,” to generate outputs for new inputs.

2. Reinforcement Learning-based Meta-Learning Methodology

PROMPT-MII frames instruction induction as a meta-learning task across a diverse collection of datasets:

• The instruction induction model πθ is a parameterized sequence-to-sequence model (typically a T5-based architecture) that ingests sampled support sets and produces a natural language instruction describing the task logic.
• The quality of an instruction is assessed by evaluating the downstream performance (macro-F1 score) of the instruction follower LLM (also a frozen T5 variant) when prompted with this instruction and applied to the (unseen) test set S_test^i.

The RL objective function for meta-training is:

$$J(\theta) = \mathbb{E}_s \left[ \frac{1}{n} \sum_k \min\left( r_k(\theta) A_k, \mathrm{clip}(r_k(\theta), 1-\rho_L, 1+\rho_H) A_k \right) \right]$$

with the importance ratio

$$r_k(\theta) = \frac{\pi_\theta(I_k\,|\,T(S_\text{train}^{(i)}))}{\pi_{\theta_\text{old}}(I_k\,|\,T(S_\text{train}^{(i)}))}$$

and the group-relative advantage

$$A_k = R(I_k, S_\text{test}^{(i)}) - \frac{1}{n}\sum_j R(I_j, S_\text{test}^{(i)})$$

where $R(I, S_\text{test}^{(i)})$ denotes the downstream macro-F1 for instruction I, and $T$ is a canonical meta-prompt template. Clipping (with $\rho_L$ and $\rho_H$) ensures stable policy updates and diverse exploration. This RL approach enables the induction model to generalize prompt-synthesis strategies across a broad range of tasks.

3. Training Corpus and Meta-Training Setup

PROMPT-MII is meta-trained on over 3,000 text classification datasets (e.g., from the HuggingFace hub) covering diverse domains, label spaces, and linguistic phenomena. For every meta-training episode, a random sample of k labeled examples is drawn as the support set, and a held-out query set is used to evaluate the downstream task metric. The instruction induction model is not exposed to ground-truth instructions; instead, it is rewarded purely on the macro-F1 achieved by the instruction follower when conditioned on its induced instruction.

At meta-test time, the model is evaluated on 90 wholly unseen datasets representing new classification tasks and domains, further demonstrating the generality and transferability of the meta-learned instruction policy.

4. Performance and Token Efficiency

PROMPT-MII achieves 4–9 absolute F1 point gains (10–20% relative improvement) on 90 held-out tasks compared to baselines such as naïve instruction prompting or an untrained instruction generator (“Prompt-MII-Zero”). When compared to ICL with the full set of training examples prepended, PROMPT-MII matches or approaches the downstream performance with a dramatic reduction (3–13×) in the number of tokens required at inference. This compressive effect is particularly pronounced as the number of support examples increases; ICL’s computational burden scales linearly (or worse, with long inputs), while PROMPT-MII’s cost is effectively fixed after a single instruction is induced.

The following table summarizes performance highlights:

Method	F1 Improvement over Baseline	Tokens Needed (relative)	Matches ICL Performance?
ICL (full context)	Baseline	1.0×	Yes
PROMPT-MII	+4–9 pts (10–20% rel)	0.08–0.33× (3–13× fewer)	Yes
Prompt-MII-Zero	Baseline	0.08–0.33×	No

5. Instruction Induction Policy and Task Generalization

The induced instructions are generated in natural language and typically summarize the class-label mapping in condensed form, occasionally including positive/negative criteria, required context, and unambiguous answer formats as learned over thousands of tasks. Instructions are interpretable and can often be inspected for alignment or errors, which is an advantage over opaque prompt embeddings. Because instructions are meta-learned over a wide diversity of tasks and label sets, PROMPT-MII robustly transfers to new, unseen domains without needing extensive manual prompt engineering. This supports plug-and-play adaptation: for a new task, a handful of labeled examples are sufficient to induce an effective instruction.

A plausible implication is that such meta-learned instruction induction could facilitate rapid prototyping and robust evaluation in low-shot task transfer, model self-adaptation, or in orchestrating chains of reasoning (e.g., through compositional or “pre-chain-of-thought” prompts).

6. Implications, Limitations, and Future Directions

The principal contribution of PROMPT-MII is the demonstration that LLMs can be efficiently and scalably adapted to novel tasks using meta-learned instruction induction without the inference penalties of ICL, and without requiring heavy manual prompt engineering. Inference efficiency gains are especially pronounced as context lengths or input sizes increase, making this method practical for real-world deployments where context budget is at a premium.

As PROMPT-MII is directly evaluated on classification tasks, an open avenue is the extension to generative and more complex tasks—such as multi-step reasoning, summarization, or program synthesis—potentially requiring refinements in meta-prompt templates or RL-based policy architectures. Additionally, future work may explore iterative refinement and distribution-aware synthesis (i.e., instruction induction policies that adaptively optimize instruction length or specificity based on dataset characteristics).

In summary, PROMPT-MII establishes a general, reinforcement learning-based meta-instruction induction paradigm that achieves ICL-level accuracy while drastically reducing inference token usage. It integrates scalable meta-learning, interpretability, and efficiency, representing a significant advance in the practical adaptation of LLMs to novel domains and tasks (Xiao et al., 19 Oct 2025).