PromptAL: Dynamic Soft Prompts for Active Learning

• The paper introduces PromptAL, a framework that uses sample-aware dynamic soft prompts to adjust the predictive distribution in few-shot text classification.
• It employs a hybrid query strategy that combines calibrated uncertainty with global and local diversity to select the most informative samples.
• Experimental evaluations across multiple datasets reveal significant accuracy improvements over baselines, especially in challenging multi-class scenarios.

PromptAL is a few-shot active learning framework for text classification that treats prompt learning as part of the acquisition mechanism rather than merely as a downstream model adaptation technique. Its central claim is that, when labeled data are extremely scarce, the empirical distribution induced by the labeled set can diverge substantially from the target distribution, shifting the decision boundary away from its optimal position and making standard uncertainty-based querying unreliable. PromptAL addresses this by using unlabeled samples to construct sample-aware dynamic soft prompts that adjust the model’s predictive distribution and decision boundary before uncertainty and diversity are computed, and then selects query batches using a hybrid criterion that combines uncertainty with global and local diversity (Xiang et al., 22 Jul 2025).

1. Problem setting and motivation

PromptAL is formulated for the standard pool-based active learning setting with a small labeled training set $D_{\text{train}}$, a large unlabeled pool $D_{\text{pool}}=\{x_i\}_{i=1}^N$, a model $\mathcal{M}$, and a budget $B$ governing how many labels may be acquired (Xiang et al., 22 Jul 2025). The objective is to choose a subset $Q \subset D_{\text{pool}}$ for annotation, add it to $D_{\text{train}}$, retrain, and repeat for $t=|B|/|Q|$ rounds.

The paper emphasizes that few-shot active learning differs materially from conventional active learning because the labeled set is too small to be treated as a reliable approximation of the target distribution. With only a handful of labeled instances, some classes or modes may be underrepresented or absent, so the model trained on $D_{\text{train}}$ learns a biased approximation of the task distribution. Under that condition, uncertainty estimates are computed relative to a distorted decision boundary, and the most uncertain samples may not be the most informative or representative ones (Xiang et al., 22 Jul 2025).

PromptAL is therefore motivated by a distribution-alignment argument. Existing methods are described as overlooking the role of unlabeled samples in enhancing the empirical distribution so that it better aligns with the target distribution. The result is a suboptimal decision boundary and selection of samples that inadequately represent the target distribution. PromptAL reframes the unlabeled pool not merely as a set to be scored, but as a source of information for correcting the model’s current view of the task before query selection begins (Xiang et al., 22 Jul 2025).

The paper supports this motivation with a Jensen–Shannon divergence analysis. A model trained only on 32 labeled instances is reported to be farther from the fully trained target distribution than a model whose predictive behavior is augmented with sample-aware unlabeled information. This suggests that unlabeled-aware boundary adjustment can move the current model distribution closer to the target one, making later query decisions more meaningful (Xiang et al., 22 Jul 2025).

2. Sample-aware dynamic soft prompts

The defining mechanism of PromptAL is the use of dynamic soft prompts that are generated separately for each unlabeled sample. Rather than using a fixed prompt for all inputs, PromptAL constructs a prompt $\mathbf{P}(x)$ that depends on the sample $x$ itself. This prompt combines a shared task-specific component and a sample-specific component, then fuses them by multi-head self-attention (Xiang et al., 22 Jul 2025).

The task-specific prompt is denoted $D_{\text{pool}}=\{x_i\}_{i=1}^N$0, while the sample-specific prompt is $D_{\text{pool}}=\{x_i\}_{i=1}^N$1. The final prompt is defined as

$D_{\text{pool}}=\{x_i\}_{i=1}^N$2

This construction is explicitly described as both dynamic and sample-aware: dynamic because it is not fixed after training, and sample-aware because it is generated for each unlabeled instance (Xiang et al., 22 Jul 2025).

The sample-specific prompt $D_{\text{pool}}=\{x_i\}_{i=1}^N$3 is produced by a prompt generator $D_{\text{pool}}=\{x_i\}_{i=1}^N$4 consisting of an encoder $D_{\text{pool}}=\{x_i\}_{i=1}^N$5 followed by a two-layer MLP. The encoder maps the input to $D_{\text{pool}}=\{x_i\}_{i=1}^N$6, and the MLP transforms that representation into $D_{\text{pool}}=\{x_i\}_{i=1}^N$7. Prompt fusion is then carried out by multi-head self-attention over the concatenated task and sample prompts. For the $D_{\text{pool}}=\{x_i\}_{i=1}^N$8-th attention head, the paper gives

$D_{\text{pool}}=\{x_i\}_{i=1}^N$9

where

$\mathcal{M}$0

All heads are concatenated and linearly transformed to obtain $\mathcal{M}$1 (Xiang et al., 22 Jul 2025).

The prompt-augmented input is constructed as

$\mathcal{M}$2

where $\mathcal{M}$3 is the token embedding function and $\mathcal{M}$4 is the masked token used by the underlying masked LLM. The prediction for label $\mathcal{M}$5 is then

$\mathcal{M}$6

with $\mathcal{M}$7 denoting the verbalizer label word for class $\mathcal{M}$8 (Xiang et al., 22 Jul 2025).

The stated purpose of $\mathcal{M}$9 is to reshape the model’s predictive distribution on $B$0, thereby altering where the decision boundary lies relative to the input space. Because every unlabeled point induces its own prompt, every unlabeled point contributes to the model’s current approximation of the target distribution. The paper does not introduce a direct distribution-matching loss for this step, but interprets the prompt mechanism as refining the predictive distribution and shifting the decision boundary toward a more optimal placement (Xiang et al., 22 Jul 2025).

3. Calibrated uncertainty and diversity-aware querying

PromptAL does not stop at prompt-conditioned prediction. Its acquisition rule is a hybrid combination of uncertainty estimation, global diversity, and local diversity, computed after the prompt-based adjustment of the predictive distribution (Xiang et al., 22 Jul 2025).

For uncertainty, the method adopts contextual calibration from KPT because pretrained LLMs can be biased toward particular verbalizer label words. A support set $B$1 is constructed by selecting the top $B$2 samples with the highest $B$3 for each class $B$4. The contextualized prior is then estimated as

$B$5

where $B$6 is a label word in the verbalizer (Xiang et al., 22 Jul 2025).

The calibrated class distribution is defined by

$B$7

Uncertainty is then measured by entropy: $B$8 This sequencing is important: uncertainty is not computed from the raw model, but from the prompt-adjusted and calibrated distribution. The paper presents this as one reason the framework is better suited to few-shot settings, where raw uncertainty would otherwise inherit the distortions caused by a poorly placed boundary (Xiang et al., 22 Jul 2025).

Global diversity is enforced by clustering the unlabeled pool in what the paper calls a knowledge feature space. Rather than clustering on ordinary semantic embeddings, PromptAL uses the hidden-state vector at the $B$9 position, denoted $Q \subset D_{\text{pool}}$0, as the feature representation. The paper argues that this representation captures task-related knowledge induced by the dynamic soft prompts. K-means++ is then applied to $Q \subset D_{\text{pool}}$1, producing $Q \subset D_{\text{pool}}$2 clusters so that the eventual query batch is broadly distributed across the pool (Xiang et al., 22 Jul 2025).

Local diversity is defined relative to the labeled set. For a candidate $Q \subset D_{\text{pool}}$3, let $Q \subset D_{\text{pool}}$4 be its $Q \subset D_{\text{pool}}$5 nearest labeled neighbors in knowledge-feature space. The local diversity score is

$Q \subset D_{\text{pool}}$6

A larger value indicates that the candidate is less redundant with respect to already labeled samples (Xiang et al., 22 Jul 2025).

The final acquisition score combines uncertainty and local diversity: $Q \subset D_{\text{pool}}$7 with $Q \subset D_{\text{pool}}$8. The experiments use $Q \subset D_{\text{pool}}$9, which heavily weights uncertainty while preserving a nontrivial contribution from local diversity. Query selection is then performed cluster by cluster: $D_{\text{train}}$0 where $D_{\text{train}}$1 denotes the set of global clusters (Xiang et al., 22 Jul 2025).

This layered acquisition rule makes PromptAL a genuinely hybrid method. It is not purely uncertainty sampling, purely diversity sampling, or purely prompt learning. Instead, it first adjusts the decision boundary with sample-aware prompts, then estimates uncertainty under that adjusted boundary, and finally constrains batch construction with both global coverage and local nonredundancy (Xiang et al., 22 Jul 2025).

4. Active learning workflow

The full PromptAL loop starts from a tiny labeled set of 32 examples, an unlabeled pool, and a pretrained RoBERTa-base masked LLM $D_{\text{train}}$2 (Xiang et al., 22 Jul 2025). At each round, the model constructs sample-aware dynamic soft prompts for the unlabeled pool, obtains prompt-conditioned predictions, calibrates label probabilities, computes entropy, clusters the pool in knowledge-feature space, measures local diversity relative to labeled instances, scores each sample, selects a query set, obtains annotations, and retrains. The process is repeated for 10 active learning rounds (Xiang et al., 22 Jul 2025).

The paper’s description of the workflow can be summarized as follows:

Stage	Operation	Role in selection
Prompt construction	Build $D_{\text{train}}$3 from $D_{\text{train}}$4 and $D_{\text{train}}$5	Adjust predictive distribution
Uncertainty estimation	Calibrate probabilities and compute $D_{\text{train}}$6	Measure informativeness
Diversity modeling	Cluster globally and compute $D_{\text{train}}$7 locally	Ensure coverage and reduce redundancy

The framework is described as hybrid because the final query decision is based on multiple criteria after prompt-based correction of the model state. This differs from methods that score points directly from a model trained on the scarce labeled set alone. A plausible implication is that PromptAL moves the intervention point earlier in the active learning pipeline: instead of improving only the ranking of candidates, it first improves the representation under which candidates are ranked (Xiang et al., 22 Jul 2025).

The paper also frames the method as a unification of prompt learning and active learning. Prompt learning is not used merely to improve final classification accuracy after labels are acquired; it is part of the acquisition procedure itself, since the prompts are constructed over unlabeled samples and directly influence which samples are selected for annotation (Xiang et al., 22 Jul 2025).

5. Experimental evaluation

PromptAL is evaluated on six in-domain text classification datasets—IMDB, AGNews, DBpedia, Yelp-full, TREC, and Yahoo! Answers—and three out-of-domain datasets for OOD evaluation on IMDB—SST-2, IMDB-Contrast, and IMDB-Counter (Xiang et al., 22 Jul 2025). The baseline set comprises nine methods: uncertainty-based methods Entropy, LC, and BALD; diversity-based methods BERT-KM and Core-set; hybrid or prompt-based methods Patron, CAL, and BADGE; and Random sampling (Xiang et al., 22 Jul 2025).

On the six in-domain datasets, PromptAL is reported to achieve the best average accuracy on all but one dataset, AGNews, where it ranks second. The paper gives the following representative averages over 10 active learning rounds: DBpedia $D_{\text{train}}$8, TREC $D_{\text{train}}$9, Yahoo $t=|B|/|Q|$0, Yelp $t=|B|/|Q|$1, and IMDB $t=|B|/|Q|$2 (Xiang et al., 22 Jul 2025). It further highlights an average improvement of 3.56% on DBpedia over competitors, and about 2.01% over Patron on DBpedia, which the authors use to argue for the value of dynamic soft prompts over hard prompts (Xiang et al., 22 Jul 2025).

For out-of-domain evaluation, PromptAL outperforms Patron by 3.16% on average across SST-2, IMDB-Contrast, and IMDB-Counter, with the best results on SST-2 and IMDB-Contrast and the second-best result on IMDB-Counter (Xiang et al., 22 Jul 2025). This is presented as evidence that the sample-aware prompting strategy improves not only in-domain selection quality but also generalization.

The paper also analyzes selected batches using IMB, LDD, Rep, Div, and Unc. PromptAL is reported to achieve the best overall balance, with especially strong gains in uncertainty and diversity while also maintaining better representativeness and label balance. A t-SNE case study shows that PromptAL selects more spread-out, less redundant samples than Entropy and that its selected points cover categories more evenly (Xiang et al., 22 Jul 2025).

These results position PromptAL as a method whose contribution is not reducible to improved classifier adaptation alone. The empirical claim is that boundary adjustment via sample-aware dynamic prompts leads to better downstream acquisition decisions. This suggests that, at least in the low-label regime studied here, improving the reliability of uncertainty is as important as designing the uncertainty metric itself (Xiang et al., 22 Jul 2025).

6. Ablations, interpretation, and limitations

The ablation studies are central to the paper’s argument because each component of PromptAL corresponds to a distinct hypothesis about few-shot active learning. Removing sample-specific prompts causes substantial performance loss, indicating that instance-dependent prompting is crucial. Removing task-specific prompts also degrades performance, showing that shared task knowledge remains necessary alongside sample-aware adaptation (Xiang et al., 22 Jul 2025).

The paper likewise reports that removing global diversity or local diversity leads to significant drops and instability, and removing uncertainty also lowers performance. This supports the view that the method’s performance depends on the combination rather than on any single acquisition heuristic. Removing calibration weakens uncertainty estimation and therefore active learning performance, reinforcing the importance of correcting label-word bias in prompt-based masked-language-model prediction (Xiang et al., 22 Jul 2025).

A fusion ablation compares self-attention against simpler alternatives such as Hadamard product and elementwise addition. The paper reports that self-attention performs better, and interprets this as evidence that attention can better filter useful sample-aware information and suppress noise. For global diversity, the knowledge feature at the $t=|B|/|Q|$3 position outperforms both plain $t=|B|/|Q|$4 embeddings and SimCSE features, which the authors take as support for clustering in a task-informed prompt-conditioned feature space rather than a generic semantic space (Xiang et al., 22 Jul 2025).

The paper’s overarching interpretation is that, in few-shot active learning, the main problem is not only to identify uncertain samples but to identify uncertain samples under a model that better approximates the target distribution. PromptAL’s sample-aware dynamic soft prompts serve precisely this purpose. Once the boundary is corrected, entropy becomes a more meaningful informativeness signal, and diversity constraints can then be used to turn that signal into a representative query batch (Xiang et al., 22 Jul 2025).

Although the paper does not present an extended limitations section, several tradeoffs are explicitly or implicitly acknowledged. The method introduces extra computation for prompt generation, clustering, and KNN-based local diversity. Its performance depends on hyperparameters such as $t=|B|/|Q|$5, prompt sizes $t=|B|/|Q|$6, and the number of attention heads. It is implemented around a RoBERTa-base masked-language-model backbone, so transferring the design to other architectures would require additional engineering. The paper also notes that gains are strongest in harder multi-class few-shot settings; on simpler datasets such as AGNews and IMDB, the advantage is smaller and not always statistically significant (Xiang et al., 22 Jul 2025).

In that sense, PromptAL contributes a specific thesis to active learning: unlabeled data should influence not only which points are queried, but also how the current model constructs the geometry under which querying is performed. By using unlabeled samples to generate sample-aware dynamic soft prompts before uncertainty and diversity are computed, the framework turns prompt learning into a mechanism for boundary correction and query selection under distribution mismatch (Xiang et al., 22 Jul 2025).