Concept Prompt Tuning (CPT)

• Concept Prompt Tuning is a concept-conditioned prompt-learning paradigm that decomposes prompts into semantic components (e.g., clinical concepts or tasks) for improved modularity and transferability.
• It leverages mechanisms like CoPA’s concept-aware embeddings and SoftCPT’s task-conditioned prompts to inject semantic cues into frozen pretrained models.
• The approach addresses challenges in efficiency, stability, and acronym ambiguity while opening new directions for scalable and interpretable prompt-based adaptation.

Concept Prompt Tuning (CPT) is not a standardized method name in the prompt-tuning literature. A 2025 survey on prompt tuning states that it does not mention “Concept Prompt Tuning” or the acronym “CPT” as a method name, and instead treats it as a hypothetical prompt-tuning variant that would learn prompt components associated with semantic concepts and compose or select them at inference time (Li et al., 8 Jul 2025). In current usage, the label is most concretely attached to the CPT component of CoPA, where concept-aware embeddings are injected as prompts inside a frozen vision transformer for concept bottleneck diagnosis, and it is also used interpretively for methods such as SoftCPT, where the task itself functions as a concept that conditions prompt generation (Dong et al., 4 Oct 2025, Ding et al., 2022).

1. Terminological status and scope

The available literature supports two complementary readings of Concept Prompt Tuning. One is a generic, concept-conditioned prompt-learning paradigm in which prompts are factorized, routed, or transferred according to semantic concepts, tasks, domains, or attributes. The other is a specific mechanism inside CoPA, where concept-aware embeddings produced from human-defined clinical concepts are re-injected as prompts at each layer of a frozen visual encoder (Li et al., 8 Jul 2025, Dong et al., 4 Oct 2025). This suggests that the term currently functions more as an interpretive umbrella than as a settled taxonomy label.

Within that umbrella, the notion of “concept” is not uniform. In CoPA, concepts are explicit clinical attributes such as “Pigment Network”, “Dots and Globules”, and other concept-bottleneck variables. In SoftCPT, the concept is the task itself, represented by a task name and a learnable task context, then mapped to a prompt context through a shared meta network. In the survey’s hypothetical formulation, concepts can also be factors such as style, sentiment, task, or domain (Dong et al., 4 Oct 2025, Ding et al., 2022, Li et al., 8 Jul 2025).

Reference point	What functions as the concept	Prompt mechanism
Survey framing	semantic concept, task, or domain	composition or selection of prompt components
CoPA	human-defined clinical concept	layer-specific concept-aware embeddings used as prompts
SoftCPT	task	task-shared meta network generates prompt context

2. General formulation and taxonomy

The survey’s mathematical starting point is standard prompt tuning with a frozen LLM $f_\phi$, input sequence $x=[x_1,\dots,x_n]$, embedding layer $E(\cdot)$, and a learnable soft prompt matrix

$P \in \mathbb{R}^{m \times d}.$

Prompt tuning constructs

$$\tilde{X} = [P; E(x)] \in \mathbb{R}^{(m+n) \times d}$$

and optimizes

$$\min_{P} \; \mathbb{E}_{(x,y) \sim \mathcal{D}} \Big[ \mathcal{L}\big(f_\phi([P; E(x)]), y\big) \Big].$$

In the survey’s hypothetical CPT-like extension, the prompt is structured by semantic factors rather than treated as a single monolithic matrix (Li et al., 8 Jul 2025).

A canonical concept-level composition in that framing is

$P(x) = \sum_{k=1}^K \alpha_k(x) \, P^{(k)},$

where $P^{(k)}$ is a prompt associated with concept $k$, and $\alpha_k(x)$ is produced either by a router or by a concept encoder. This places CPT-like methods at the intersection of decomposition-based prompt tuning, mixture-of-prompts or mixture-of-experts prompt tuning, and transfer learning. The survey identifies decomposition-based methods such as DPT, DePT, and MPT; MoE-style methods such as SMoP and PT-MoE; and encoder-based transfer methods such as TransPrompt and CTPT as the closest existing families (Li et al., 8 Jul 2025).

That taxonomy matters because it clarifies what concept conditioning changes relative to conventional prompt tuning. A concept prompt may be a low-rank factor, a reusable expert, or an encoder-generated prompt; the common element is that the prompt space is no longer a single task-specific embedding block, but a structured space in which semantic components can be shared, composed, or transferred. The survey also emphasizes that such designs naturally target modularity, reusability, and interpretability, while inheriting open problems in computational efficiency, training stability, and explainability (Li et al., 8 Jul 2025).

3. CoPA and the explicit CPT mechanism

CoPA provides the clearest explicit realization of Concept Prompt Tuning as a named mechanism. Built on a frozen BiomedCLIP backbone, CoPA takes triplets $x=[x_1,\dots,x_n]$0, where $x=[x_1,\dots,x_n]$1 is an image, $x=[x_1,\dots,x_n]$2 is a set of human-defined concepts, and $x=[x_1,\dots,x_n]$3 is a disease label. For each concept $x=[x_1,\dots,x_n]$4, a candidate set $x=[x_1,\dots,x_n]$5 is defined, and textual concept embeddings are formed with templates of the form “This is a dermoscopic image, the {concept title} of the lesion is { $x=[x_1,\dots,x_n]$6 }.” (Dong et al., 4 Oct 2025)

The core component is the Concept-aware Embedding Generator (CEG). At visual layer $x=[x_1,\dots,x_n]$7, learnable concept anchors $x=[x_1,\dots,x_n]$8 query the layer’s token features $x=[x_1,\dots,x_n]$9 to produce concept-aware embeddings

$E(\cdot)$0

followed by

$E(\cdot)$1

These $E(\cdot)$2 are not merely outputs of a concept extractor; they are re-used as prompts for the next transformer layer (Dong et al., 4 Oct 2025).

CPT itself is the token-level injection rule

$E(\cdot)$3

where $E(\cdot)$4 is the class token, $E(\cdot)$5 the patch tokens, and $E(\cdot)$6 the concept prompts from the previous layer. The BiomedCLIP backbone remains frozen; training updates only the concept anchors, the FFN and layer normalization in CEG, the multilayer selector, the gating network, the final classifier, and the contrastive temperature (Dong et al., 4 Oct 2025).

This mechanism turns concept learning into a recurrent feedback process. CEG first extracts layer-wise concept-aware embeddings from the current visual features; CPT then injects those embeddings back into the transformer so that subsequent self-attention can amplify concept-related visual cues. The paper describes this as a way for self-attention within each layer to “progressively amplify target visual concepts,” while keeping the pretrained medical backbone intact (Dong et al., 4 Oct 2025).

4. Hierarchical concept bottlenecks and interpretability

CoPA’s CPT is embedded in a multilayer concept bottleneck. For each concept $E(\cdot)$7, the layer-wise embeddings $E(\cdot)$8 are aggregated by a linear selector into a final visual concept embedding $E(\cdot)$9. This embedding is aligned against textual concept candidates $P \in \mathbb{R}^{m \times d}.$0 through the contrastive concept alignment loss

$P \in \mathbb{R}^{m \times d}.$1

A gated aggregation module then forms the disease prediction

$P \in \mathbb{R}^{m \times d}.$2

and the total objective is

$P \in \mathbb{R}^{m \times d}.$3

Here the concepts are not post hoc explanatory tags; they are active prompt tokens that influence the internal representation at every layer (Dong et al., 4 Oct 2025).

This architecture gives CoPA a strong concept-bottleneck interpretation. Concept predictions arise from similarities between $P \in \mathbb{R}^{m \times d}.$4 and the textual concept candidates, while the gating weights $P \in \mathbb{R}^{m \times d}.$5 quantify each concept’s relevance to the disease prediction. The paper reports concept activation heatmaps, concept confidence scores, gating weights, and intervention experiments in which modifying concept predictions materially changes disease predictions, all of which support the claim that concepts are integrated into the predictive pathway rather than appended afterward (Dong et al., 4 Oct 2025).

Empirically, CoPA reports state-of-the-art results on PH$P \in \mathbb{R}^{m \times d}.$6, Derm7pt, and SkinCon. Disease AUC/ACC/F1 are reported as 98.3/98.9/98.8 on PH$P \in \mathbb{R}^{m \times d}.$7, 92.1/86.0/85.8 on Derm7pt, and 77.5/76.3/75.7 on SkinCon. Concept prediction AUC/ACC/F1 are reported as 89.0/82.2/80.6, 87.0/77.1/76.6, and 81.7/93.6/70.4 on the same datasets. The ablation study further shows that adding CPT alone already improves label accuracy, and that the full combination of multilayer aggregation, CPT, and a frozen vision backbone yields the strongest label and concept performance (Dong et al., 4 Oct 2025).

5. Related realizations: task-conditioned prompts and text-side concept modules

SoftCPT, although not named “Concept Prompt Tuning,” is explicitly described as a concrete realization of task-/concept-level prompt tuning for vision-LLMs. It treats each task $P \in \mathbb{R}^{m \times d}.$8 as a concept described by a task name, computes a task feature

$P \in \mathbb{R}^{m \times d}.$9

and maps that feature to a task-specific prompt context through

$$\tilde{X} = [P; E(x)] \in \mathbb{R}^{(m+n) \times d}$$0

These prompt tokens are then prepended to class names for CLIP-style classification. SoftCPT’s central distinction is soft sharing: prompts are neither globally shared across all tasks nor learned independently, but generated by a shared meta network conditioned on task semantics (Ding et al., 2022).

The method was evaluated across four multi-task few-shot datasets covering 44 tasks and 1593 categories. SoftCPT-NATA improved average accuracy over CoOp-CA from 66.39% to 67.12% on General-10, from 62.59% to 67.68% on Plant-6, from 73.23% to 76.86% on RS-8, and from 59.95% to 62.75% on Fashion-20. The paper also reports that the correlation between an oracle task-similarity matrix and SoftCPT’s multi-task prompt similarity matrix is about 0.32, compared with about $$\tilde{X} = [P; E(x)] \in \mathbb{R}^{(m+n) \times d}$$1 for single-task CoOp, which indicates that the learned prompt space better reflects actual task relatedness (Ding et al., 2022).

A text-side analogue appears in clinical concept and relation extraction, where task-specific soft prompts were used to steer frozen or unfrozen clinical LLMs toward trigger concepts, attribute concepts, and relation patterns. In that study, soft prompting with unfrozen GatorTron-3.9B achieved strict F1-scores of 0.9118 and 0.8604 for concept extraction, and soft prompting with frozen large models became competitive with unfrozen models; for cross-institute evaluation, soft prompting with a frozen GatorTron-8.9B achieved the best performance (Peng et al., 2023). This suggests that concept-oriented prompt modules are not confined to vision-language concept bottlenecks; they also appear as continuous concept interfaces in clinical NLP.

6. Limitations, acronym ambiguity, and open directions

The literature also makes clear that Concept Prompt Tuning remains methodologically unsettled. The survey identifies computational efficiency, training stability, and explainability as persistent prompt-tuning challenges (Li et al., 8 Jul 2025). CoPA depends on predefined concepts, concept candidate sets, and textual concept templates, which ties performance to concept vocabulary quality and label granularity. SoftCPT assumes meaningful task names and some degree of semantic relatedness among tasks; its gains are larger on specialized domains than on heterogeneous multi-task mixtures (Dong et al., 4 Oct 2025, Ding et al., 2022).

A further source of confusion is acronym overload. In arXiv usage, “CPT” also denotes “Colorful Prompt Tuning” for visual grounding (Yao et al., 2021), “Configurable Preference Tuning” for rubric-guided preference optimization (Gallego, 13 Jun 2025), “Contrastive Prompt Tuning” in both language and vision-language settings (Xu et al., 2022, Zhu et al., 2024), “Cluster-Aware Neural Collapse Prompt Tuning” for long-tailed vision-language generalization (Guo et al., 12 May 2026), and “Controllable Prompt Tuning” for group distributional robustness (Phan et al., 2024). The acronym therefore does not identify a single family unless the surrounding context is explicit.

Taken together, these works suggest a stable conceptual core even in the absence of a standardized name. Concept Prompt Tuning, in the broadest defensible sense, refers to prompt-learning schemes in which semantic factors—clinical concepts, tasks, domains, attributes, or reusable prompt experts—are represented as tunable or generated prompt vectors, usually on top of frozen pretrained backbones, and are trained to support modularity, transfer, and interpretability (Li et al., 8 Jul 2025, Dong et al., 4 Oct 2025, Ding et al., 2022). The main open question is not whether prompts can encode concepts, but how to make those concept encodings stable, scalable, and faithful across tasks, modalities, and deployment settings.