--- title: Concept Prompting and Aggregating (CoPA) url: https://www.emergentmind.com/topics/concept-prompting-and-aggregating-copa type: topic --- # Concept Prompting and Aggregating (CoPA) Concept Prompting and Aggregating (CoPA) is a name used for several concept-centric frameworks that couple an explicit prompting or concept-construction stage with a subsequent aggregation stage. In the arXiv literature, the term denotes at least three distinct instantiations: a text-scaling framework based on concept-guided chain-of-thought prompting and pairwise comparison scoring; a zero-shot image-recognition framework that marginalizes over class-specific concepts in a Bayesian formulation; and a hierarchical concept-prompting network for explainable diagnosis that aggregates multilayer visual concept representations [2310.12049] [2603.07911] [2510.03767]. ## 1. Terminological scope and recurring structure The acronym CoPA does not refer to a single canonical algorithm. In one usage, it is a text scoring framework summarized as Concept-Guided Chain-of-Thought (CGCoT) prompting with pairwise comparison scaling. In a second usage, it is a Concept-Guided Bayesian Classification framework for zero-shot image recognition, explicitly described as also being referred to as Concept Prompting and Aggregating. In a third usage, it names a “Concept Prompting and Aggregating Network” for explainable diagnosis [2310.12049] [2603.07911] [2510.03767]. A shared pattern nevertheless recurs across these works. Each framework first constructs concept-sensitive intermediate representations rather than operating directly on a raw input alone. It then aggregates those intermediate objects into a final score, posterior, or diagnosis. In the text case, the intermediate object is a concept-specific breakdown \(B_i\) and the aggregation is a Bradley–Terry latent score. In the zero-shot vision case, the intermediate objects are class-specific concepts \(c\) and the aggregation is Bayesian marginalization with adaptive soft-trim refinement. In the diagnosis case, the intermediate objects are layer-wise concept embeddings \(z_i^l\) and the aggregation is a learned cross-layer combination aligned with textual concept embeddings. ## 2. CoPA as concept-guided pairwise text scaling In "Concept-Guided Chain-of-Thought Prompting for Pairwise Comparison Scoring of Texts with Large Language Models," CoPA consists of two sequential stages: concept prompting and aggregating pairwise comparisons. For each text \(T_i\), a researcher-designed series of sub-prompts elicits a concept-specific breakdown \(B_i\). The LLM is prompted, in sequence, to summarize the text, to identify the focal entity, and to judge the presence or intensity of the target concept relative to that entity. The resulting breakdown is the concatenation \[ \bigl[T_i;\; \mathrm{Answer}_1;\; \mathrm{Answer}_2;\;\mathrm{Answer}_3;\;\mathrm{Answer}_4\bigr]. \] Once every text has an associated breakdown, the LLM compares pairs \((B_i,B_j)\) and indicates which breakdown exhibits more of the concept, or returns “Tie.” These wins, losses, and ties are then fed into a Bradley–Terry model to produce a continuous latent score \(\lambda_i\) for each text [2310.12049]. The pairwise comparison stage is formalized through \[ p_{ij}=\Pr\bigl(\text{“\(B_i\) exhibits more of the concept than \(B_j\)”}\bigr), \] with hard outcomes encoded as \[ w_{ij} = \begin{cases} 1, & \text{if } B_i \text{ wins},\\ 0.5, & \text{if “Tie,”}\\ 0, & \text{if } B_j \text{ wins.} \end{cases} \] Under the Bradley–Terry model, \[ \Pr(i\succ j)=\frac{\exp(\lambda_i)}{\exp(\lambda_i)+\exp(\lambda_j)}=\sigma(\lambda_i-\lambda_j), \] and the parameters \(\{\lambda_i\}_{i=1}^N\) are fitted by maximizing the tie-adjusted log-likelihood \[ \mathcal{L}(\lambda) =\sum_{i\lambda\,\mathrm{MAD}_i\bigr], \] sets \[ \alpha_i = \log\bigl((1-\hat\rho_i)/\hat\rho_i\bigr), \] and defines weights \[ w_{i,j} = \sigma\!\Bigl(-\,\alpha_i\;\frac{\lvert s_{i,j}-m_i\rvert}{\mathrm{MAD}_i}\;k\Bigr). \] The final refined class score is \[ \hat p(y_i\mid x) = \frac{\sum_{j=1}^{M_i}w_{i,j}\,p(y_i\mid x,c_{i,j})} {\sum_{j=1}^{M_i}w_{i,j}}. \] The paper emphasizes that this requires only one forward pass and automatically down-weights extreme outliers. The theoretical analysis is stated under a Huber-contamination model with sub-Gaussian inliers. If \[ M_i \ge \frac{C_0\ln(1/\delta)}{\rho_i^2}, \] then with probability at least \(1-\delta\), \[ \bigl|\hat\mu_i-\mu_i\bigr| \le C_1\,\sigma_i\,\rho_i + C_2\,\sigma_i\, \sqrt{\frac{\ln(1/\delta)}{M_i} + \frac{C_3\,\sigma_i}{k}}. \] A corollary bounds multi-class excess risk through the margin event \[ \mathcal R(\hat h)-\mathcal R^* \le \Pr\!\Bigl[ \mathrm{margin}_\mu(x)\le 2\max_i\lvert\hat\mu_i(x)-\mu_i(x)\rvert \Bigr]. \] Implementation details are concrete: GPT-4.1-Turbo (or Mini/Nano) for concept synthesis, \(H=10\), \(M_A=50\), \(|\widehat{\mathcal{C}}_i|=500\) with \(k=3\), DPP subset size \(M_i\in\{16,50\}\), outlier threshold \(\lambda=2.5\), sigmoid slope \(k=e^{4.6}\), offline DPP complexity \(O(|\widehat{\mathcal{C}}_i|^3)\), and inference cost \(O(K\,M_i\cdot d)\). ## 4. CoPA as a hierarchical concept-prompting network for explainable diagnosis In "CoPA: Hierarchical Concept Prompting and Aggregating Network for Explainable Diagnosis," CoPA is a supervised concept-based architecture for clinical imaging. It consists of four main components: a multilayer visual encoder, a Concept-aware Embedding Generator (CEG), Concept Prompt Tuning (CPT), and an aggregation-and-alignment module. A pre-trained vision backbone such as ViT is split into \(L\) sequential layers \(\{\Phi_1,\dots,\Phi_L\}\), each producing token-wise features \(F^l\). CEG maintains learnable concept anchors \(\{q_i\in\mathbb{R}^d\}_{i=1}^N\), one per concept, and uses them to query each layer’s features to produce concept embeddings \(z_i^l\). CPT then treats these layer-wise embeddings as prompt tokens injected into the next transformer layer while keeping backbone weights frozen, and the final module aggregates \(\{z_i^l\}_{l=1}^L\) into a final visual concept representation \(Z^i\), aligns \(Z^i\) with textual concept embeddings, and fuses the chosen text embedding \(D_i\) into a gated aggregation for disease prediction [2510.03767]. For flattened key/value matrices \(K_l,V_l\in\mathbb{R}^{m_l\times d}\), the CEG computations are \[ \hat z^l_i = \mathrm{Softmax}\!\Bigl(\tfrac{q_i\,K_l^\top}{\sqrt{d}}\Bigr)\,V_l \quad\in\mathbb{R}^d, \] \[ z^l_i = \mathrm{LayerNorm}\!\bigl(\mathrm{FFN}(\hat z^l_i)+q_i\bigr) \quad\in\mathbb{R}^d. \] At transformer layer \(l\), the model prepends the concept prompts from layer \(l-1\): \[ [\mathbf{a}_{l-1};\,\mathbf{Z}_{l-1};\,\mathbf{P}_{l-1}] \longmapsto [\mathbf{a}_{l};\;\ldots;\;\mathbf{P}_{l}] = \Phi_l\!\bigl([\mathbf{a}_{l-1},\,\mathbf{Z}_{l-1},\,\mathbf{P}_{l-1}]\bigr). \] Only \(\{\mathbf{Z}_l\}\) and the concept anchors \(\{q_i\}\) are updated; the backbone parameters are frozen. The cross-layer aggregation is defined as \[ Z^i=\sum_{l=1}^L w_l\,z_i^l, \] where \(\{w_l\}\) are learned by a small selector network. For each concept \(i\), the textual candidate set \(\mathcal{C}_i=\{c_i^1,\ldots,c_i^{k_i}\}\) is embedded by a frozen text encoder into \[ T_i=[t_i^1,\dots,t_i^{k_i}]\in\mathbb{R}^{k_i\times d}. \] Alignment uses the contrastive loss \[ \mathcal{L}_{\mathrm{align}} = -\sum_{i=1}^N \log \frac{\exp\!\bigl(\cos(Z^i,t_i^*)/\tau\bigr)} {\sum_{j=1}^{k_i}\exp\!\bigl(\cos(Z^i,t_i^j)/\tau\bigr)}, \] where \(t_i^*\) is the ground-truth candidate and \(\tau\) is a learnable temperature. Disease diagnosis is trained jointly with concept alignment through \[ \mathcal{L}_{\mathrm{diagnosis}} = \mathrm{CE}(\hat y,y), \qquad \hat y = \mathrm{FC}\Bigl(\sum_{i=1}^N \alpha_i\,D_i\Bigr), \] and the full loss \[ \mathcal{L} = \lambda_1\,\mathcal{L}_{\mathrm{concept}} + \lambda_2\,\mathcal{L}_{\mathrm{diagnosis}}, \qquad \lambda_1+\lambda_2=1. \] In practice, \(\lambda_1=\lambda_2=0.5\). The optimization details given are Adam, learning rate \(1\times10^{-5}\), batch size \(32\) (typical), and \(50\) epochs with early stopping on validation AUC. BiomedCLIP provides the pre-trained backbone initialization, the text encoder is frozen, and only prompts and query anchors are updated beyond the gating network. The experimental evaluation uses three public datasets with a \(70\%/15\%/15\%\) split: PH\(^2\) with 200 dermoscopic images and 5 morphological concepts; Derm7pt with 1,011 images and 7 checklist concepts; and SkinCon with 3,690 clinical photos, 22 high-frequency skin features, and 3 disease classes. Reported disease-diagnosis test-set averages are: PH\(^2\), AUC \(98.3\), accuracy \(98.9\), F1 \(98.8\); Derm7pt, AUC \(92.1\), accuracy \(86.0\), F1 \(85.8\); SkinCon, AUC \(77.5\), accuracy \(76.3\), F1 \(75.7\). Reported concept-prediction results are: PH\(^2\), AUC \(89.0\), accuracy \(82.2\), F1 \(80.6\); Derm7pt, AUC \(87.0\), accuracy \(77.1\), F1 \(76.6\); SkinCon, AUC \(81.7\), accuracy \(93.6\), F1 \(70.4\). The ablation study compares Baseline, MLA only, CPT only, CPT and Freeze, MLA and CPT, and Full (MLA and CPT and Freeze). The full configuration yields PH\(^2\) label accuracy/F1 \(98.9/98.8\) and concept accuracy/F1 \(82.2/80.6\), and Derm7pt label accuracy/F1 \(86.0/85.8\) and concept accuracy/F1 \(77.1/76.6\). Qualitative analysis reports concept heatmaps concentrated around expert-annotated regions, gate weights \(\{\alpha_i\}\) correlated with clinical relevance, and predictable shifts under test-time intervention by flipping a concept’s confidence. ## 5. Aggregation mechanisms across CoPA variants Although the three frameworks operate in different domains, each instantiates aggregation in a formally explicit way. | Variant | Prompting or concept stage | Aggregation stage | |---|---|---| | CGCoT text scoring | Researcher-designed sub-prompts produce breakdowns \(B_i\) | Bradley–Terry latent score \(\lambda_i\) and normalized score \(s_i\) | | CGBC zero-shot recognition | LLM-generated atomic and compositional concepts selected by DPP | Bayesian marginalization and adaptive soft-trim refinement | | Explainable diagnosis network | CEG and CPT produce layer-wise concept embeddings \(z_i^l\) | Learned multilayer aggregation, contrastive alignment, and gated diagnosis | In the text-scaling version, aggregation turns pairwise judgments into a continuous latent score. In the zero-shot vision version, aggregation marginalizes over a set of selected concepts and then reweights prompt contributions to suppress outliers. In the diagnosis version, aggregation operates across representational depth, combining concept information extracted from multiple transformer layers. This suggests that “aggregating” in CoPA is not tied to a single estimator; rather, it denotes a general commitment to making the final prediction depend on explicit concept-level intermediates rather than on an undifferentiated end-to-end latent representation. A related distinction concerns what counts as a “prompt.” In the text framework, prompts are sequential natural-language instructions posed to an LLM. In the zero-shot image framework, prompts are class descriptions of the form “A photo of \(y\) with \(c\).” In the diagnosis framework, prompts are injected concept tokens \(\mathbf{Z}_{l-1}\) within a transformer. The terminology is therefore stable at the level of design intent but heterogeneous at the level of implementation. ## 6. Empirical properties, limitations, and extensions The text-scoring CoPA emphasizes precise targeting of abstract concepts via researcher-crafted prompts, no or minimal reliance on large labeled corpora, robust continuous scaling with meaningful intervals through Bradley–Terry, and competitive binary classification performance against fully supervised LLMs. Its stated limitations are manual and concept-specific prompt design, dependence on LLM idiosyncrasies, the sampling trade-off between stability and API cost, and the black-box nature of LLM reasoning steps [2310.12049]. The zero-shot image-recognition CoPA is designed to improve prompt quality through class-specific concepts, diversity enforcement by DPP, and robustness to outlier prompts through adaptive soft-trim likelihood. Its theoretical section supplies a robust guarantee under a Huber-contamination model and a multi-class excess-risk corollary. The implementation summary also makes its computational trade-off explicit: offline DPP selection per class is \(O(|\widehat{\mathcal{C}}_i|^3)\), typically \(500^3\), whereas inference is \(O(K\,M_i\cdot d)\) after text prompts are pre-cached [2603.07911]. The explainable-diagnosis CoPA emphasizes multilayer concept capture under prompt guidance, preservation of pre-trained vision-language alignment through frozen backbones, and effective use of concept-wise information for concept and disease prediction. Its discussion identifies reliance on human-annotated concepts and increased compute or memory for storing \(L\times N\) prompts as limitations. The same section proposes automatic concept discovery via clustering of \(F^l\), dynamic prompt pruning, and application to other modalities such as radiology and to weakly supervised settings [2510.03767]. A common misconception is to treat CoPA as synonymous with a single text-oriented prompting pipeline. In the cited literature, the acronym names three different architectures with different supervision regimes and different notions of prompting. The text version is unsupervised beyond prompt-tuning, the zero-shot image version is a Bayesian prompt-aggregation framework with training-free adaptive soft-trim at inference, and the diagnosis version is a supervised concept-alignment model trained with concept and disease losses. A plausible implication is that CoPA is best understood as a reusable methodological pattern: define concept-sensitive intermediates, then aggregate them with an explicitly specified probabilistic or architectural mechanism. Source: https://www.emergentmind.com/topics/concept-prompting-and-aggregating-copa