PGCMs: Prototype-Grounded Concept Models

• Prototype-Grounded Concept Models (PGCMs) represent concepts with explicit, learned prototype vectors, making semantic information concrete and inspectable.
• They employ diverse architectures such as iCSNs, multi-modal networks, and ProtoPNets that integrate latent swapping, clustering losses, and projection methods for robust concept grounding.
• Empirical results show enhanced zero-shot classification, interactive debugging, and verifiable intervention, though challenges in fine-grained concept communication persist.

Prototype-Grounded Concept Models (PGCMs) are a family of models in which concepts are represented explicitly by prototypes rather than only by abstract latent coordinates or scalar bottleneck activations. Across the literature, the term has been used in several closely related ways: as a concept-learning framework with discrete prototype slots and latent swapping in Interactive Concept Swapping Networks (iCSNs) (Stammer et al., 2021); as a multi-modal architecture in which each perceptual category is represented by a single prototype vector that drives both description generation and zero-shot classification (Noble et al., 2023); as a “gray-box” classifier family in which predictions are made by matching unseen inputs against learned prototypes (Bontempelli et al., 2022); and as a concept-bottleneck-style architecture in which each concept is tied to learned visual prototypes, enabling direct inspection of concept semantics and targeted intervention at the prototype level (Colamonaco et al., 17 Apr 2026). In all of these formulations, the central claim is not merely that prototypes aid prediction, but that they make concept semantics concrete, inspectable, and, in some settings, editable.

1. Definitions and conceptual scope

The 2023 vision–language formulation defines a PGCM as “a multi-modal neural architecture that represents each perceptual category $\mathcal{C}$ by a single prototype vector $\mathbf{p}_{\mathcal{C}\in\mathbb{R}^d}$,” with both generation of natural-language descriptions and zero-shot classification driven by these prototype vectors (Noble et al., 2023). In that account, prototype-only representations are contrasted with exemplar-based models that store one or more specific example embeddings per category, and the stated motivation is that a single abstract prototype emphasizes “the central tendency of the category,” yielding “more compact representations” and “(often) better generalisation in zero-shot settings” (Noble et al., 2023).

A broader machine-learning definition appears in work on ProtoPNets and concept-level debugging, where Prototype-Grounded Concept Models are described as a family of “gray-box” classifiers built on two core ideas: concepts are represented explicitly as a finite set of learned prototypes in latent space, and predictions are made by matching unseen inputs against these concept prototypes (Bontempelli et al., 2022). In that usage, interpretability and transparency are co-equal design goals: every prototype corresponds to a concrete part or example drawn from the training data, and the decision procedure is a distance-based matching stage followed by a simple aggregation (Bontempelli et al., 2022).

The 2026 concept-alignment formulation narrows the emphasis further. It presents PGCMs as revisiting the familiar “input→concepts→task” pipeline of Concept Bottleneck Models while removing “the unspoken—and uncheckable—assumption that an abstract scalar concept ‘means what its name says’” (Colamonaco et al., 17 Apr 2026). There, each concept is explicitly tied to a finite bank of learned visual prototypes, and this grounding is presented as the mechanism that enables “verifiable alignment” and prototype-level intervention (Colamonaco et al., 17 Apr 2026).

Outside contemporary deep learning, a much earlier cognitive formalization represents prototypes and features as vectors and subspaces in a complex Hilbert space, using state vectors, prototype vectors, and entangled measurements to model typicality, graded membership, context dependence, and concept combination (Aerts, 2013). This suggests that the phrase “prototype-grounded” spans both interpretable neural architectures and formal theories in which prototype structure is treated as fundamental to concept representation.

2. Architectural patterns

Despite substantial variation, several recurring architectural patterns appear across PGCM instantiations.

In iCSNs, the model builds on a deterministic autoencoder with encoder $f_\theta$ and decoder $g_\phi$, but inserts concept read-out modules and prototype slots into the latent space (Stammer et al., 2021). The encoded latent vector $z=f_\theta(x)\in\mathbb{R}^Z$ is transformed by category-specific read-out encoders $\phi_j=m_j(z)\in\mathbb{R}^Q$, and each category $j$ has a codebook of learnable prototype vectors $\mathrm{P}_j=[p_j^1,\dots,p_j^{K_j}]$ (Stammer et al., 2021). A scaled dot-product followed by two nested softmaxes yields softened but sharpenable assignments $\Pi_j$, which are concatenated into a joint discrete code $y=[\Pi_1|\Pi_2|\dots|\Pi_J]$ and decoded to reconstruct the input (Stammer et al., 2021).

In the multi-modal perceptual-description setting, the architecture is explicitly split into two separately trained networks, Generator and Interpreter (Noble et al., 2023). Both share a vision encoder and a label embedding matrix $\mathbf{p}_{\mathcal{C}\in\mathbb{R}^d}$0; in the Generator, each row $\mathbf{p}_{\mathcal{C}\in\mathbb{R}^d}$1 acts as the prototype $\mathbf{p}_{\mathcal{C}\in\mathbb{R}^d}$2, and in the Interpreter the same matrix contains the target prototypes for classification (Noble et al., 2023). The Generator uses a Transformer decoder conditioned on the prototype, while the Interpreter uses a BERT-based text encoder plus a projection layer intended to match one of the prototypes (Noble et al., 2023).

In ProtoPNets, the architecture comprises three stages: a pre-trained convolutional feature extractor mapping an image to a latent tensor, a part-prototype matching stage over $\mathbf{p}_{\mathcal{C}\in\mathbb{R}^d}$3 latent patches, and a linear aggregation stage with class-specific weights (Bontempelli et al., 2022). Prototype activations are computed by either a “difference of logs” function or an exponential form, both monotonically decreasing in prototype–patch distance, followed by max-pooling over latent patches (Bontempelli et al., 2022). The resulting explanation template is explicit: a prediction is justified because the image strongly matches specific part-prototypes that are themselves image patches from training examples (Bontempelli et al., 2022).

The 2026 PGCM architecture is closer to a latent-variable concept bottleneck model (Colamonaco et al., 17 Apr 2026). It uses a shared encoder $\mathbf{p}_{\mathcal{C}\in\mathbb{R}^d}$4, a bank of learnable prototype embeddings $\mathbf{p}_{\mathcal{C}\in\mathbb{R}^d}$5, an image decoder $\mathbf{p}_{\mathcal{C}\in\mathbb{R}^d}$6, a concept decoder $\mathbf{p}_{\mathcal{C}\in\mathbb{R}^d}$7, and a transparent task predictor $\mathbf{p}_{\mathcal{C}\in\mathbb{R}^d}$8 (Colamonaco et al., 17 Apr 2026). Each image part $\mathbf{p}_{\mathcal{C}\in\mathbb{R}^d}$9 is assigned a latent variable $f_\theta$0 with a uniform prior, and the posterior selector is a softmax over prototype similarities (Colamonaco et al., 17 Apr 2026). Conditioned on a selected prototype, the model simultaneously reconstructs the image part and predicts its concept vector (Colamonaco et al., 17 Apr 2026).

Other variants retain the prototype-centered logic while changing the surrounding machinery. Concept-Guided Prototype Networks store prototypes in a learnable matrix partitioned into positive and negative pools for each binary concept, compute each prototype’s activation by negative Euclidean distance to the nearest patch, and feed the resulting similarity vector into a linear concept-prediction head (Carballo-Castro et al., 2024). The Locality-aware Concept Bottleneck Model associates one prototype with each concept, computes prototype–patch cosine similarities over spatial features, and uses CLIP-guided masking plus attention-based aggregation to improve localization (Jeon et al., 20 Aug 2025).

3. Grounding mechanisms and training objectives

The distinguishing feature of PGCMs is not only the presence of prototypes, but the mechanism by which they are semantically bound to concepts.

In iCSNs, the key grounding mechanism is latent swapping between paired inputs that share a specific category (Stammer et al., 2021). For two inputs that share exactly the basic concept in category $f_\theta$1, the model swaps only the $f_\theta$2-th category’s prototype assignments between their codes, then requires the swapped decodings to reconstruct the original images (Stammer et al., 2021). The stated effect is that the $f_\theta$3-th prototype slot is forced to encode only the category-$f_\theta$4 concept, because that is the only information being exchanged (Stammer et al., 2021). Training uses a single paired reconstruction loss, with no explicit KL, mutual-information, clustering, or sparsity terms; the report states that “the swapping plus the sharpened softmax” suffice to bind category information to prototype slots (Stammer et al., 2021).

In ProtoPNets, grounding is enforced through cluster and separation regularizers together with projection of prototypes onto actual latent patches from training examples (Bontempelli et al., 2022). The loss combines the negative log-likelihood of the class label with $f_\theta$5 and $f_\theta$6, where the former encourages each training example to be close to a same-class prototype and the latter repels it from prototypes of other classes (Bontempelli et al., 2022). Projection then ensures that prototypes correspond to actual image regions rather than unconstrained latent points (Bontempelli et al., 2022).

In the 2026 concept-alignment PGCM, grounding is probabilistic and joint: prototypes are responsible both for reconstructing image parts and for predicting concept vectors (Colamonaco et al., 17 Apr 2026). The total loss is written as

$f_\theta$7

where the reconstruction term forces the prototype bank to “faithfully cover the visual variability of parts,” and the concept loss forces each prototype to carry semantically correct concept labels $f_\theta$8 (Colamonaco et al., 17 Apr 2026). This is presented as a VAE-style ELBO objective on the joint of parts, concepts, and task labels (Colamonaco et al., 17 Apr 2026).

In Concept-Guided Prototype Networks, grounding is reinforced by prototype-conformity losses and, optionally, orthogonality losses (Carballo-Castro et al., 2024). The cluster cost pushes at least one patch of an example toward a same-class prototype, the separation cost repels patches from opposite-class prototypes, and the total loss may combine classification, prototype conformity, diffusion, and orthogonality terms (Carballo-Castro et al., 2024). In Locality-aware CBM, grounding is spatial: a locality loss matches concept influence distributions to masked prototype-similarity distributions via per-concept KL divergence, while an auxiliary classification head regularizes aggregated prototype representations (Jeon et al., 20 Aug 2025).

A different grounding mechanism appears in the multi-modal PGCM for category descriptions. There, prototypes are not attached to local image parts but to categories themselves, and both language generation and interpretation are conditioned on the same prototype vectors (Noble et al., 2023). The interpreter’s text projection is trained with a cosine-embedding loss with margin $f_\theta$9, and the Generator combines a visual classification term with an autoregressive language loss (Noble et al., 2023). This formulation grounds prototypes at the category level rather than at the part level.

4. Interpretability, alignment, and intervention

A persistent theme across PGCM work is that prototype grounding should make concept semantics inspectable rather than merely interpretable in principle.

The 2026 formulation makes this explicit by defining the semantics of concept $g_\phi$0 as

$g_\phi$1

It proposes a “Concept Alignment Table” in which each row shows the image prototype $g_\phi$2 and its concept vector $g_\phi$3, so that a human can directly verify whether the visual evidence matches the concept assignment (Colamonaco et al., 17 Apr 2026). Once a misaligned prototype is identified, the stated interventions include removing the prototype, editing its concept labels, or adding a new prototype sourced from a real image part (Colamonaco et al., 17 Apr 2026). The same report also notes that prototype-selection interventions—fixing $g_\phi$4 to prefer or avoid certain prototypes—provide an additional level of low-level control (Colamonaco et al., 17 Apr 2026).

Interactive repair is also central to iCSNs. At test time, the model can project an input through its prototype assignment pipeline, present the user with prototypical reconstructions of each concept slot by decoding one-hot codes, and accept logical constraints that merge or prune prototype slots for curated subsets of inputs (Stammer et al., 2021). These constraints are implemented as simple $g_\phi$5 penalties on relevant coordinates of the discrete code during fine-tuning (Stammer et al., 2021).

ProtoPDebug extends the intervention idea to concept-level debugging of ProtoPNets (Bontempelli et al., 2022). Users inspect the top-activating training images for each prototype and label a prototype as confounded or high-quality; the corresponding image cut-outs are then stored in forbidden or valid concept sets (Bontempelli et al., 2022). Fine-tuning introduces a forgetting loss

$g_\phi$6

and a remembering loss

$g_\phi$7

which are designed respectively to discourage activation on confounders and preserve valid concepts (Bontempelli et al., 2022). The paper frames this as concept-level rather than pixel-level debugging.

The ProtoPNet and concept-guided prototype literature also emphasizes faithful visualization. ProtoPNet explanations are case-based: each prediction can be traced to specific learned part-prototypes drawn from training data (Bontempelli et al., 2022). Concept-Guided ProtoPNet and ProtoPools report that each learned prototype can be visualized via its nearest training patch and overlaid on new inputs to show “this looks like that” explanations (Carballo-Castro et al., 2024). LCBM likewise treats the reshaped prototype–patch similarity map $g_\phi$8 as a localization map for concept $g_\phi$9, with thresholding or GradCAM used to visualize spatial support (Jeon et al., 20 Aug 2025).

A plausible implication is that PGCMs are best understood not as a single architecture class, but as a family of mechanisms for making concept evidence inspectable at the level of prototypes, patches, or prototype-conditioned generations.

5. Representative empirical results

Reported results vary substantially because the models address different tasks, supervision regimes, and evaluation protocols.

For iCSNs on the Elementary Concept Reasoning benchmark, the reported quantitative findings are unusually strong for a weakly supervised concept learner (Stammer et al., 2021). The model attains near-zero latent-code variance for all concepts and outperforms $z=f_\theta(x)\in\mathbb{R}^Z$0-VAE, Ada-VAE, Gaussian VAE with pair-wise averaging, and a categorical (Gumbel-softmax) VAE on that metric (Stammer et al., 2021). On linear probing of the learned one-hot-like code, iCSN achieves “≈ 100 % accuracy,” while all VAE baselines plateau at “70–98 %” (Stammer et al., 2021). Additional ablations report that replacing prototype discretization with categorical Gumbel-softmax “reduces performance drastically,” and replacing latent swapping by simply averaging pair encodings “also collapses performance” (Stammer et al., 2021).

In the same iCSN study, online addition of novel concepts is explicitly quantified (Stammer et al., 2021). Introducing a new basic concept, half-circle, and binding it to an empty shape-slot via user feedback raises probing accuracy from “≈ 79 %” to “≈ 98 %,” while adding a new superordinate concept, spot/no-spot, by inserting a new read-out encoder and codebook pushes spot classification accuracy from “≈ 67 %” to “≈ 98.3 %” (Stammer et al., 2021).

The 2023 multi-modal PGCM yields more mixed results (Noble et al., 2023). On CUB-200–2011 with five zero-shot folds, the interpreter taught by one ground-truth description achieves acc@1 = 19.1%, acc@5 = 44.1%, mean rank = 30.0, whereas when taught by Gen-Ex descriptions it reaches acc@1 = 8.6%, acc@5 = 25.1%, mean rank = 46.0 (Noble et al., 2023). In the direct comparison among generator variants, prototype-only Gen-Prot with beam search obtains BLEU-1 = 0.61, BLEU-4 = 0.55, CIDEr = 1.80, mean rank = 73.0, acc@1 = 2.7%, and acc@5 = 13.6%, while exemplar-based Gen-Ex with beam search achieves BLEU-1 = 0.64, BLEU-4 = 0.58, CIDEr = 1.92, mean rank = 46.0, acc@1 = 8.6%, and acc@5 = 25.1% (Noble et al., 2023). The paper’s stated conclusion is that prototype-grounded generation can produce plausible category descriptions but “struggle[s] to convey fine-grained distinctions unless supported by exemplar information” (Noble et al., 2023).

For concept prediction with interpretable prototypes, Concept-Guided ProtoPNet achieves up to $z=f_\theta(x)\in\mathbb{R}^Z$1 and $z=f_\theta(x)\in\mathbb{R}^Z$2, while Concept-Guided ProtoPools reaches up to $z=f_\theta(x)\in\mathbb{R}^Z$3 and $z=f_\theta(x)\in\mathbb{R}^Z$4; the cited black-box oracle from Koh et al. (2020) is $z=f_\theta(x)\in\mathbb{R}^Z$5 and $z=f_\theta(x)\in\mathbb{R}^Z$6 (Carballo-Castro et al., 2024). The discussion characterizes this as performance “close to black-box oracles (within ~5 %)” (Carballo-Castro et al., 2024).

The 2026 PGCM for verifiable concept alignment reports dataset-specific comparisons against CBM-style baselines (Colamonaco et al., 17 Apr 2026). On ColorMNIST+, it achieves 99.2% concept accuracy and 99.7% task accuracy, described as “identical to CBM’s 99.2/99.6” (Colamonaco et al., 17 Apr 2026). On CelebA, it reports 78.5% concept accuracy versus 81.3% for CBM, and 83.0% task accuracy versus 84.0% for CBM (Colamonaco et al., 17 Apr 2026). On a noisy-label version of ColorMNIST+, model editing by dropping or relabeling 30% misaligned prototypes improves test concept accuracy from “≈93%” to “≈97%” (Colamonaco et al., 17 Apr 2026).

LCBM focuses on concept localization as well as classification (Jeon et al., 20 Aug 2025). On CUB-200-2011, the reported numbers are: accuracy 75.2% versus 76.5% for VLG-CBM and 73.2% for LfCBM; explanation precision@10 of 76.6% versus 68.2% and 44.7%; inclusion 72.4 versus 65.3 and 53.8; mIoU 29.7 versus 28.3 and 22.7; REP 51.2 versus 37.6 and 30.6; and deletion ratio/difference 79.3%/0.177 versus baselines “≈ 87–90%/0.086–0.11” (Jeon et al., 20 Aug 2025). The conclusion given is that localization and explanation faithfulness improve while downstream accuracy is maintained (Jeon et al., 20 Aug 2025).

For concept-level debugging, ProtoPDebug reports on three settings (Bontempelli et al., 2022). On a synthetic confounded dataset, ProtoPDebug matches the clean baseline’s test $z=f_\theta(x)\in\mathbb{R}^Z$7, approximately 0.90, while IAIA-BL reaches approximately 0.76 even with 100% masks (Bontempelli et al., 2022). On a natural-confounder bird dataset, average test $z=f_\theta(x)\in\mathbb{R}^Z$8 improves from 0.56 to 0.69 and Attribution Precision rises from 0.68 to 0.90 after two debugging rounds (Bontempelli et al., 2022). On confounded COVID-19 chest X-rays, test $z=f_\theta(x)\in\mathbb{R}^Z$9 increases from approximately 0.26 to approximately 0.54 after four debugging rounds (Bontempelli et al., 2022).

6. Limitations, debates, and extensions

The literature presents PGCMs as interpretable and intervenable, but it also records clear limitations.

In iCSNs, three limitations are stated directly (Stammer et al., 2021). First, a reconstruction-only loss “may struggle when concepts are very fine-grained or when pixel-perfect fidelity is not aligned with concept fidelity.” Second, the temperature schedule $\phi_j=m_j(z)\in\mathbb{R}^Q$0 “requires careful tuning to avoid premature discretization or slot collapse.” Third, the Elementary Concept Reasoning benchmark is “synthetic and object-centric,” so real-world scenes with backgrounds, occlusions, or multiple objects require additional preprocessing such as “object proposals or slot-attention modules” (Stammer et al., 2021).

The 2023 multi-modal PGCM raises a different issue: prototypes may be too abstract for fine-grained communicative grounding (Noble et al., 2023). The paper identifies failure modes in description generation as “overly generic templates, omission of distinguishing attributes, and distributional shift from human-written descriptions,” and argues that there is “a clear mismatch between intrinsic NLG metrics (BLEU, CIDEr) and task-based communicative success” (Noble et al., 2023). This is an explicit caution against equating fluent prototype-conditioned generation with effective concept communication.

Prototype-based concept predictors also inherit annotation and representation trade-offs. Concept-Guided Prototype Networks require concept annotations at train time, note that prototype projection ties prototypes to actual patches and “may not capture abstract shape variations,” and describe diffusion-based concept-pure image generation as “computationally intensive” (Carballo-Castro et al., 2024). The 2026 PGCM for concept alignment states a “capacity vs. interpretability trade-off”: more prototypes cover more nuance but increase cognitive load, and extending the framework to “continuous or relational concepts (beyond binary flags)” remains future work (Colamonaco et al., 17 Apr 2026). LCBM likewise depends on a concept vocabulary and uses foundation-model guidance to localize concepts, which suggests a hybrid regime rather than fully annotation-free concept grounding (Jeon et al., 20 Aug 2025).

A further extension appears in one-shot concept learning with Abstracted Gaussian Prototypes (Zou et al., 2024). There, a concept is represented by a GMM-derived set of topological subparts whose parameters support both one-shot classification and generation. The framework is described as “uniquely low in theoretical and computational complexity” and “completely standalone,” but it is also stated to be “not state-of-the-art” in classification accuracy and “restricted to 2-D binary stroke images” (Zou et al., 2024). This suggests that prototype grounding can also be formulated generatively without large-scale pre-training, though current evidence is domain-specific.

Taken together, these limitations indicate that PGCMs do not solve concept alignment or interpretability in a single step. Rather, they shift the problem into a form in which concept semantics are grounded in prototypes, and therefore become inspectable, localizable, and, in several of the reported systems, directly modifiable.