Language-Guided Semantic Typicality Modeling

• Language-guided semantic typicality modeling is a framework that uses language as supervision to represent graded category membership rather than binary classification.
• It integrates prototype theory and information-theoretic approaches to define and score typicality via diverse formalisms, including cosine similarity and density models.
• Applications span image classification, caption evaluation, and anomaly detection by aligning language priors with perceptual and multimodal features.

Language-guided semantic typicality modeling is a family of approaches for representing the graded internal structure of semantic categories by using language-derived information as supervision, prior knowledge, or semantic geometry. In this literature, typicality is the degree to which an instance is a good example of its category rather than an all-or-none membership property. The resulting models range from prototype-centered distance functions in learned feature spaces to information-theoretic typical sets, document-frequency models over reference descriptions, cosine-based exemplar–prototype similarity, human-association-based lexical norms, and language-supervised density models for multimodal perception and anomaly detection (Pino et al., 2021, Feinglass et al., 2021, Vemuri et al., 2024, Rodriguez et al., 18 Mar 2026, Tang et al., 14 Sep 2025).

1. Conceptual and cognitive basis

The dominant conceptual basis is Prototype Theory. In that view, categories are heterogeneous, with central members, peripheral members, blurred boundaries, extensional non-equality, extensional non-discreteness, intensional non-equality, and intensional non-discreteness. The prototype is treated not merely as the most representative element but as a cognitive abstraction expressing central meaning, and family resemblance is described as a function of the frequency, or learned attribute relevance, and the distribution of attributes (Pino et al., 2021).

A second foundation is information theory. In that formulation, a sequence is typical if its features occupy a high-probability region of the underlying distribution; the typical set $A_\epsilon^{(n)}$ is used to motivate why only a small subset of all possible sequences count as typical realizations of language or semantics. This view underlies referenceless fluency measures, semantic comparison metrics, and transformer-internal probes that interpret typicality as closeness to a typical set representation rather than as membership in a rigid class (Feinglass et al., 2021).

Human studies operationalize typicality in several ways. Production norms treat typicality as $p(\text{exemplar} \mid \text{category})$; direct rating studies use graded goodness-of-example judgments; term-similarity datasets use relative triplet choices; and free word association studies use cue-conditioned human response probabilities. These paradigms converge on the claim that category structure is graded and measurable, but they also show that different tasks emphasize different aspects of typicality, such as taxonomic centrality, associative salience, or inductive strength (Vemuri et al., 2024, Linhardt et al., 10 Apr 2025, Rodriguez et al., 18 Mar 2026, Misra et al., 2021).

A recurrent theme is that language supplies semantic structure that can complement or exceed purely perceptual structure for typicality judgments. This is explicit in comparative work showing that LLMs align more closely with human typicality judgments than unimodal vision models, and in multimodal work where language-shaped priors stabilize degraded visual semantics (Vemuri et al., 2024, Park et al., 27 Apr 2026).

2. Formal representations and scoring functions

Several mathematically distinct formalisms are used to assign typicality scores. They differ in what counts as a category center, which features are privileged, and whether typicality is interpreted as centrality, likelihood, semantic support, or conformity to human response distributions.

Formal family	Core score	Interpretation
Prototype distance	$$\delta(o, P_i) = \sum_{j=1}^{m} |\omega_{ij}| \, |f_j - \mu_{ij}|$$	Smaller distance means greater typicality
Exemplar–prototype cosine	$T_M(i,c)=\cos(\mathbf{e}_i^M,\mathbf{p}_c^M)$	Higher cosine means exemplar is closer to category prototype
Association typicality	$$SS1(c,r)=\frac{S1(c,r) - \overline{S1(c,*)}}{SD\bigl(S1(c,*)\bigr)}$$	Higher standardized strength means more human-typical response
Flow-based action typicality	$\mathbf{S}^t_i = - \log p_{X_n}(\mathbf{F}^s_i)$	Higher score means greater atypicality relative to normal semantic distribution

In the Computational Prototype Model, the prototype of category $c_i$ is $P_i=(M_i,\Sigma_i,\Omega_i)$, where $M_i$ is the mean vector over typical objects, $\Sigma_i$ is the standard deviation vector, and $p(\text{exemplar} \mid \text{category})$0 is the relevance vector. Typicality is the weighted $p(\text{exemplar} \mid \text{category})$1 distance from an instance to $p(\text{exemplar} \mid \text{category})$2, family resemblance is a weighted $p(\text{exemplar} \mid \text{category})$3 distance between instances, and Chebyshev-style thresholds define fuzzy prototype boundaries in feature space (Pino et al., 2021).

In caption evaluation, typicality is split into semantic and linguistic components. Semantic typicality is implemented by SPARCS, which uses document frequencies of stemmed content-word concepts in reference captions to define adjusted precision, adjusted recall, and an $p(\text{exemplar} \mid \text{category})$4 score. Linguistic typicality is implemented through MIMA, a surrogate for distance from a typical set representation derived from mutual information in transformer attention, while SPURTS isolates stylistic typicality on stopword-stripped content words (Feinglass et al., 2021).

In contextual language modeling and lexical-semantic-network approaches, typicality is often represented as similarity between an exemplar and a category in a semantic space. One line uses cosine similarity between embeddings or cluster-derived multi-prototype representations; another uses Lin similarity in WordNet, $p(\text{exemplar} \mid \text{category})$5, with max-sim disambiguation over synset pairs (Renner et al., 2023, Vemuri et al., 2024).

A different statistical notion appears in matrix distributional semantics. There, typicality is the standard deviation of a permutation-invariant observable across word matrices, and the reported expectation values of many cubic and quartic observables show high gaussianity at levels between 90 to 99 percent. In that setting, typicality characterizes how tightly a structural statistic is shared across lexical items rather than how close a single item is to a category prototype (Ramgoolam et al., 2019).

3. Sources of language guidance

Language guidance enters these models through several distinct mechanisms. One mechanism uses pretrained LLMs as priors over well-formed language. MIMA treats attention distributions as joint distributions over token positions, computes normalized mutual information as information flow, and aggregates it into a scalar typicality score; SPARCS uses the empirical concept distribution of reference captions for an image as a semantic prior. In both cases, language defines what counts as typical without task-specific fine-tuning of the core LLM (Feinglass et al., 2021).

A second mechanism uses language to define or refine prototypes. In the Computational Prototype Model, the paper is vision-centered, but the formalism is explicitly defined in an abstract feature space and is described as portable to word, phrase, sentence, or multimodal embeddings. Typical examples can be selected by high classifier confidence or human ratings, and category-specific weights $p(\text{exemplar} \mid \text{category})$6 can be inherited from classifier parameters or attention-like relevance estimates (Pino et al., 2021).

A third mechanism is disambiguation. BERT-MPro clusters contextual token embeddings into multiple sense prototypes and defines a category–exemplar score by max-sim over prototype pairs. WordNet-based methods analogously compute similarities over all synset pairs and use the maximum to pick the most compatible sense pairing. FLAN-T5 feature verification introduces another form of language guidance: concept–feature applicability is queried in natural language, producing machine-verified feature norms over a 2600-dimensional feature space (Renner et al., 2023, Suresh et al., 2023).

A fourth mechanism is explicit cross-modal supervision. Language-Guided Semantic Cues align visual object features to CLIP text embeddings through a Semantic Cue Extractor and then reintegrate those cues into the visual pipeline through cross-attention. In zero-shot skeleton-based anomaly detection, CLIP text features align skeleton snippets with action semantics, and GPT-3.5-Turbo is used once offline to select typical normal and typical abnormal action classes whose distributions are then modeled with normalizing flows (Park et al., 27 Apr 2026, Tang et al., 14 Sep 2025).

Taken together, these mechanisms show that “language-guided” does not denote a single architecture. It can refer to prototype construction, sense selection, feature verification, reference-conditioned semantic distributions, language-shaped multimodal embedding spaces, or offline distillation of typical semantic priors.

4. Empirical findings across language, vision, and multimodal systems

Comparative concept-modeling results are especially informative. On 27 categories, LLMs aligned with human typicality judgments more strongly than vision models. The best LLM, all-MiniLM-L12-v2, achieved mean Spearman $p(\text{exemplar} \mid \text{category})$7, whereas the best pure vision model, ViT-h-14, reached $p(\text{exemplar} \mid \text{category})$8. A combined AlexNet + MiniLM model reached $p(\text{exemplar} \mid \text{category})$9, outperforming either modality alone. In the same study, CLIP ViT-L/14 showed $$\delta(o, P_i) = \sum_{j=1}^{m} |\omega_{ij}| \, |f_j - \mu_{ij}|$$0 for the text-only category-prototype approach, $$\delta(o, P_i) = \sum_{j=1}^{m} |\omega_{ij}| \, |f_j - \mu_{ij}|$$1 for the vision-only mean-prototype approach, and $$\delta(o, P_i) = \sum_{j=1}^{m} |\omega_{ij}| \, |f_j - \mu_{ij}|$$2 for cross-modality logits, indicating that language-guided image representations were more human-aligned than ImageNet-only visual features, but that raw image–text alignment scores were weak typicality signals (Vemuri et al., 2024).

Results on contextual-language-model probing show that probe choice and disambiguation matter substantially. BERT-MPro achieved mean category-wise Spearman correlations of 0.473, 0.451, and 0.386 on M&D-Young, M&D-Old, and Rosch, respectively. The WordNet Information Content model with disambiguation achieved 0.467, 0.456, and 0.448, and a simple standardized ensemble of BERT-MPro and WNIC increased performance to 0.547, 0.531, and 0.528. These values remain below human–human agreement, but they substantially exceed the performance of simpler, non-disambiguated probes (Renner et al., 2023).

Text-only predictive LLMs show weaker but non-zero typicality sensitivity. In taxonomic sentence verification, the reported Spearman correlations with human typicality ratings were in the range $$\delta(o, P_i) = \sum_{j=1}^{m} |\omega_{ij}| \, |f_j - \mu_{ij}|$$3, and in category-based induction they were in the range $$\delta(o, P_i) = \sum_{j=1}^{m} |\omega_{ij}| \, |f_j - \mu_{ij}|$$4, both with $$\delta(o, P_i) = \sum_{j=1}^{m} |\omega_{ij}| \, |f_j - \mu_{ij}|$$5. The same study notes that the 5-gram LLM was weak but competitive with some smaller transformers in the taxonomic task, which suggests that part of the apparent typicality signal can arise from shallow sequential statistics rather than richer conceptual structure (Misra et al., 2021).

Behavioral and lexical studies expose another dimension: the interaction of typicality with variability. In word association generation from SWOW-style cues, humans had $$\delta(o, P_i) = \sum_{j=1}^{m} |\omega_{ij}| \, |f_j - \mu_{ij}|$$6 and $$\delta(o, P_i) = \sum_{j=1}^{m} |\omega_{ij}| \, |f_j - \mu_{ij}|$$7, while Qwen at $$\delta(o, P_i) = \sum_{j=1}^{m} |\omega_{ij}| \, |f_j - \mu_{ij}|$$8 had $$\delta(o, P_i) = \sum_{j=1}^{m} |\omega_{ij}| \, |f_j - \mu_{ij}|$$9, $T_M(i,c)=\cos(\mathbf{e}_i^M,\mathbf{p}_c^M)$0, and $T_M(i,c)=\cos(\mathbf{e}_i^M,\mathbf{p}_c^M)$1 compared with the human $T_M(i,c)=\cos(\mathbf{e}_i^M,\mathbf{p}_c^M)$2. Mistral and Llama were more variable but less typical, and higher temperature increased variability while decreasing typicality. The paper interprets larger models as approximating a single “prototypical” participant rather than the full diversity of human lexical behavior (Rodriguez et al., 18 Mar 2026).

Related evidence from the Three Terms Task shows that even small decoder-only models can reach human-level representational alignment on relative semantic similarity, instruction tuning often improves representational agreement, layer-wise alignment is highly model dependent, and behavioral alignment matches representational alignment only for the largest evaluated models (Linhardt et al., 10 Apr 2025).

In vision-centered prototype modeling, typicality-aware structure also produces measurable gains. In the Computational Prototype Model, PS-Layer “pretrain” variants on CIFAR10 improved Top-1 accuracy from a baseline of about 69.5% to about 75.8–75.9%, and GSDP clustering on the first 20 ImageNet classes frequently produced Homogeneity, Completeness, V-measure, ARI, and AMI values at or above 0.94–0.98. The same work argues that semantic value or softmax score does not reliably reflect typicality, whereas prototypical distance correlates better with visual representativeness (Pino et al., 2021).

5. Architectures, tasks, and applications

Prototype-based architectures use typicality both as representation and as decision rule. In the Computational Prototype Model, a standard CNN is first trained conventionally, prototypes are built offline from highly confident examples, and classification can then be performed with a Prototypical Similarity Layer in which class probabilities are a softmax over negative prototypical distances. The same model supports image classification, a global semantic description through GSDP, and transfer learning from ImageNet prototypes to COCO (Pino et al., 2021).

In caption evaluation, language-guided semantic typicality becomes an explicit decomposition of quality. SMURF combines SPARCS for semantic typicality, SPURTS for style, and a grammatical outlier penalty derived from MIMA. On COCO system-level caption evaluation, SMURF achieved Pearson $T_M(i,c)=\cos(\mathbf{e}_i^M,\mathbf{p}_c^M)$3 for M1 and $T_M(i,c)=\cos(\mathbf{e}_i^M,\mathbf{p}_c^M)$4 for M2, while on CoNLL-2014 grammar evaluation DistilBERT-MIMA achieved Pearson $T_M(i,c)=\cos(\mathbf{e}_i^M,\mathbf{p}_c^M)$5 and Spearman $T_M(i,c)=\cos(\mathbf{e}_i^M,\mathbf{p}_c^M)$6. This use case treats typicality not as category membership but as a joint measure of semantic correctness, stylistic distinctiveness, and grammatical conformity to human distributions (Feinglass et al., 2021).

In multimodal grounding, language-guided semantic cues function as internal semantic priors. On crowded-scene grounding, adding LGSCs to ChatRex-7B improved AP$T_M(i,c)=\cos(\mathbf{e}_i^M,\mathbf{p}_c^M)$7 from 67.4 to 72.8 on CrowdHuman, from 22.7 to 24.7 on VisDrone, from 67.1 to 69.9 on UAVDT, and from 51.5 to 52.8 on RefDrone. In zero-shot transfer, the same approach improved AP$T_M(i,c)=\cos(\mathbf{e}_i^M,\mathbf{p}_c^M)$8 from 19.5 to 32.1 on CityPersons and from 38.3 to 59.3 on HazyDet. The key architectural pattern is training-time alignment of visual features to text embeddings, followed by inference-time use of language-shaped cues without text input (Park et al., 27 Apr 2026).

In zero-shot video anomaly detection, semantic typicality is operationalized as likelihood under a language-curated normal action distribution. The action-hints framework aligns skeleton snippets with CLIP text embeddings, uses GPT-3.5-Turbo to identify 20 most typical normal actions and 20 most typical abnormal actions from Kinetics-400, and models the corresponding feature distributions with normalizing flows. Without target-domain training data, the reported frame-level AUC-ROC values are 84.1 on ShanghaiTech, 74.5 on UBnormal, 62.1 on NWPU, and 62.7 on UCF-Crime (Tang et al., 14 Sep 2025).

Language-guided typicality modeling also supports lexical and conceptual resource construction. FLAN-T5 semantic feature verification queried 597,670 concept–feature pairs, produced a whole-matrix hit rate of 0.76, false alarm rate of 0.10, and $T_M(i,c)=\cos(\mathbf{e}_i^M,\mathbf{p}_c^M)$9, and yielded machine-verified norms that improved prediction of human similarity judgments for distally related items when combined with human feature norms. This application treats language not as a scoring surface but as a scalable semantic verifier that expands feature spaces beyond what humans explicitly listed (Suresh et al., 2023).

6. Limits, controversies, and open directions

A central limitation is measurement. Several frameworks do not begin with human-annotated typicality labels. The Computational Prototype Model approximates typicality using classifier confidence and builds prototypes from examples with predicted probability at least 98%; the authors explicitly note the absence of human typicality datasets and the off-line, non-end-to-end character of prototype construction. In comparative vision–language work, human typicality is attached to exemplar names rather than to specific images, which introduces noise when image embeddings are averaged to form exemplar vectors (Pino et al., 2021, Vemuri et al., 2024).

A second limitation concerns rarity, bias, and calibration. SPARCS penalizes rare but correct concepts because document frequency across references is its main signal; MIMA depends on language-model priors that may diverge from domain-specific language; and SMURF’s threshold $$SS1(c,r)=\frac{S1(c,r) - \overline{S1(c,*)}}{SD\bigl(S1(c,*)\bigr)}$$0 assumes approximate normality of human-score distributions. The same work notes that referenceless metrics may mitigate dataset-specific bias while still inheriting the broader biases of pretrained LLMs (Feinglass et al., 2021).

A third limitation is that text-only models remain incomplete models of human typicality. Direct LM probability probes yield only modest correlations with human ratings, simple n-gram models can explain a substantial portion of some effects, and triplet studies report that models often favor type or taxonomic similarity where humans sometimes favor association. Word-association experiments similarly show that larger models can become over-typical and under-variable, approximating a prototypical agent rather than a population of speakers (Misra et al., 2021, Linhardt et al., 10 Apr 2025, Rodriguez et al., 18 Mar 2026).

A fourth limitation is contextual scope. Skeleton-only anomaly detection suppresses background and appearance variation but also ignores richer scene–behavior relations; CLIP-aligned visual priors can mislead in unusual contexts; and symbolic lexical resources such as WordNet contribute structure but remain incomplete and static. The literature therefore points toward context-conditioned typicality, richer textual descriptions, human-supervised prototype learning, uncertainty-aware priors, hierarchical object–relation–scene typicality, and multimodal fusion mechanisms that go beyond simple concatenation or fixed thresholds (Park et al., 27 Apr 2026, Tang et al., 14 Sep 2025, Renner et al., 2023, Suresh et al., 2023).

Taken together, these results suggest that language-guided semantic typicality modeling is best understood not as a single method but as a design space. Its common commitment is that language can define, sharpen, or regularize semantic structure: by supplying prototype centers, selecting typical examples, disambiguating senses, verifying features, shaping multimodal manifolds, or calibrating distributions of normal and abnormal behavior. Its unresolved question is not whether language carries typicality information, but how that information should be combined with perception, human supervision, and context so that graded semantic structure remains both cognitively plausible and operationally robust.