Polysemanticity Index (PSI) Explained

• Polysemanticity Index (PSI) is a family of operational measures that quantifies whether a representation conflates multiple meanings or concepts.
• It employs diverse methods, including radial-profile fitting, surrounding uniformity tests, and entropy-based measures, to detect semantic ambiguity.
• PSI is applied in lexical ambiguity detection, neural and visual interpretability, and mechanistic analysis to refine our understanding of complex representations.

Polysemanticity Index (PSI) denotes a family of operational measures for quantifying whether a representation conflates multiple meanings or concepts. The term is not standardized across subfields. In lexical ambiguity detection, PSI has been defined as the residual error of the best single-peaked fit to a radial profile of contextual terms (Sproat et al., 2019). In distributional semantics, a closely related detector uses surrounding uniformity to test whether a single embedding vector is consistent with multiple senses (Oomoto et al., 2017). In explainability and mechanistic interpretability, PSI-like quantities include a normalized Shannon entropy over concept atoms for visual concepts (Yu et al., 19 Mar 2025), an average pairwise cosine similarity among multiple textual descriptions of a feature (Kopf et al., 18 Jun 2025), and a null-calibrated product of geometric separability, class-label alignment, and open-vocabulary distinctness for CNN neurons (Gupta et al., 23 Aug 2025). A later critique argues that some purported polysemanticity in LLMs is substantially attributable to lexical identity confounds, and proposes a lexically adjusted PSI (Hou et al., 1 Apr 2026).

1. Scope, sign conventions, and conceptual role

Across the cited literature, PSI serves a common purpose—measuring whether one representational unit corresponds to one meaning or several—but the measured object, the mathematical construction, and even the score direction differ. Some formulations treat a high value as evidence for stronger polysemanticity, whereas PRISM’s score increases with semantic coherence and therefore indicates greater monosemanticity (Kopf et al., 18 Jun 2025). This variation is central to interpreting published results.

Formulation	Core quantity	High value indicates
Lexical PSI	Residual sum-of-squared error of the best unimodal fit to a radial profile	More ambiguity/polysemy
Surrounding uniformity test	Collinearity of a word vector and its nearest neighbors	More uniformity; lower values suggest polysemy
Concept Polysemanticity Entropy	Normalized Shannon entropy over concept-atom probabilities	More concept uncertainty/polysemanticity
PRISM PSI	Mean pairwise cosine similarity among description embeddings	More semantic coherence/monosemanticity
Null-calibrated CNN PSI	Product $\hat S \times \hat Q \times \hat D$	Stronger evidence of a polysemantic neuron

The conceptual role also differs by workflow. In lexical settings, PSI is an ambiguity score over words in corpora. In embedding-based word analysis, surrounding uniformity is used as a fast detector to decide which tokens merit more expensive sense analysis. In visual and neural interpretability, PSI-like quantities are used either to quantify uncertainty in a concept representation or to rank neurons and features for further investigation (Oomoto et al., 2017, Yu et al., 19 Mar 2025, Gupta et al., 23 Aug 2025).

2. Radial-profile PSI for lexical ambiguity

In “Automatic Ambiguity Detection” (Sproat et al., 2019), the PSI of a target word $w$ is the residual sum-of-squared errors obtained by fitting the word’s radial-distance profile with the best possible single-peaked curve. If the full circle is divided into $n$ angular bins and $x_1,\dots,x_n$ are the smoothed total radial distances in those bins, then

$$\mathrm{PSI}(w) = \min_{1 \le i \le n} \left[ \sum_{j=1}^{i} (\lambda_j - x_j)^2 + \sum_{j=i}^{n} (\mu_j - x_j)^2 \right],$$

where $\lambda_1,\dots,\lambda_i$ is the isotonic least-squares fit to $x_1,\dots,x_i$, and $\mu_i,\dots,\mu_n$ is the antitonic least-squares fit to $x_i,\dots,x_n$.

The algorithm begins by collecting contextual terms around each occurrence of $w$. With context-window size $w$0—typically $w$1—all tokens $w$2 in the $w$3 words immediately to the left and right are gathered. A term enters the candidate set $w$4 if it satisfies $w$5, with $w$6 typically $w$7, and $w$8, with threshold typically $w$9. Pairwise distances among candidate terms are then computed using

$n$0

Classical Metric Multidimensional Scaling is applied to this distance matrix to obtain two-dimensional coordinates $n$1.

Those coordinates are converted to radial form via

$n$2

with $n$3 mapped into $n$4 by the sign of $n$5. Angles are quantized into $n$6 bins; in the paper, $n$7, so each bin is $n$8 wide. For each bin, the summed radius is accumulated, optionally with light $n$9-bin smoothing, producing the profile $x_1,\dots,x_n$0. PSI is then the minimum single-peak fit error across all possible peak locations.

The rationale is geometric. A largely unambiguous term produces a single broad hump in the radial histogram, so the unimodal fit incurs little error. A polysemous term yields multiple peaks at different angles, and the enforced single-lobe fit incurs large residuals. The paper’s examples make this explicit: “bass” yields two tight clusters, one musical and one biological, with $x_1,\dots,x_n$1, whereas “oxidation” yields a single coherent chemistry cloud with $x_1,\dots,x_n$2 (Sproat et al., 2019).

The reported experimental setting used a 10 million-word Grolier’s Encyclopedia corpus and about 9,000 lower-case words with $x_1,\dots,x_n$3. Computing PSI for the full list took about 500 CPU-hours on an SGI Origin 200. Among the 25 highest-scoring words were bill ($x_1,\dots,x_n$4), mild ($x_1,\dots,x_n$5), seasons ($x_1,\dots,x_n$6), inherited ($x_1,\dots,x_n$7), moderate ($x_1,\dots,x_n$8), bass ($x_1,\dots,x_n$9), and tip ($$\mathrm{PSI}(w) = \min_{1 \le i \le n} \left[ \sum_{j=1}^{i} (\lambda_j - x_j)^2 + \sum_{j=i}^{n} (\mu_j - x_j)^2 \right],$$0). Most matched distinct WordNet entries, though some such as inherited, aromatic, and r were not listed in WordNet, indicating that the procedure can surface unexpected ambiguity (Sproat et al., 2019).

3. Surrounding-uniformity as a fast polysemy detector for word embeddings

The method in “Polysemy Detection in Distributed Representation of Word Sense” (Oomoto et al., 2017) addresses a different question: given a single vector embedding $$\mathrm{PSI}(w) = \min_{1 \le i \le n} \left[ \sum_{j=1}^{i} (\lambda_j - x_j)^2 + \sum_{j=i}^{n} (\mu_j - x_j)^2 \right],$$1, determine whether it conflates more than one sense. The central quantity is the “surrounding uniformity” (SU), which measures how collinear a word vector is with its nearest neighbors. Let $$\mathrm{PSI}(w) = \min_{1 \le i \le n} \left[ \sum_{j=1}^{i} (\lambda_j - x_j)^2 + \sum_{j=i}^{n} (\mu_j - x_j)^2 \right],$$2 be the $$\mathrm{PSI}(w) = \min_{1 \le i \le n} \left[ \sum_{j=1}^{i} (\lambda_j - x_j)^2 + \sum_{j=i}^{n} (\mu_j - x_j)^2 \right],$$3 nearest-neighbor embeddings of $$\mathrm{PSI}(w) = \min_{1 \le i \le n} \left[ \sum_{j=1}^{i} (\lambda_j - x_j)^2 + \sum_{j=i}^{n} (\mu_j - x_j)^2 \right],$$4 by smallest angle. Define

$$\mathrm{PSI}(w) = \min_{1 \le i \le n} \left[ \sum_{j=1}^{i} (\lambda_j - x_j)^2 + \sum_{j=i}^{n} (\mu_j - x_j)^2 \right],$$5

and

$$\mathrm{PSI}(w) = \min_{1 \le i \le n} \left[ \sum_{j=1}^{i} (\lambda_j - x_j)^2 + \sum_{j=i}^{n} (\mu_j - x_j)^2 \right],$$6

If all vectors point in the same direction, $$\mathrm{PSI}(w) = \min_{1 \le i \le n} \left[ \sum_{j=1}^{i} (\lambda_j - x_j)^2 + \sum_{j=i}^{n} (\mu_j - x_j)^2 \right],$$7; if they fan out widely, $$\mathrm{PSI}(w) = \min_{1 \le i \le n} \left[ \sum_{j=1}^{i} (\lambda_j - x_j)^2 + \sum_{j=i}^{n} (\mu_j - x_j)^2 \right],$$8 is much smaller.

The test compares $$\mathrm{PSI}(w) = \min_{1 \le i \le n} \left[ \sum_{j=1}^{i} (\lambda_j - x_j)^2 + \sum_{j=i}^{n} (\mu_j - x_j)^2 \right],$$9 with the SU values of its neighbors. If

$\lambda_1,\dots,\lambda_i$0

then $\lambda_1,\dots,\lambda_i$1 is declared polysemic when

$\lambda_1,\dots,\lambda_i$2

with $\lambda_1,\dots,\lambda_i$3, yielding a $\lambda_1,\dots,\lambda_i$4 outlier rule.

The algorithmic recipe is explicit. A corpus is lowercased, word2vec is trained with 200 dimensions, default window, and skip-gram, and the top limit words by frequency are retained as “stable words.” For each stable word, the top $\lambda_1,\dots,\lambda_i$5 nearest stable neighbors are fetched by cosine or angle, SU is computed for the target and each neighbor, and the outlier rule is applied. If any neighbor’s SU is undefined because it lacks $\lambda_1,\dots,\lambda_i$6 neighbors, the target is skipped. The paper highlights limit, $\lambda_1,\dots,\lambda_i$7, and $\lambda_1,\dots,\lambda_i$8 as the only tuning knobs, and gives complexity per word as naive $\lambda_1,\dots,\lambda_i$9 for k-NN search plus $x_1,\dots,x_i$0 for SU computation, with overall $x_1,\dots,x_i$1 (Oomoto et al., 2017).

The worked example is “may,” whose neighbors are can ($x_1,\dots,x_i$2), should ($x_1,\dots,x_i$3), might ($x_1,\dots,x_i$4), and will ($x_1,\dots,x_i$5). The neighbor mean is $x_1,\dots,x_i$6, the sample standard deviation is $x_1,\dots,x_i$7, and the threshold is $x_1,\dots,x_i$8. Since $x_1,\dots,x_i$9, the word is flagged as polysemic (Oomoto et al., 2017).

On the top 1,000 stable words, 127 were undetermined, 33 passed the polysemy test, and 840 were labeled monosemic. A human evaluation on 24 words produced the confusion matrix with 19 mono/mono, 1 mono/poly, 1 poly/mono, and 3 poly/poly, and a $\mu_i,\dots,\mu_n$0 test with Yates’ correction showed significance at $\mu_i,\dots,\mu_n$1 (Oomoto et al., 2017).

The caveats are substantial. Lowercasing can artificially introduce polysemy, as with may as month versus modal. Rare senses may be missed, as in march as verb or august as adjective. The choice $\mu_i,\dots,\mu_n$2 is minimal for a statistical test; larger $\mu_i,\dots,\mu_n$3 may stabilize estimates but risks admitting weakly related neighbors. SU also correlates with part-of-speech, with prepositions having low SU and proper nouns high SU, so absolute thresholds can mix POS effects with polysemy (Oomoto et al., 2017).

4. Entropy-based PSI in visual concept explainability

In “CoE: Chain-of-Explanation via Automatic Visual Concept Circuit Description and Polysemanticity Quantification” (Yu et al., 19 Mar 2025), the general notion of a Polysemanticity Index is instantiated as Concept Polysemanticity Entropy (CPE), a normalized Shannon entropy over concept atoms associated with a visual concept. A single visual concept $\mu_i,\dots,\mu_n$4 is represented by $\mu_i,\dots,\mu_n$5 image patches. A disentanglement procedure $\mu_i,\dots,\mu_n$6 yields concept atoms, which are clustered into a final atom set

$\mu_i,\dots,\mu_n$7

With $\mu_i,\dots,\mu_n$8 atoms proposed per patch before clustering and $\mu_i,\dots,\mu_n$9 patches per concept, padding is introduced as

$x_i,\dots,x_n$0

and the padded probability of atom $x_i,\dots,x_n$1 is

$x_i,\dots,x_n$2

where $x_i,\dots,x_n$3 is the total count of atom $x_i,\dots,x_n$4 across patches.

The entropy score is

$x_i,\dots,x_n$5

Layer- and model-level averages are then

$x_i,\dots,x_n$6

The interpretation is distributional. The term $x_i,\dots,x_n$7 turns a visual concept into a discrete probability distribution over semantic atoms. The Shannon entropy measures the uncertainty of that distribution. A broad, flat distribution yields high entropy and thus high polysemanticity; a sharply peaked distribution yields low entropy and near-monosemanticity. The normalization by $x_i,\dots,x_n$8 makes the score comparable across concepts with different atom inventories (Yu et al., 19 Mar 2025).

The workflow has five stages: disentangle, cluster, count, pad, and compute. The padding heuristic is introduced so that a concept with very few clustered atoms does not appear maximally polysemantic merely because the support size is too small. The paper explicitly contrasts this with a naive PSI that would simply count the number of atoms $x_i,\dots,x_n$9; CPE instead weights atoms by empirical frequency, aggregates them via Shannon entropy, and normalizes to $w$0 (Yu et al., 19 Mar 2025).

The empirical illustrations are reported on ResNet152. In Stage 4, channel 75 had $w$1, described as moderately polysemantic, whereas channel 697 scored $w$2, described as very polysemantic. Stage 3 typically had the highest averaged CPE, while Stages 1 and 4 had lower CPE. The paper also reports that deeper or more complex networks tend to have higher global $w$3, that human agreement with CPE ordering on 300 VC-pairs was 75%, and that no hard threshold is prescribed, though in practice $w$4 is regarded as strongly polysemantic and $w$5 as nearly monosemantic (Yu et al., 19 Mar 2025).

The reported caveats are methodologically important. CPE depends on disentanglement quality from the LVLM and clustering quality from the NLI model. The padding heuristic is explicitly described as ad hoc. Shannon entropy also treats all atom differences equally, even when two atoms are semantically close. Suggested extensions include semantic-distance-weighted entropy, Multi-Information decompositions, conditional CPE tied to downstream relevance, and ensemble PSI across different concept-extraction methods (Yu et al., 19 Mar 2025).

5. Feature-level PSI in mechanistic interpretability

PRISM, introduced in “Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework” (Kopf et al., 18 Jun 2025), defines PSI directly from multiple textual descriptions of a feature. Let feature $w$6 have $w$7 descriptions $w$8, and let a sentence encoder $w$9 produce embeddings $w$00. PRISM computes all pairwise cosine similarities

$w$01

and defines

$w$02

Here, the sign convention reverses relative to entropy-based or residual-based scores: high PSI means the descriptions are semantically close and the feature is more monosemantic; low PSI means the descriptions are diverse and the feature is more polysemantic.

The score depends only on $w$03 and the sentence encoder. The default is $w$04, and the paper uses gte-Qwen2-1.5B-instruct with $w$05. No additional thresholds or overlap-handling steps are required at scoring time. In the reported case studies, a GPT-2 XL neuron with descriptions such as “personal experiences,” “US presidents,” “special occasions,” and “encryption” had $w$06 and a human polysemanticity rating of approximately $w$07, whereas a monosemantic neuron with labels centered on “units of time” had $w$08 and a human rating of approximately $w$09. Random-description and random-sentence sanity checks produced uniformly low PSI values, around $w$10–$w$11 and approximately $w$12 respectively, and a small human annotation study reported Pearson correlation greater than $w$13 between PSI and human-rated coherence (Kopf et al., 18 Jun 2025).

A more explicitly calibrated neuron-level PSI appears in “Disentangling Polysemantic Neurons with a Null-Calibrated Polysemanticity Index and Causal Patch Interventions” (Gupta et al., 23 Aug 2025). For a CNN channel $w$14, the top $w$15 activating image patches are embedded with CLIP into $w$16, with $w$17. Three raw components are computed. Geometric separability is

$w$18

where $w$19 is the average silhouette score for K-means clustering with cosine distance. Class-label alignment is normalized mutual information,

$w$20

with

$w$21

Open-vocabulary distinctness uses cluster prototypes

$w$22

top-2 CLIP text similarities, purity gaps

$w$23

and

$w$24

Each raw component is then null-calibrated. For $w$25, a null distribution is generated with $w$26 samples, a mean $w$27 and standard deviation $w$28 are estimated, and the calibrated score is

$w$29

The final PSI is

$w$30

This construction is explicitly multi-evidence. A high score requires that the top-activating patches cluster geometrically, align with labeled categories, and remain semantically distinct under an open-vocabulary text head. On ResNet-50 with Tiny-ImageNet, Layer 4 showed AUROC $w$31 against null for full PSI, compared with $w$32 for $w$33 alone, $w$34 for $w$35, and $w$36 for $w$37; Layer 3 showed the same ordering with PSI $w$38, $w$39 $w$40, $w$41 $w$42, and $w$43 $w$44. Layer 4 neurons had substantially higher PSI than Layer 3, with median Layer 4 exceeding the 75th percentile of Layer 3 and a Kolmogorov–Smirnov test yielding $w$45. A high-PSI example, channel $w$46 in Layer 4, split approximately $w$47 versus $w$48 into “vertical/column-like structures” and “foliage/texture” prototypes, whereas a low-PSI channel $w$49 was dominated by dog faces and had low $w$50 (Gupta et al., 23 Aug 2025).

The same paper adds causal validation by patch-swap interventions. For 10 high-PSI neurons in Layer 4, aligned patch replacements increased activation by mean $w$51, whereas non-aligned, random, shuffled-position, and ablate-elsewhere controls yielded $w$52, with shuffled-position producing approximately zero. Paired $w$53-tests gave $w$54 for aligned versus each control (Gupta et al., 23 Aug 2025). This does not redefine PSI mathematically, but it provides causal evidence that the discovered prototypes are activation-driving rather than merely correlational.

6. Confounds, limitations, and methodological consequences

A major critique of PSI-style superposition measurements appears in “Polysemanticity or Polysemy? Lexical Identity Confounds Superposition Metrics” (Hou et al., 1 Apr 2026). The paper argues that conventional measures often conflate lexical identity with semantic multiplicity. In the classical setup, a raw neuron-level PSI can be written as

$w$55

where $w$56 measures overlap across different senses of the same word, $w$57 is a different-word different-sense baseline, and $w$58 is the same-word same-sense upper bound. The critique is that the polysemy condition still shares word form, so the score can be inflated by lexical overlap.

To isolate this effect, the paper introduces a 2×2 design with SL, PS, SYN, and CL conditions and defines

$w$59

along with

$w$60

A lexically adjusted score is then proposed: $w$61 where $w$62 if neuron $w$63 is a form detector and $w$64 is the layer-specific expected lexical inflation.

Across nine Transformer families, the paper reports lexical dominance at every layer in every model: $w$65 with Wilcoxon $w$66 under Holm-Bonferroni correction. Lexical contribution ratios start very high in early layers, for example $w$67–$w$68 at layer $w$69, decline with depth to $w$70–$w$71 in mid-to-late layers, but never fall below zero. In sparse autoencoders, $w$72–$w$73 of active features conflate multiple senses of the same word. On GPT-2, the correction reclassifies an extra $w$74–$w$75 of top-polysemantic neurons as purely lexical, of which $w$76–$w$77 are confirmed form detectors with $w$78. Filtering out lexical-only activations improves downstream tasks: for CoarseWSD-20, top sense-selective neurons yield $w$79 accuracy versus $w$80 for sense-blind neurons, and in ROME editing selectivity rises from $w$81 to $w$82 with $w$83 (Hou et al., 1 Apr 2026).

Other limitations are formulation-specific. The surrounding-uniformity test is sensitive to embedding method, lowercasing, and part-of-speech, and may miss rare senses (Oomoto et al., 2017). CPE inherits biases from LVLM-based atom extraction and NLI-based clustering, and its padding rule is ad hoc (Yu et al., 19 Mar 2025). PRISM’s score depends on the number of descriptions and the sentence encoder (Kopf et al., 18 Jun 2025). The null-calibrated CNN PSI depends on the text encoder and prompt set, and the paper reports only moderate correlation, $w$84, when swapping CLIP text heads for raw $w$85 scores (Gupta et al., 23 Aug 2025).

Taken together, these results suggest that PSI is best understood as a task-specific operational statistic rather than a single canonical property. In lexical ambiguity detection, the central question is whether contextual geometry departs from unimodality. In word embeddings, the question is whether a target vector and its nearest neighbors fail to form a tight cone. In explainability, the question may be uncertainty over concept atoms, dispersion among textual descriptions, or null-calibrated evidence that top activations decompose into semantically distinct prototypes. The practical implication is that PSI values are only comparable within a shared definition, preprocessing pipeline, and sign convention.