Topological Polysemy Score Analysis

• Topological polysemy score is a continuous measure that quantifies a word's multiple meanings by analyzing the geometric and topological structure of its embedding space.
• It employs persistent homology and multiresolution grids to extract semantic diversity from both static and contextual word representations.
• Empirical results show strong correlations with sense inventories such as WordNet and Oxford, outperforming traditional clustering approaches.

The topological polysemy score refers to a family of computational methods that quantify the polysemy of a word—i.e., the number of distinct senses or meanings the word attests—by exploiting the topological or geometric structure of its representation in semantic space. These approaches, grounded in either geometric properties of contextual embeddings or the algebraic topology of static embedding neighborhoods, yield continuous, unsupervised measures that correlate with classic sense inventories such as WordNet and Oxford. The topological polysemy paradigm stands in contrast to traditional clustering or frequency-based methods, leveraging advances in both topological data analysis and representation learning.

1. Geometric and Topological Foundations

Central to topological polysemy scoring is the hypothesis that semantic representations of words—whether as static embedding vectors or contextualized embeddings across usages—organize non-uniformly in high-dimensional space. The manifold hypothesis posits that word vectors cluster on a low-dimensional submanifold within ambient space, but polysemous words violate "clean" manifold structure, becoming singularities or "pinch points" due to multiple senses being geometrically juxtaposed (Jakubowski et al., 2020). In the contextual-embedding paradigm, a word is represented by a cloud of vectors, each derived from a different sentential context, whose geometric spread reflects the diversity of usage and thus semantic multiplicity (Xypolopoulos et al., 2020).

2. Persistent Homology and the TPS_n Score

The "TPS_n(w)" score is rigorously defined via persistent homology in a neighborhood of the embedding space. For a target word $w$ with unit-norm static embedding $v_w\in\mathbb{R}^n$, one constructs the punctured, recentered, and re-normalized neighborhood $\bar N_n(w) = \{(v-v_w)/\|v-v_w\|_2: v\in N_n(w)\}$, where $N_n(w)$ denotes the $n$ nearest neighbors of $w$. The Vietoris–Rips filtration is applied to this cloud, and the 0th degree persistent homology (connected component lifespans) is computed, yielding a persistence diagram $D_0(w)$. The topological polysemy score is the sum of death times:

$\text{TPS}_n(w) = \sum_{(b, d) \in D_0(w)} (d-b)$

Large values indicate that multiple clusters in the neighborhood persist for wide parameter ranges, reflecting diverse semantic neighborhoods and thus higher polysemy (Jakubowski et al., 2020).

Empirical validation demonstrates that $\text{TPS}_n(w)$ correlates positively with human-annotated sense counts. In SemEval-2010 datasets, $\text{TPS}_{50}(w)$ achieves Pearson correlation ρ ≈ 0.424 with gold sense annotations, outperforming clustering-based intrinsic dimension methods (ρ ≈ 0.31), and remains uncorrelated with word frequency (ρ ≈ −0.006), indicating it isolates semantic, not merely distributional, multiplicity.

3. Multiresolution Grid (“Pyramid”) Scoring with Contextual Embeddings

The multiresolution grid method provides a topological measure using contextualized embeddings. Given $n$ occurrences of a word $w$, each represented as a $D$-dimensional PCA-reduced vector $x'_i$, a hierarchy of $L$ axis-aligned grids with bin counts $2^l$ per dimension is constructed. For each level $l=1...L$, the coverage is:

$$\text{coverage}_w^l = \frac{\# \text{distinct grid cells occupied at level } l}{(2^l)^D}$$

The weighted sum yields the final score:

$$\text{score}(w) = \sum_{l=1}^{L} \frac{\text{coverage}_w^l}{2^{L-l}}$$

Larger scores reflect greater "spread" of contexts, hypothesized to track the diversity of senses for $w$ (Xypolopoulos et al., 2020).

This method is parameterized chiefly by $D$ (after PCA) and $L$ (number of levels). Empirical results show strong rank correlations (up to ρ ≈ 0.8, NDCG ≈ 0.99) with sense-count resources such as WordNet, OntoNotes, and Wikipedia disambiguation categories.

4. Semantic Graph Models and Sense-node Count

A distinct topological approach models the lexicon as a two-part graph $G=(V,E)$ consisting of a large "reservoir" of context-free meaning nodes and word-specific subgraphs (Fumarola, 2016). Highly polysemous (short) words are represented by $k_w$ "sense-nodes," each linked to distinct reservoir nodes, while monosemous (long) words are encoded by a single node of higher degree. The topological polysemy parameter is $k_w$, the count of sense-nodes, or its normalized variant:

$\Pi(w) = \frac{k_w}{\max_v k_v} \in [0,1]$

Sequential-recall statistics in free-recall experiments can be mapped to $k_w$ by inversion against theoretical calibration curves, rendering $k_w$ recoverable from behavioral data.

5. Algorithmic Procedures and Computational Complexity

The computation of topological polysemy scores differs by method:

• Persistent Homology: Requires global normalization, nearest-neighbor search ($k$-NN, $O(|V| \log |V|)$), and persistent homology computation on $n$-point clouds. Recommended neighborhood size is $n ≈ 50$.
• Multiresolution Grids: Involves PCA ($O(\min\{N^2D_0, ND_0^2\})$ for $N$ total vectors), cell assignment for each occurrence ($O(nDL)$), and hash-set construction per level. Practical parameters are $D=2...5$, $L=6...12$, $n≈3000$ contexts per word.

Both approaches yield a scalar score per word, enabling large-vocabulary, unsupervised quantification with reasonable computational resources.

6. Evaluation Protocols and Empirical Results

Evaluation leverages vocabularies of frequent tokens with at least $n$ context sentences each, comparing topological scores against six human-constructed sense inventories: WordNet (all synsets, reduced synsets, domain labels), OntoNotes, Oxford Dictionary entries, and Wikipedia disambiguation categories (Xypolopoulos et al., 2020). Metrics include cosine similarity, Spearman's ρ, Kendall's τ, Precision@10%, NDCG, and RBO, all after normalizing scores to [0,100]. For the grid-based method, statistical significance is established at $p \leq 10^{-4}$ across all six human reference rankings.

For the persistent homology approach, peak correlation occurs for $n=50$ neighbors, and the method outperforms clustering and intrinsic-dimension baselines on established sense induction benchmarks (Jakubowski et al., 2020).

7. Illustrative Examples and Interpretability

In two-dimensional multiresolution grids, words with context vectors scattered widely across the plane (indicative of multiple senses) display high coverage and polysemy scores, while tightly clustered words exhibit low scores. Example word "count": grid cells correspond one-to-one with distinct senses ("Count Dracula," numerical act, and validity). In word embedding neighborhoods, persistent multi-cluster structure corresponds directly to semantic diversity. Topologically derived sense clusters can be inspected directly by sampling cells or clusters, yielding interpretable examples illustrating sense distinctions. For "metal," topological scoring allows retrieval of contexts corresponding to chemical elements, Olympic medals, and music genres (Xypolopoulos et al., 2020).

---

The topological polysemy score provides a mathematically principled approach to quantifying word sense multiplicity directly from data-driven representations, robustly validated against human sense inventories and applicable within unsupervised pipelines for word-sense induction and disambiguation (Xypolopoulos et al., 2020, Jakubowski et al., 2020, Fumarola, 2016).